Netflow incorrect traffic

[AlexeyZ_86]

Hello!

I'm testing Netflow on Catalyst 6509. I've tried Netflow Analyzer and Scrutinizer. They show same results. So the problem is that Inbound traffic on Te5/4 and Te5/5 (upstreams with MPLS configured) is incorrect, about 50% less than real traffic. I beleive that it influences on outbound statistics of another interfaces on the switch. I've tried nde version 7, flow export version 9, ingress and egress together, but results were the same.

Here is some config from 6509:

ip flow-cache timeout inactive 15

ip flow-cache timeout active 1

mls aging long 64

mls aging normal 32

mls netflow interface

mls netflow usage notify 100 1000000

mls flow ip interface-full

mls nde sender version 7

ip flow-export source Loopback0

ip flow-export version 9

ip flow-export destination XXX 9996

snmp-server ifindex persist

On all vlan interfaces "ip flow ingress" is enabled.

[Don Jacob]

Hi,

The possible reason is that hardware switched flows (flows from PFC) are not being exported. I have seen such a case before, but could not find the solution. Check the outputs of the below commands and see if the number of flows shown are much lesser than expected.

sh ip cache flow

To display the status and the  statistics for NetFlow accounting data export, including the main cache  and all other enabled caches

show mls nde

To display information about the NetFlow Data Export (NDE) hardware-switched flow

Regards,

Don Thomas Jacob

ManageEngine NetFlow Analyzer

[AlexeyZ_86]

Hi!

I have problem on the Supervisor module. All interfaces on DFC and СFC cards have correct traffic in Collector.

[AlexeyZ_86]

I've installed DFC card with 10GE. Results are the same.

[Don Jacob]

Hi Alexey,

I think the issue is related to PFC where hardware switched flows are not being exported. You may need to verify with Cisco TAC or we will have to wait for someone who is aware of the issue to reply on why the PFC flows are not being exported.

Regards,

Don Thomas Jacob

ManageEngine NetFlow Analyzer

[jakewilson]

Hello Alexey,

I don't have a 6509 to test this on.  Will this command work on the switch:

ip flow ingress layer2-switched vlan (insert vlans X,Y,X) <---- this will enable flows for all bridged traffic

Also, we usually stick with ip flow-export version 5 and mls nde sender version 5.

Contact Joanne at Plixer if you need support.

Jake

Join NetFlow Developments on Linkedin

www.plixer.com

[AlexeyZ_86]

Hello Jake!

Thank you for responce.

My Te5/4 and 5/5 interfaces are L3.

!

interface TenGigabitEthernet5/5

dampening

mtu 9216

ip address 10.xxx 255.xxx

bandwidth 10000000

no ip address

no ip redirects

no ip proxy-arp

ip flow ingress

ip ospf message-digest-key 1 md5 7 03090B5C161F154168592D

ip ospf hello-interval 1

ip ospf dead-interval 10

mpls label protocol ldp

mpls ip

end

I've tried version 5 also with same results. Interesting that wrong inbound traffic is only on 10GE interfaces (both on supervisor or DFC card with 10GE). 1GE interfaces are okay. I'm still trying to find out the reason why 6509 sends not all flows to collectors.

And I tried to make same config as on 1GE interfaces. I made 10GE access port and interface vlan for it, but nothing changed

[AlexeyZ_86]

I've tried to turn on netflow on another two 6509, which have several 10GE connections. On some interfaces collector shows correct inbound traffic,  on another - no traffic at all. It seems it happens randomly.

[jakewilson]

I got to thinking about this.  Ages ago Michael Patterson wrote a blog on Overflow with the TCAM tables on this switch.  At the bottom of the post, a contributor named Roland Dobbins made an interesting series of comments that I think you should read.

"All these issues combine to make NetFlow on 6500/7600 with current and past hardware quite unreliable and generally operationally useless, IMHO."

Please read all of Roland's comment.  It could provide you with some valuable insight.

[AlexeyZ_86]

My TCAM table is utilyzed at about 40%. I have nothing to say about second issue about TCP flags.