Issues with exporting netflow from Cisco 3850

[micke_eri]

Hello!

We are trying to export netflow data from our Cisco 3850-24XS-switches to a lancope flow collector but seem to have some issues with this.

We are using software version 03.07.03E and the following netflow configuration

flow record IXNF01
match ipv4 tos
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface input
match flow direction
collect interface output
collect counter bytes long
collect counter packets long
collect timestamp absolute first
collect timestamp absolute last

!
flow exporter IXEXPORT01
destination X.X.X:X vrf Mgmt-vrf
source GigabitEthernet0/0
transport udp 2055
template data timeout 60

!
flow monitor IXMONITOR01
exporter IXEXPORT01
cache timeout active 60
record IXNF01

!

Interface te1/0/1
ip flow monitor IXMONITOR01 input

!

interface te1/0/17
ip flow monitor IXMONITOR01 input

With command show flow monitor IXMONITOR01 cache i see alot of records in the switch and with command show flow exporter IXEXPORT01 statistics i see the following.

Flow Exporter IXEXPORT01:
Packet send statistics (last cleared 4d19h ago):
Successfully sent: 4416646 (6000578554 bytes)

Client send statistics:
Client: Flow Monitor IXMONITOR01
Records added: 92397212
- sent: 92397212
Bytes added: 5081846660
- sent: 5081846660

However, our flow collector dont seem to receive any netflow data and in our firewalls i cant see any logrecords from the switches that are supposed to send netflow data. I dont see anything when i debug flow export in the switch either. We are using the management port and management VRF to export data, or we want to do that atleast.

We have almost the exact same netflow configuration in 4500X-switches and that works. They also export netflow data thru their management interface and VRF. The 4500X-switches are on the same management subnet as the 3850 and have the same firewall rules. We also have asr9001-routers exporting netflow data without any problems.

I'm thinking of creating a TAC-case but wanted to check here first to see if anyone has any ideas or if you are exporting netflow data from 3850-switches successfully.

[Michael Patterson]

Hello,

You may want to compare your configuration to this 3850 NetFlow Configuration page. If you still aren't receiving flows, can you position Wireshark to get a packet capture?  This will verify if it is sending flows.

If it is sending flows but, they still don't show up in Stealthwatch, send the packet capture to our support team and they will help you verify that the export is valid.  

MIke

[micke_eri]

Hello Michael!

I have used stealthwatch recommended configuration for 3850-switches. Comparing the record configuration in the link to mine i only find two differences, match datalink mac output and collect transport tcp flags.  The match datalink mac output command has changed to match datalink mac destination address input, output seem to be an unsupported match field for input direction. Can match on both source and dest mac address. Tried adding transport flags and datalink match statements without any change.

The exporter and monitor looks almost the same apart from what interface and VRF i'm using. 

Sure i can place a wireshark node on a span port in the management switch. I have already done a capture in the firewall(FW monitor) where the traffic needs to pass to go outside the management network and can't see the traffic that should pass there on it's way to the collector. I can't see any traffic between the switch and the collector at all but if i send a few ICMP-packets from the switch they show up in the firewall capture.

If i do a firewall capture for traffic between the 4500X-switch and the collector i see alot of traffic on port 2055.

I dont think i will learn any new information by doing another capture on the 3850 management interface.

I have also done a capture on the collector and can only see traffic from the 4 other nodes on the same network as the 3850-switches. 

[micke_eri]

We have done a packet capture and no traffic is seen from the management port of the switch on UDP port 2055 or to the collectors destination. So it seems like the switch isn't sending any netflow data at all.

I will be creating a TAC-case about this

[csp-reseau.rt1]

Hi,

We have exactly the same issue. 

Had you got a reply from the TAC?

[micke_eri]

Hey!

I have contacted the support with a problem description and a link to this post so i will hopefully get a response soon.

I'm on vacation at the moment but i have collagues that will assist the TAC with further troubleshooting. I will post the resolution as soon as i know and have the ability to do so.

[micke_eri]

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/3e/release_notes/rn-3dot7e-3850.html#pgfId-1029358

• You cannot configure NetFlow export using the Ethernet Management port (g0/0).

[csp-reseau.rt1]

Hi Micke,

Thanks for this reply.

It's the conclusion we just had today because it's working when we use as source an other interface of the swicth  and a VLAN interface IP@.

[micke_eri]

I think we might create a new management-VRF and use another interface for management, log and netflow traffic. We have asked Cisco if this restriction will be solved in the future since we would like to use the dedicated mgmt-interface for the above traffic.