08-17-2022 01:42 PM
Ok so this is going to be kinda of a book on Netflow and some very useful items I also have some issues with the setup. So lets jump right in. I have a Cisco 9500 and it is used as our core user switch. it connects to our data coure switch which connects all of our server and the ASA that we have. On any give switch we have 20 or so vlans and so forth. THe task was to create a net flow configuration on the core switch that would allow both ingress and egress traffic to our collector in this case that collector is solar winds. Solarwinds and support both version 5 and version 9 in my case I'm using version 9. I have read just about everything that I can on the netflow configurations thus far. So for the 9500 in order for each layer 3 port to have ingress and egress you have to configure two of each thing. Two flow records an input and an output record. Then you have to configure two exporters. But there is a got ya on this one each of the exporters have to have different ports configured for them. In my case because I'm using solarwinds i'm using 2055 and 9995. Then you have to have two flow monitors an input and an output but there is another i got ya on this one each flow monitor has to be configured with both the udp ports that you have configured. Then you simply apply the configuration to you layer three ports. Ha you thought that was all so here are the issues. First you can apply the config to any physical interface that you want ie gi1/0/1 and so forth. You can apply it to a port channel. Oh you that that was easy no its not you have to apply to only a layer three port channel but there is a another got ya your port members have to be layer 2 and your port channel has to be layer three and you cant have both configured at the same time so in my case sat my port channel is port-channel100 and name is core uplink it has to have the command no switchport and it has to be configured with an ip address ie 192.168.0.1 255.255.255.0 once it is configured in this way then you can assign your config to the port channel group. I wont go into the solarwinds config part but there are a lot of back end changes that have to be made in order for this config to work see the working config below ip addresses have been changed to protect the innocent.
flow record RECORD-IN
match flow direction
match interface input
match ipv4 destination address
match ipv4 protocol
match ipv4 source address
match ipv4 tos
match transport destination-port
match transport source-port
collect counter bytes long
collect counter packets long
collect interface output
collect transport tcp flags
!
!
flow record RECORD_OUT
match flow direction
match interface output
match ipv4 destination address
match ipv4 protocol
match ipv4 source address
match ipv4 tos
match transport destination-port
match transport source-port
collect counter bytes long
collect counter packets long
collect interface input
collect transport tcp flags
!
!
flow exporter Exporter_UDP_9995
destination 192.168.0.1
source Loopback0
transport udp 9995
!
!
flow exporter Exporter_UDP_2055
destination 192.168.0.1
source Loopback0
transport udp 2055
!
!
flow monitor MONITOR_IN
exporter Exporter_UDP_2055
exporter Exporter_UDP_9995
cache timeout inactive 10
cache timeout active 60
record RECORD-IN
!
!
flow monitor MONITOR_OUT
exporter Exporter_UDP_2055
exporter Exporter_UDP_9995
cache timeout inactive 10
cache timeout active 60
record RECORD_OUT
This is a config from a live environment and everything works on this piece as expected data is flowing into and out of the ports and its getting reported back to the collerctor. Below is the clean config with out all the extra junk and things you do not need to know but wish you did.
version 17.3
service timestamps debug datetime localtime
service timestamps log datetime localtime
service password-encryption
service sequence-numbers
service counters max age 10
service call-home
platform punt-keepalive disable-kernel-core
!
hostname Corp-9500-1
!
!
vrf definition Mgmt-vrf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
logging buffered 1000000
no logging console
logging cns-events notifications
!
aaa new-model
!
!
aaa authentication login local_auth local
!
!
!
!
!
!
aaa session-id common
clock timezone est -5 0
clock summer-time est recurring
switch 1 provision c9500-24y4c
switch 2 provision c9500-24y4c
boot system bootflash:packages.conf
stackwise-virtual
domain 1
!
ip multicast-routing
ip domain name notyours.net
!
!
!
login on-success log
!
!
!
!
!
!
!
vtp mode transparent
no device-tracking logging theft
!
flow record RECORD-IN
match flow direction
match interface input
match ipv4 destination address
match ipv4 protocol
match ipv4 source address
match ipv4 tos
match transport destination-port
match transport source-port
collect counter bytes long
collect counter packets long
collect interface output
collect transport tcp flags
!
!
flow record RECORD_OUT
match flow direction
match interface output
match ipv4 destination address
match ipv4 protocol
match ipv4 source address
match ipv4 tos
match transport destination-port
match transport source-port
collect counter bytes long
collect counter packets long
collect interface input
collect transport tcp flags
!
!
flow exporter Exporter_UDP_9995
destination 192.168.0.1
source Loopback0
transport udp 9995
!
!
flow exporter Exporter_UDP_2055
destination 192.168.0.1
source Loopback0
transport udp 2055
!
!
flow monitor MONITOR_IN
exporter Exporter_UDP_2055
exporter Exporter_UDP_9995
cache timeout inactive 10
cache timeout active 60
record RECORD-IN
!
!
flow monitor MONITOR_OUT
exporter Exporter_UDP_2055
exporter Exporter_UDP_9995
cache timeout inactive 10
cache timeout active 60
record RECORD_OUT
!
password encryption aes
!
!
!
license boot level network-advantage addon dna-advantage
!
!
diagnostic bootup level complete
!
spanning-tree mode rapid-pvst
spanning-tree extend system-id
spanning-tree backbonefast
memory free low-watermark processor 290409
!
redundancy
mode sso
!
!
!
!
!
!
transceiver type all
monitoring
!
vlan 2
name VoIP
!
vlan 3
name Data
!
vlan 60
name Printers
!
vlan 70
name Cameras
!
vlan 80
name Entrances
!
vlan 90
name Workstations
!
vlan 116
name Maint_Ext
!
vlan 117
name Vorne
!
vlan 118
name Scalehead
!
vlan 120
name Maint_NUB
!
vlan 130
name Power
!
vlan 160
name Clocks
!
vlan 190
name Vending
!
vlan 191
name HVAC
!
vlan 192
name Credit_Card
!
vlan 700
name Mgmt_VLAN
!
vlan 703
name AP+PTP
interface Loopback0
no ip address
!
interface Port-channel1
description Uplink to MDF
switchport mode trunk
!
interface Port-channel2
description Uplink to IT-UPS
switchport mode trunk
!
interface Port-channel3
description Uplink to Giftshop
switchport mode trunk
!
interface Port-channel12
description Uplink to Datacore
no switchport
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address x.x.x.x 255.255.255.252
ip pim dense-mode
!
interface GigabitEthernet0/0
vrf forwarding Mgmt-vrf
no ip address
negotiation auto
!
interface TwentyFiveGigE1/0/1
description Uplink to MDF
switchport mode trunk
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
channel-group 1 mode on
!
interface TwentyFiveGigE1/0/2
description Uplink to IT-UPS
switchport mode trunk
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
channel-group 2 mode on
!
interface TwentyFiveGigE1/0/3
description Uplink to Giftshop
switchport mode trunk
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
channel-group 3 mode on
interface TwentyFiveGigE1/0/12
description Uplink to Datacore
no switchport
no ip address
channel-group 12 mode on
interface TwentyFiveGigE1/0/24
stackwise-virtual dual-active-detection
description DAD link
!
interface HundredGigE1/0/25
!
interface HundredGigE1/0/26
!
interface HundredGigE1/0/27
stackwise-virtual link 1
description Stacking Cable 1
!
interface HundredGigE1/0/28
stackwise-virtual link 1
description Stacking Cable 2
!
interface TwentyFiveGigE2/0/1
description Uplink to MDF
switchport mode trunk
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
channel-group 1 mode on
!
interface TwentyFiveGigE2/0/2
description Uplink to IT-UPS
switchport mode trunk
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
channel-group 2 mode on
!
interface TwentyFiveGigE2/0/3
description Uplink to Giftshop
switchport mode trunk
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
channel-group 3 mode on
!
interface TwentyFiveGigE2/0/12
description Uplink to Datacore
no switchport
no ip address
channel-group 12 mode on
!
interface TwentyFiveGigE2/0/24
stackwise-virtual dual-active-detection
description DAD link
!
interface HundredGigE2/0/25
!
interface HundredGigE2/0/26
!
interface HundredGigE2/0/27
stackwise-virtual link 1
description Stacking Cable 1
!
interface HundredGigE2/0/28
stackwise-virtual link 1
description Stacking Cable 2
!
interface Vlan1
no ip address
shutdown
!
interface Vlan2
description VoIP
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address 192.50.123.2 255.255.248.0
no ip redirects
no ip unreachables
ip pim dense-mode
ntp broadcast
!
interface Vlan3
description Data VLAN
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address 192.50.123.5 255.255.0.0 secondary
ip address 192.50.123.7 255.255.0.0
ip helper-address 192.50.123.9
ip helper-address 192.50.123.11
no ip redirects
no ip unreachables
!
interface Vlan60
description Printers
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address 192.50.123.13 255.255.0.0
ip helper-address 192.50.123.15
no ip redirects
no ip unreachables
!
interface Vlan70
description Cameras
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address 192.50.123.17 255.255.0.0
no ip redirects
no ip unreachables
!
interface Vlan80
description Entrances
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address 192.50.123.19 255.255.0.0
ip helper-address 192.50.123.21
no ip redirects
no ip unreachables
!
interface Vlan90
description Workstations
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address 192.50.123.23 255.255.0.0
ip helper-address 192.50.123.25
no ip redirects
no ip unreachables
!
interface Vlan116
description Maint_Ext
ip address 192.50.123.27 255.255.0.0 secondary
ip address 192.50.123.229 255.255.0.0
!
interface Vlan117
description Vorne
ip address 192.50.123.31 255.255.0.0
ip helper-address 192.50.123.33
!
interface Vlan118
description Scalehead VLAN
ip address 192.50.123.35 255.255.255.0
no ip redirects
no ip unreachables
!
interface Vlan120
description Vlan 120
ip address 192.50.123.37 255.255.0.0 secondary
ip address 192.50.123.41 255.255.0.0
!
interface Vlan130
description Power
ip address 192.50.123.43 255.255.0.0
no ip redirects
no ip unreachables
!
interface Vlan160
description Clocks
ip address 192.50.123.45 255.255.0.0
ip helper-address 192.50.123.47
no ip redirects
no ip unreachables
!
interface Vlan190
description Vending
ip address 192.50.123.49 255.255.255.0
ip helper-address 192.50.123.51
no ip redirects
no ip unreachables
!
interface Vlan191
description HVAC
ip address 192.50.123.53 255.255.255.0
no ip redirects
no ip unreachables
!
interface Vlan192
description Credit Card Machine
ip address 192.50.123.55 255.255.255.0
ip helper-address 192.50.123.57
no ip redirects
no ip unreachables
!
interface Vlan700
description Mgmt VLAN
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
ip address 192.50.123.59 255.255.255.0
no ip redirects
no ip unreachables
!
interface Vlan703
description AP+PTP
ip address 192.50.123.61 255.255.255.0
ip helper-address 192.50.123.63
no ip redirects
no ip unreachables
!
router ospf 1
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.7.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.0.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.0.255 area 0
network x.x.x.x 0.0.0.255 area 0
network x.x.x.x 0.0.0.255 area 0
network x.x.x.x 0.0.0.255 area 0
network x.x.x.x 0.0.255.255 area 0
network x.x.x.x 0.0.0.3 area 0
!
ip default-gateway x.x.x.x
ip forward-protocol nd
no ip http server
ip http authentication local
no ip http secure-server
ip http secure-trustpoint TP-self-signed-2970070898
ip route 0.0.0.0 0.0.0.0 x.x.x.x
ip ssh source-interface Vlan700
ip ssh version 2
!
!
Alright there it is all nice and neat and cut and dried. So as I have said everything is working and sending traffic. What I cant get working is the Vlan's none of them are sending their data to the collector. Is that because the ports that are switched to the vlan already have the netflow config or is there additional Information that is needed in order to get them to start sending their data.
08-17-2022 11:28 PM
Hello,
I'll check your config, the one thing I did notice right away was the presence of this command:
--> ip default-gateway x.x.x.x
This is needed on layer 2 devices only, so you want to remove that line.
08-17-2022 11:45 PM
Hello,
I seem to recall that Flexible Netflow is not supported at all on SVIs on the 9500. Bridged Netflow is an option:
--> The device does not support tunnels and SVI interfaces; however Layer 2 and Layer 3 physical interfaces and VLAN configuration mode are supported.
08-18-2022 05:17 AM
So the question on that is do I have to configure that for each of the vlans or does that config work on all the vlan's
example
config t
vlan configuration 2
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
Does that work for all the vlan's or do I have to apply configuration as
vlan configuration 3
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
vlan configuration 90
ip flow monitor MONITOR_IN input
ip flow monitor MONITOR_OUT output
and then it will send the netflow traffic
And as far as the ip default gateway for my environment that line is required but thank you for the insight on that one line.
08-18-2022 07:48 AM - edited 08-18-2022 07:50 AM
Hello,
good question...I honestly have no idea. Does it work for one Vlan when you put this under the configuration for that one Vlan ?
The 'ip default-gateway' is only needed if your switch is a layer 2 only switch. Judging from your config, it is not (you are running OSPF, multiple SVIs, etc.). The line can cause unexpected problems with routing, so I suggest to remove it.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide