Cat9500-24Q Performance Issues

[Dan Werntz]

I have a Catalyst 9500-24Q that is not passing traffic well.  It is connected to a Cat9300 and several UCS servers via 40G bidi transceivers.  I cannot get iperf to push more than 4 or 5Gbps port to port in the same VLAN on the switch.  This is far from line rate and is concerning me that the switch is defective.

I know it isn't the servers because if I connect the 40G links direct I can get about 20G of throughput.

Has anyone else experienced this or have some ideas for performance tuning?

[balaji.bandi]

how is your bare metal Linxu config for Port-channel ?  what Linux distribution.

For testing why not try Single interface both the side server and client, before you test using port-channel ?

[Dan Werntz]

I went ahead and tried your suggestion.  Two servers, single link each.  I get to about 8Gbps over the 40Gbps link.

Server A

FortyGigabitEthernet1/0/9 is up, line protocol is up (connected)
Hardware is Forty Gigabit Ethernet, address is 00bf.77b9.1b09 (bia 00bf.77b9.1b09)
MTU 9000 bytes, BW 40000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 40Gb/s, link type is auto, media type is QSFP 40G SR BD SFP
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:10, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 55490000 bits/sec, 4837 packets/sec
5 minute output rate 123622000 bits/sec, 10319 packets/sec
34645693 packets input, 46132997585 bytes, 0 no buffer
Received 58109 broadcasts (48403 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 48403 multicast, 0 pause input
0 input packets with dribble condition detected
51222959 packets output, 65304435891 bytes, 0 underruns
Output 673798 broadcasts (0 multicasts)
0 output errors, 0 collisions, 2 interface resets
23962 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Server B

FortyGigabitEthernet1/0/3 is up, line protocol is up (connected)
Hardware is Forty Gigabit Ethernet, address is 00bf.77b9.1b03 (bia 00bf.77b9.1b03)
MTU 9000 bytes, BW 40000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 40Gb/s, link type is auto, media type is QSFP 40G SR BD SFP
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:13, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 4000 bits/sec, 6 packets/sec
344145127 packets input, 519690344676 bytes, 0 no buffer
Received 5723 broadcasts (4971 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 4971 multicast, 2 pause input
0 input packets with dribble condition detected
50196463 packets output, 58863769269 bytes, 0 underruns
Output 62823 broadcasts (0 multicasts)
0 output errors, 0 collisions, 2 interface resets
3658 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Results

-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.10.100.222, port 49706
[ 5] local 10.10.100.249 port 5201 connected to 10.10.100.222 port 49707
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 982 MBytes 8.24 Gbits/sec
[ 5] 1.00-2.00 sec 972 MBytes 8.15 Gbits/sec
[ 5] 2.00-3.00 sec 972 MBytes 8.15 Gbits/sec
[ 5] 3.00-4.00 sec 973 MBytes 8.16 Gbits/sec
[ 5] 4.00-5.00 sec 974 MBytes 8.17 Gbits/sec
[ 5] 5.00-6.00 sec 971 MBytes 8.14 Gbits/sec
[ 5] 6.00-7.00 sec 972 MBytes 8.15 Gbits/sec
[ 5] 7.00-8.00 sec 970 MBytes 8.14 Gbits/sec
[ 5] 8.00-9.00 sec 973 MBytes 8.16 Gbits/sec
[ 5] 9.00-10.00 sec 976 MBytes 8.19 Gbits/sec
[ 5] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.00 sec 9.51 GBytes 8.16 Gbits/sec receiver

[balaji.bandi]

is that good or bad or worst ?

can you post on Linux side what NIC you have - it may be 10GB NIC or Linux has 40GB NIC both the side ?

[Dan Werntz]

This has been my best case result.  With Linux I'm getting 4 to 5 Gbps.  The most recent test with single interfaces was with Windows Server 2019 hosts, which makes it even more surprising that it has performed the best so far.  They are both showing 40Gbps nics.  

[Leo Laohoo]

The output provided is useless. 

The issue, as claimed, is when the servers are connected to the 9300 but the output are from the 9500.  I am requesting the output from the 9300.

Next, the output has been edited.  

[Dan Werntz]

No, that is not at all what my issue was stated as.  I have explained the servers are all connected to the 9500, all traffic is between servers on the same VLAN on the same switch.  How would I even connect servers and a 9500 with 40G to a 9300?  I have 2 ports to work with.  The detail of the 9300 was provided for topology completion's sake.  I can give you an entire "show run" of the 9300 if you really think it will help.

Not sure what you mean about my command output being edited.  I did "show interface fo1/0/X" for each of the interfaces used in testing and pasted it exactly as it was shown.  There are no output drops, which is what you said might have been the problem.  Not really understanding what you are getting at with that comment.

I apologize if there was miscommunication but I'm honestly just looking for some help getting this switch to perform.

[Leo Laohoo]

---

@Dan Werntz wrote:

337649686 packets input, 510636039874 bytes, 0 no buffer
Received 3534 broadcasts (2883 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 2883 multicast, 0 pause input
0 input packets with dribble condition detected
27986208 packets output, 26327847690 bytes, 0 underruns
Output 7282 broadcasts (0 multicasts)
0 output errors, 0 collisions, 2 interface resets
1880 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

---

This is not an edited output?  

[Dan Werntz]

No sir, it's pasted exactly as displayed.  Does something look strange with it?

[Leo Laohoo]

---

@Dan Werntz wrote:

No sir, it's pasted exactly as displayed.  Does something look strange with it?

---

Yes, the top juicy bits are missing.  

[Dan Werntz]

Oh jeeze, you're right.  I've provided so many different outputs in this thread I got mixed up, sorry about that.  If you look at these, I simplified the setup and provided new outputs that have everything you're looking for.

Server A

FortyGigabitEthernet1/0/9 is up, line protocol is up (connected)
Hardware is Forty Gigabit Ethernet, address is 00bf.77b9.1b09 (bia 00bf.77b9.1b09)
MTU 9000 bytes, BW 40000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 40Gb/s, link type is auto, media type is QSFP 40G SR BD SFP
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:10, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 55490000 bits/sec, 4837 packets/sec
5 minute output rate 123622000 bits/sec, 10319 packets/sec
34645693 packets input, 46132997585 bytes, 0 no buffer
Received 58109 broadcasts (48403 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 48403 multicast, 0 pause input
0 input packets with dribble condition detected
51222959 packets output, 65304435891 bytes, 0 underruns
Output 673798 broadcasts (0 multicasts)
0 output errors, 0 collisions, 2 interface resets
23962 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Server B

FortyGigabitEthernet1/0/3 is up, line protocol is up (connected)
Hardware is Forty Gigabit Ethernet, address is 00bf.77b9.1b03 (bia 00bf.77b9.1b03)
MTU 9000 bytes, BW 40000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 40Gb/s, link type is auto, media type is QSFP 40G SR BD SFP
input flow-control is on, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:13, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 4000 bits/sec, 6 packets/sec
344145127 packets input, 519690344676 bytes, 0 no buffer
Received 5723 broadcasts (4971 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 4971 multicast, 2 pause input
0 input packets with dribble condition detected
50196463 packets output, 58863769269 bytes, 0 underruns
Output 62823 broadcasts (0 multicasts)
0 output errors, 0 collisions, 2 interface resets
3658 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

[balaji.bandi]

Still i would like to see Linux settings, you should get more best results on Linux compare to Windows as per my experience.

worth looking finetuning.

Linux :

https://fasterdata.es.net/host-tuning/linux/test-measurement-host-tuning/

windows 2019

https://docs.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-performance-tuning-nics

[Dan Werntz]

That has been my experience as well, so I found it odd.  As I said though, with these same exact servers and same server interfaces connected directly to each other and using the same fiber patch I get 20+Gbps.  I really don't believe this is a server NIC issue or a tuning bottleneck at this point.  I would perfectly happy with 20Gbps through the switch if I can get it there.

[balaji.bandi]

that is good input, back to back you able to get that throughput. if this switch do not have any config just layer2 and testing

can you post full configuration along with show version

[Dan Werntz]

Datacenter-9500#show version
Cisco IOS XE Software, Version 16.12.04
Cisco IOS Software [Gibraltar], Catalyst L3 Switch Software (CAT9K_IOSXE), Version 16.12.4, RELEASE SOFTWARE (fc5)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2020 by Cisco Systems, Inc.
Compiled Thu 09-Jul-20 21:49 by mcpre

Cisco IOS-XE software, Copyright (c) 2005-2020 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0. For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

ROM: IOS-XE ROMMON
BOOTLDR: System Bootstrap, Version 16.9.1r [FC2], RELEASE SOFTWARE (P)

Datacenter-9500 uptime is 1 week, 5 days, 12 hours, 7 minutes
Uptime for this control processor is 1 week, 5 days, 12 hours, 9 minutes
System returned to ROM by PowerOn at 00:54:01 EST Thu Dec 24 2020
System restarted at 00:56:51 EST Thu Dec 24 2020
System image file is "flash:packages.conf"
Last reload reason: PowerOn

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Technology Package License Information:

------------------------------------------------------------------------------
Technology-package Technology-package
Current Type Next reboot
------------------------------------------------------------------------------
network-advantage Smart License network-advantage
dna-advantage Subscription Smart License dna-advantage
AIR License Level: AIR DNA Advantage
Next reload AIR license Level: AIR DNA Advantage

Smart Licensing Status: UNREGISTERED/EVAL EXPIRED

cisco C9500-24Q (X86) processor with 1343576K/6147K bytes of memory.
Processor board ID FCW2204A25B
2 Virtual Ethernet interfaces
96 Ten Gigabit Ethernet interfaces
24 Forty Gigabit Ethernet interfaces
2048K bytes of non-volatile configuration memory.
16777216K bytes of physical memory.
1638400K bytes of Crash Files at crashinfo:.
11264000K bytes of Flash at flash:.
0K bytes of WebUI ODM Files at webui:.

Base Ethernet MAC Address : 00:bf:77:b9:1b:00
Motherboard Assembly Number : 73-18138-02
Motherboard Serial Number : FOC22028N77
Model Revision Number : E0
Motherboard Revision Number : B0
Model Number : C9500-24Q
System Serial Number : FCW2204A25B

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 120 C9500-24Q 16.12.4 CAT9K_IOSXE INSTALL

Configuration register is 0x102

Datacenter-9500#show running-config
Building configuration...

Current configuration : 17870 bytes
!
! Last configuration change at 11:17:07 EST Mon Jan 4 2021 by user
! NVRAM config last updated at 11:17:07 EST Mon Jan 4 2021 by user
!
version 16.12
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service call-home
no platform punt-keepalive disable-kernel-core
!
hostname Datacenter-9500
!
!
vrf definition Mgmt-vrf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
logging buffered 4096000
!
aaa new-model
!
!
aaa group server tacacs+ ISE_GROUP
server name ISE
!
aaa authentication login default group ISE_GROUP local none
aaa authentication enable default group ISE_GROUP enable none
aaa authorization config-commands
aaa authorization exec default group ISE_GROUP local none
aaa authorization commands 2 default group ISE_GROUP local none
aaa authorization commands 3 default group ISE_GROUP local none
aaa authorization commands 4 default group ISE_GROUP local none
aaa authorization commands 5 default group ISE_GROUP local none
aaa authorization commands 6 default group ISE_GROUP local none
aaa authorization commands 7 default group ISE_GROUP local none
aaa authorization commands 8 default group ISE_GROUP local none
aaa authorization commands 9 default group ISE_GROUP local none
aaa authorization commands 10 default group ISE_GROUP local none
aaa authorization commands 11 default group ISE_GROUP local none
aaa authorization commands 12 default group ISE_GROUP local none
aaa authorization commands 13 default group ISE_GROUP local none
aaa authorization commands 14 default group ISE_GROUP local none
aaa authorization commands 15 default group ISE_GROUP local none
aaa authorization reverse-access default group tacacs+
aaa accounting send stop-record authentication failure
aaa accounting update newinfo
aaa accounting exec default start-stop group tacacs+
aaa accounting commands 1 default start-stop group tacacs+
aaa accounting commands 2 default start-stop group tacacs+
aaa accounting commands 3 default start-stop group tacacs+
aaa accounting commands 4 default start-stop group tacacs+
aaa accounting commands 5 default start-stop group tacacs+
aaa accounting commands 6 default start-stop group tacacs+
aaa accounting commands 8 default start-stop group tacacs+
aaa accounting commands 9 default start-stop group tacacs+
aaa accounting commands 10 default start-stop group tacacs+
aaa accounting commands 11 default start-stop group tacacs+
aaa accounting commands 12 default start-stop group tacacs+
aaa accounting commands 13 default start-stop group tacacs+
aaa accounting commands 14 default start-stop group tacacs+
aaa accounting commands 15 default start-stop group tacacs+
aaa accounting network default start-stop group tacacs+
aaa accounting connection default start-stop group tacacs+
aaa accounting system default start-stop group tacacs+
!
!
!
!
!
!
aaa session-id common
clock timezone EST -5 0
switch 1 provision c9500-24q
!
!
!
!
call-home
! If contact email address in call-home is configured as sch-smart-licensing@cisco.com
! the email address configured in Cisco Smart License Portal will be used as contact email address to send SCH notifications.
contact-email-addr sch-smart-licensing@cisco.com
profile "CiscoTAC-1"
active
destination transport-method http
no destination transport-method email
ip routing
!
!
!
!
!
ip name-server 10.10.10.10
ip domain lookup source-interface Loopback0
ip domain name lab.local
!
!
!
login on-success log
!
!
!
!
!
!
!
no device-tracking logging theft
!
flow record FLOW-IN
match ipv4 source address
match ipv4 destination address
match ipv4 protocol
match transport source-port
match transport destination-port
match interface input
collect interface output
collect counter bytes long
collect counter packets long
collect timestamp absolute first
collect timestamp absolute last
!
!
flow exporter STEALTHWATCH
destination 10.10.10.43
source Loopback0
transport udp 2055
option interface-table
option application-table timeout 10
!
!
flow monitor FLOW-MONITOR-IN
exporter STEALTHWATCH
cache timeout active 60
record FLOW-IN
!
!
crypto pki trustpoint TP-self-signed-4183636855
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-4183636855
revocation-check none
rsakeypair TP-self-signed-4183636855
!
crypto pki trustpoint SLA-TrustPoint
enrollment pkcs12
revocation-check crl
!
!
crypto pki certificate chain TP-self-signed-4183636855
certificate self-signed 01
30820330 30820218 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 34313833 36333638 3535301E 170D3230 31323234 30333233
32395A17 0D333030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D34 31383336
33363835 35308201 22300D06 092A8648 86F70D01 01010500 0382010F 00308201
0A028201 0100D9AE C6C3C203 C0D2C4A0 BFA4220B 98C82F29 8259089B F7D835A4
9A4A5FB1 41E8FF8B 65B9F4CD E5FE5393 9E21F842 54A8DAA3 579AC052 2C3558B3
749DD607 3944068E 8062723F 0513E828 F222DE8C 24AA008E 82529406 0B30B941
F86062C8 0970B40D 4D40B562 2A5BFCF0 AACFAD93 637F3B5E 79365E85 13CA66ED
ACCECB30 F0076005 40349E65 8A72B5DA A5C7F8CD 34B28893 6BECF1D8 B05E9685
E7C3EA27 DF9AA0A6 5667876C 3963F8E0 18495520 2266E206 DBAC9940 530DABAD
B530CAF7 CD88352C 06577FB6 3C10BCD7 58366407 B9CBCB99 C6E052FE 57084E72
062ADCDB 073D1F30 83E73A29 B3DE2C3E E24FD3CC 7C4B84F0 91E890C7 A3F12070
9419EC48 5E070203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF
301F0603 551D2304 18301680 149528F0 9AB195A8 F4A00B29 4C5CECFC F3368620
7F301D06 03551D0E 04160414 9528F09A B195A8F4 A00B294C 5CECFCF3 3686207F
300D0609 2A864886 F70D0101 05050003 82010100 617743C6 A23BCB1F D294DA43
1C896F8D 80FD9B8D 2DF42D6E 832F9081 5208EE53 B4315456 614CD7C4 0D5E3E5C
3F951FFA 58751CB8 BCE43738 9AB45DAE FF881589 7ACE9D41 7BC49C48 CC0E2726
813548EE 7EA80DCE 3BE35310 443F4371 A49D7928 63DDBCC3 18540775 1037B526
1F9891FE 0DC88C47 A924EDAD 82B1C947 7599E312 79006C03 F6ABF854 5AF43C1B
1FF5F618 3F3BA513 3CEF6BA7 D563EAEA 88E32C06 2A8A1EA3 EAC7511E 1D16E6A3
3F57CF63 BDD357C1 C02C56FB 46DFDCF1 F8BA9CFC CC3578D4 8CD218B7 3AE58EC0
32E2881A 1D122590 48952582 4D8D83C5 82A0AB70 4568C3DA 4B549C2C 70573E4B
521606FC A65C55C7 12EB35BB 84B6985F 889F66AE
quit
crypto pki certificate chain SLA-TrustPoint
certificate ca 01
30820321 30820209 A0030201 02020101 300D0609 2A864886 F70D0101 0B050030
32310E30 0C060355 040A1305 43697363 6F312030 1E060355 04031317 43697363
6F204C69 63656E73 696E6720 526F6F74 20434130 1E170D31 33303533 30313934
3834375A 170D3338 30353330 31393438 34375A30 32310E30 0C060355 040A1305
43697363 6F312030 1E060355 04031317 43697363 6F204C69 63656E73 696E6720
526F6F74 20434130 82012230 0D06092A 864886F7 0D010101 05000382 010F0030
82010A02 82010100 A6BCBD96 131E05F7 145EA72C 2CD686E6 17222EA1 F1EFF64D
CBB4C798 212AA147 C655D8D7 9471380D 8711441E 1AAF071A 9CAE6388 8A38E520
1C394D78 462EF239 C659F715 B98C0A59 5BBB5CBD 0CFEBEA3 700A8BF7 D8F256EE
4AA4E80D DB6FD1C9 60B1FD18 FFC69C96 6FA68957 A2617DE7 104FDC5F EA2956AC
7390A3EB 2B5436AD C847A2C5 DAB553EB 69A9A535 58E9F3E3 C0BD23CF 58BD7188
68E69491 20F320E7 948E71D7 AE3BCC84 F10684C7 4BC8E00F 539BA42B 42C68BB7
C7479096 B4CB2D62 EA2F505D C7B062A4 6811D95B E8250FC4 5D5D5FB8 8F27D191
C55F0D76 61F9A4CD 3D992327 A8BB03BD 4E6D7069 7CBADF8B DF5F4368 95135E44
DFC7C6CF 04DD7FD1 02030100 01A34230 40300E06 03551D0F 0101FF04 04030201
06300F06 03551D13 0101FF04 05300301 01FF301D 0603551D 0E041604 1449DC85
4B3D31E5 1B3E6A17 606AF333 3D3B4C73 E8300D06 092A8648 86F70D01 010B0500
03820101 00507F24 D3932A66 86025D9F E838AE5C 6D4DF6B0 49631C78 240DA905
604EDCDE FF4FED2B 77FC460E CD636FDB DD44681E 3A5673AB 9093D3B1 6C9E3D8B
D98987BF E40CBD9E 1AECA0C2 2189BB5C 8FA85686 CD98B646 5575B146 8DFC66A8
467A3DF4 4D565700 6ADF0F0D CF835015 3C04FF7C 21E878AC 11BA9CD2 55A9232C
7CA7B7E6 C1AF74F6 152E99B7 B1FCF9BB E973DE7F 5BDDEB86 C71E3B49 1765308B
5FB0DA06 B92AFE7F 494E8A9E 07B85737 F3A58BE1 1A48A229 C37C1E69 39F08678
80DDCD16 D6BACECA EEBC7CF9 8428787B 35202CDC 60E4616A B623CDBD 230E3AFB
418616A9 4093E049 4D10AB75 27E86F73 932E35B5 8862FDAE 0275156F 719BB2F0
D697DF7F 28
quit
!
!
system mtu 9000
license boot level network-advantage addon dna-advantage
!
!
diagnostic bootup level minimal
!
spanning-tree mode rapid-pvst
spanning-tree extend system-id
memory free low-watermark processor 134344
!
username user privilege 15 password 0 password
!
redundancy
mode sso
!
!
!
!
!
transceiver type all
monitoring
!
!
class-map match-any system-cpp-police-ewlc-control
description EWLC Control
class-map match-any system-cpp-police-topology-control
description Topology control
class-map match-any system-cpp-police-sw-forward
description Sw forwarding, L2 LVX data packets, LOGGING, Transit Traffic
class-map match-any system-cpp-default
description EWLC Data, Inter FED Traffic
class-map match-any system-cpp-police-sys-data
description Openflow, Exception, EGR Exception, NFL Sampled Data, RPF Failed
class-map match-any system-cpp-police-punt-webauth
description Punt Webauth
class-map match-any system-cpp-police-l2lvx-control
description L2 LVX control packets
class-map match-any system-cpp-police-forus
description Forus Address resolution and Forus traffic
class-map match-any system-cpp-police-multicast-end-station
description MCAST END STATION
class-map match-any system-cpp-police-high-rate-app
description High Rate Applications
class-map match-any system-cpp-police-multicast
description MCAST Data
class-map match-any system-cpp-police-l2-control
description L2 control
class-map match-any system-cpp-police-dot1x-auth
description DOT1X Auth
class-map match-any system-cpp-police-data
description ICMP redirect, ICMP_GEN and BROADCAST
class-map match-any system-cpp-police-stackwise-virt-control
description Stackwise Virtual OOB
class-map match-any non-client-nrt-class
class-map match-any system-cpp-police-routing-control
description Routing control and Low Latency
class-map match-any system-cpp-police-protocol-snooping
description Protocol snooping
class-map match-any system-cpp-police-dhcp-snooping
description DHCP snooping
class-map match-any system-cpp-police-ios-routing
description L2 control, Topology control, Routing control, Low Latency
class-map match-any system-cpp-police-system-critical
description System Critical and Gold Pkt
class-map match-any system-cpp-police-ios-feature
description ICMPGEN,BROADCAST,ICMP,L2LVXCntrl,ProtoSnoop,PuntWebauth,MCASTData,Transit,DOT1XAuth,Swfwd,LOGGING,L2LVXData,ForusTraffic,ForusARP,McastEndStn,Openflow,Exception,EGRExcption,NflSampled,RpfFailed
!
policy-map system-cpp-policy
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 10.255.255.5 255.255.255.255
!
interface Port-channel1
switchport mode trunk
!
interface Port-channel10
description Alpha
switchport mode trunk
load-interval 30
!
interface Port-channel12
description Gamma
switchport mode trunk
load-interval 30
!
interface Port-channel20
description Tanker-A
switchport mode trunk
!
interface GigabitEthernet0/0
vrf forwarding Mgmt-vrf
no ip address
shutdown
negotiation auto
!
interface FortyGigabitEthernet1/0/1
description Alpha-VIC-Port1
switchport mode trunk
ip flow monitor FLOW-MONITOR-IN input
load-interval 30
channel-group 10 mode active
!
interface FortyGigabitEthernet1/0/2
description Alpha-VIC-Port2
switchport mode trunk
ip flow monitor FLOW-MONITOR-IN input
load-interval 30
channel-group 10 mode active
!
interface FortyGigabitEthernet1/0/3
switchport access vlan 100
switchport trunk native vlan 100
switchport mode trunk
load-interval 30
!
interface FortyGigabitEthernet1/0/4
!
interface FortyGigabitEthernet1/0/5
description Gamma-VIC-Port1
switchport mode trunk
ip flow monitor FLOW-MONITOR-IN input
load-interval 30
channel-group 12 mode active
!
interface FortyGigabitEthernet1/0/6
description Gamma-VIC-Port2
switchport mode trunk
ip flow monitor FLOW-MONITOR-IN input
load-interval 30
channel-group 12 mode active
!
interface FortyGigabitEthernet1/0/7
description Tanker-A-Port1
switchport mode trunk
channel-group 20 mode active
!
interface FortyGigabitEthernet1/0/8
description Tanker-A-Port2
switchport mode trunk
channel-group 20 mode active
!
interface FortyGigabitEthernet1/0/9
switchport access vlan 10
switchport mode trunk
!
interface FortyGigabitEthernet1/0/10
switchport access vlan 10
switchport mode access
shutdown
!
interface FortyGigabitEthernet1/0/11
!
interface FortyGigabitEthernet1/0/12
!
interface FortyGigabitEthernet1/0/13
!
interface FortyGigabitEthernet1/0/14
!
interface FortyGigabitEthernet1/0/15
!
interface FortyGigabitEthernet1/0/16
!
interface FortyGigabitEthernet1/0/17
!
interface FortyGigabitEthernet1/0/18
!
interface FortyGigabitEthernet1/0/19
!
interface FortyGigabitEthernet1/0/20
!
interface FortyGigabitEthernet1/0/21
!
interface FortyGigabitEthernet1/0/22
!
interface FortyGigabitEthernet1/0/23
description Switchatron-Fo1/1/1
switchport mode trunk
ip flow monitor FLOW-MONITOR-IN input
load-interval 30
channel-group 1 mode active
!
interface FortyGigabitEthernet1/0/24
description Switchatron-Fo1/1/2
switchport mode trunk
ip flow monitor FLOW-MONITOR-IN input
load-interval 30
channel-group 1 mode active
!
interface TenGigabitEthernet1/0/1
!
interface TenGigabitEthernet1/0/2
!
interface TenGigabitEthernet1/0/3
!
interface TenGigabitEthernet1/0/4
!
interface TenGigabitEthernet1/0/5
!
interface TenGigabitEthernet1/0/6
!
interface TenGigabitEthernet1/0/7
!
interface TenGigabitEthernet1/0/8
!
interface TenGigabitEthernet1/0/9
!
interface TenGigabitEthernet1/0/10
!
interface TenGigabitEthernet1/0/11
!
interface TenGigabitEthernet1/0/12
!
interface TenGigabitEthernet1/0/13
!
interface TenGigabitEthernet1/0/14
!
interface TenGigabitEthernet1/0/15
!
interface TenGigabitEthernet1/0/16
!
interface TenGigabitEthernet1/0/17
!
interface TenGigabitEthernet1/0/18
!
interface TenGigabitEthernet1/0/19
!
interface TenGigabitEthernet1/0/20
!
interface TenGigabitEthernet1/0/21
!
interface TenGigabitEthernet1/0/22
!
interface TenGigabitEthernet1/0/23
!
interface TenGigabitEthernet1/0/24
!
interface TenGigabitEthernet1/0/25
!
interface TenGigabitEthernet1/0/26
!
interface TenGigabitEthernet1/0/27
!
interface TenGigabitEthernet1/0/28
!
interface TenGigabitEthernet1/0/29
!
interface TenGigabitEthernet1/0/30
!
interface TenGigabitEthernet1/0/31
!
interface TenGigabitEthernet1/0/32
!
interface TenGigabitEthernet1/0/33
!
interface TenGigabitEthernet1/0/34
!
interface TenGigabitEthernet1/0/35
!
interface TenGigabitEthernet1/0/36
!
interface TenGigabitEthernet1/0/37
!
interface TenGigabitEthernet1/0/38
!
interface TenGigabitEthernet1/0/39
!
interface TenGigabitEthernet1/0/40
!
interface TenGigabitEthernet1/0/41
!
interface TenGigabitEthernet1/0/42
!
interface TenGigabitEthernet1/0/43
!
interface TenGigabitEthernet1/0/44
!
interface TenGigabitEthernet1/0/45
!
interface TenGigabitEthernet1/0/46
!
interface TenGigabitEthernet1/0/47
!
interface TenGigabitEthernet1/0/48
!
interface TenGigabitEthernet1/0/49
!
interface TenGigabitEthernet1/0/50
!
interface TenGigabitEthernet1/0/51
!
interface TenGigabitEthernet1/0/52
!
interface TenGigabitEthernet1/0/53
!
interface TenGigabitEthernet1/0/54
!
interface TenGigabitEthernet1/0/55
!
interface TenGigabitEthernet1/0/56
!
interface TenGigabitEthernet1/0/57
!
interface TenGigabitEthernet1/0/58
!
interface TenGigabitEthernet1/0/59
!
interface TenGigabitEthernet1/0/60
!
interface TenGigabitEthernet1/0/61
!
interface TenGigabitEthernet1/0/62
!
interface TenGigabitEthernet1/0/63
!
interface TenGigabitEthernet1/0/64
!
interface TenGigabitEthernet1/0/65
!
interface TenGigabitEthernet1/0/66
!
interface TenGigabitEthernet1/0/67
!
interface TenGigabitEthernet1/0/68
!
interface TenGigabitEthernet1/0/69
!
interface TenGigabitEthernet1/0/70
!
interface TenGigabitEthernet1/0/71
!
interface TenGigabitEthernet1/0/72
!
interface TenGigabitEthernet1/0/73
!
interface TenGigabitEthernet1/0/74
!
interface TenGigabitEthernet1/0/75
!
interface TenGigabitEthernet1/0/76
!
interface TenGigabitEthernet1/0/77
!
interface TenGigabitEthernet1/0/78
!
interface TenGigabitEthernet1/0/79
!
interface TenGigabitEthernet1/0/80
!
interface TenGigabitEthernet1/0/81
!
interface TenGigabitEthernet1/0/82
!
interface TenGigabitEthernet1/0/83
!
interface TenGigabitEthernet1/0/84
!
interface TenGigabitEthernet1/0/85
!
interface TenGigabitEthernet1/0/86
!
interface TenGigabitEthernet1/0/87
!
interface TenGigabitEthernet1/0/88
!
interface TenGigabitEthernet1/0/89
!
interface TenGigabitEthernet1/0/90
!
interface TenGigabitEthernet1/0/91
!
interface TenGigabitEthernet1/0/92
!
interface TenGigabitEthernet1/0/93
!
interface TenGigabitEthernet1/0/94
!
interface TenGigabitEthernet1/0/95
!
interface TenGigabitEthernet1/0/96
!
interface Vlan1
no ip address
shutdown
!
interface Vlan11
ip address 10.10.11.13 255.255.255.0
!
!
router eigrp 1
network 10.10.11.0 0.0.0.255
redistribute connected
eigrp stub connected summary
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip tacacs source-interface Loopback0
ip ssh version 2
!
!
!
!
!
snmp-server community REDACTED RO
snmp ifmib ifindex persist
tacacs server ISE
address ipv4 10.10.10.40
key 7 REDACTED
!
!
!
control-plane
service-policy input system-cpp-policy
!
!
line con 0
logging synchronous
stopbits 1
line vty 0 4
exec-timeout 60 0
logging synchronous
transport input ssh
line vty 5 31
exec-timeout 60 0
logging synchronous
transport input ssh
!
ntp server time.lab.local
!
!
!
!
!
!
end

[Georg Pauwen]

Hello Dan,

I don't want to add unnecessary stuff to this thread, but what is the function of the switch in your network ? Does it really have to be a layer 3 (ip routing enabled) switch ? I am just asking because I see only one Vlan (11), which looks like it is used for management ?