High fabric utilization on a 3560-e

ncsuser · ‎11-03-2010

I am looking at the utilization on a Catalyst 3560-E. I am running a test with eight 1 GIGE links RXing at 95 % utilization. All of the traffic is going to a TenGigE port. The device on the TenGigE port is echoing back all of the traffic received, so the 1 GIG links are TXing at 95% as well.

When I do a "show controllers utilization", the port utilization numbers are correct, but the fabric utilization is higher than I would expect. The fabric should be 128 gigbits / second in this switch, and there is 7.6 gigabits TX/RX to the ten gig port. The fabric utilization is at 49%. I would expect 7.6 / 128 = 5.9% or possible double that. Am I doing the utilization calculation incorrect?

Switch#show controllers utilization
Port       Receive Utilization Transmit Utilization
Gi0/1              0                    0
Gi0/2              0                    0
Gi0/3              0                    0
Gi0/4              0                    0
Gi0/5              0                    0
Gi0/6              0                    0
Gi0/7              0                    0
Gi0/8              0                    0
Gi0/9              95                   95
Gi0/10             0                    0
Gi0/11             95                   95
Gi0/12             95                   95
Gi0/13             0                    0
Gi0/14             0                    0
Gi0/15             0                    0
Gi0/16             0                    0
Gi0/17             0                    0
Gi0/18             0                    0
Gi0/19             95                   95
Gi0/20             0                    0
Gi0/21             95                   95
Gi0/22             95                   95
Gi0/23             95                   95
Gi0/24             95                   95
Gi0/25             0                    0
Gi0/26             0                    0
Gi0/27             0                    0
Gi0/28             0                    0
Te0/1              76                   76
Te0/2              0                    0

Total Ports : 30
Switch Receive Bandwidth Percentage Utilization : 49
Switch Transmit Bandwidth Percentage Utilization : 49

Switch Fabric Percentage Utilization : 49

ncsuser · ‎11-19-2010

Anyone have any ideas on why the fabric utilization is so high?

Tim

lomonaco · ‎02-03-2011

Please Tim, send to us the show version or the exact model....

ncsuser · ‎02-04-2011

Here is the output from show version, with serial numbers removed.

Cisco IOS Software, C3560E Software (C3560E-UNIVERSAL-M), Version 12.2(52)SE, R)
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Fri 25-Sep-09 02:04 by sasyamal
Image text-base: 0x00003000, data-base: 0x02400000

ROM: Bootstrap program is C3560E boot loader
BOOTLDR: C3560E Boot Loader (C3560E-HBOOT-M) Version 12.2(44r)SE3, RELEASE SOFT)

Switch uptime is 3 weeks, 1 day, 22 hours, 24 minutes
System returned to ROM by power-on
System image file is "flash:c3560e-universal-mz.122-52.SE/c3560e-universal-mz.1"

License Level: ipbase
License Type: Permanent
Next reload license Level: ipbase

cisco WS-C3560E-24TD (PowerPC405) processor (revision G0) with 131072K bytes of.
Processor board ID FDO1424R0CG
Last reset from power-on
1 Virtual Ethernet interface
1 FastEthernet interface
28 Gigabit Ethernet interfaces
2 Ten Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : XXXXXXXXXX
Motherboard assembly number     : 73-11173-14
Motherboard serial number       : XXXXXXXXX
Model revision number           : G0
Motherboard revision number     : A0
Model number                    : WS-C3560E-24TD-S
Daughterboard assembly number   : 800-29737-01
Daughterboard serial number     : XXXXXXXXXX
System serial number            : XXXXXXXXX
Top Assembly Part Number        : 800-28914-03
Top Assembly Revision Number    : A0
Version ID                      : V04
CLEI Code Number                : COMCA00ARA
Hardware Board Revision Number : 0x00

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 30    WS-C3560E-24TD     12.2(52)SE            C3560E-UNIVERSAL-M

Configuration register is 0xF

lomonaco · ‎02-04-2011

I think strange too..

I used the following calcs:

Total 128 Gbp Full - Duplex

8 Ports x 1Gbp TX
8 Ports x 1gbps RX
1 Port x 10 Gb TX
1 Port x 10 gb RX

36

36/128 = 30%

What about some L3 routing, ACLs, etc

ncsuser · ‎02-04-2011

No L3 or ACLs. Here is my config in case there is somthing strange in it:

Building configuration...

Current configuration : 1977 bytes
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Switch
!
boot-start-marker
boot-end-marker
!
enable secret 5 XXXXXXXXXXXXXXXX
enable password XXXXXXXXX
!
!
!
no aaa new-model
system mtu routing 1500
authentication mac-move permit
ip subnet-zero
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree etherchannel guard misconfig
spanning-tree extend system-id
no spanning-tree vlan 1
!
vlan internal allocation policy ascending
!
!
!
interface FastEthernet0
no ip address
shutdown
!
interface GigabitEthernet0/1
switchport mode access
switchport nonegotiate
no keepalive
!
interface GigabitEthernet0/2
!
interface GigabitEthernet0/3
!
interface GigabitEthernet0/4
!
interface GigabitEthernet0/5
!
interface GigabitEthernet0/6
!
interface GigabitEthernet0/7
switchport mode access
switchport nonegotiate
no keepalive
!
interface GigabitEthernet0/8
!
interface GigabitEthernet0/9
!
interface GigabitEthernet0/10
!
interface GigabitEthernet0/11
!
interface GigabitEthernet0/12
!
interface GigabitEthernet0/13
!
interface GigabitEthernet0/14
!
interface GigabitEthernet0/15
!
interface GigabitEthernet0/16
!
interface GigabitEthernet0/17
!
interface GigabitEthernet0/18
!
interface GigabitEthernet0/19
!
interface GigabitEthernet0/20
!
interface GigabitEthernet0/21
!
interface GigabitEthernet0/22
!
interface GigabitEthernet0/23
!
interface GigabitEthernet0/24
!
interface GigabitEthernet0/25
!
interface GigabitEthernet0/26
!
interface GigabitEthernet0/27
!
interface GigabitEthernet0/28
!
interface TenGigabitEthernet0/1
switchport mode access
switchport nonegotiate
!
interface TenGigabitEthernet0/2
!
interface Vlan1
ip address 192.168.175.23 255.255.255.0
!
ip classless
ip http server
!
ip sla enable reaction-alerts
no cdp run
!
!
line con 0
exec-timeout 0 0
speed 115200
line vty 0 4
password XXXXXXXX
login
line vty 5 15
password XXXXXXXX
login
!
end

cwarner_klove · ‎03-18-2012

Well over a year later...I am reading this as I am looking at some details within my switches and wondering if this ever got answered? Tim, if you ever see this....do you remember if this was ever answered?

Respectfully,

ncsuser · ‎03-19-2012

No, I never was able to determine why the fabric utilization was so high.

Tim

cwarner_klove · ‎03-19-2012

Well ok.... thank you for the reply.

Chad