Any way to audit for VLAN overload?

[lchance]

We have a 6513 with multiple VLANs. One VLAN is for servers and we probably have nearly 200 systems on it (combination of Windows / ESX).

From various performance issues I'm seeing I think this single VLAN is overloaded. How do I approach proving it?

Is there anything within the IOS to use or maybe a sniffer capture to help detect this? Maybe a wireshark capture on that VLAN?

Thanks for any help.

Larry

[stephenshaw]

Hi Larry,

post the "show mod"  and "show ver" outputs.

Without knowing the specifics of the performance problems ..... this is one possibility.

Some older 6500 series modules use 8:1 oversubscription i.e. 8 ports are controlled by 1 ASIC and share a 1MB buffer. We literally used to have server VLANs drop due to the oversubscription until we installed modules with better oversubscription parameters (2.1:1).

*quoted from this blog:

http://blog.super-networking.net/2006/10/performance-issues-on-your-76006500-series-cisco-devices/

"Check this output in order to verify that the module experiences drops related to over utilized buffers:

• CatOSCat6500 (enable) show asicreg pinnacle errCheck this output in the list of registers. If the settings in this output are non-zero, it indicates that there were drops due to the buffer overrun.015B: PI_PBT_S_QOS3_OUTLOST_REG = 0011

  015F: PI_PBT_S_HOLD_REG = D26C

  
  
• NativeIOSCat6500# show counters interface gigabitEthernet | include qos3Outlost51. qos3Outlost = 768504851
  

Run the show commands several times to check if asicreg steadily increments. The asicreg outputs are cleared every time they are run. If the asicreg outputs remain non-zero then this indicates active drops. Based on the rate of traffic, this data might need to be collected over several minutes in order to get significant increments."

Steve

[lchance]

Here it is Steve...Let me know what you think...thanks

(Sorry the paste doesn't format well)

show mod

Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1   48  48 port 10/100/1000mb EtherModule      WS-X6148-GE-TX    
  2   48  48 port 10/100 mb RJ45                 WS-X6348-RJ-45    
  3   16  SFM-capable 16 port 1000mb GBIC        WS-X6516-GBIC     
  4   16  16 port GE RJ45                        WS-X6316-GE-TX    
  5   48  48 port 10/100 mb RJ45                 WS-X6348-RJ-45    
  6   48  48 port 10/100 mb RJ45                 WS-X6348-RJ-45    
  7    2  Supervisor Engine 720 (Active)         WS-SUP720-3B      
  8    2  Supervisor Engine 720 (Hot)            WS-SUP720-3B      
  9   48  48 port 10/100 mb RJ45                 WS-X6348-RJ-45    
13   48  48-port 10/100/1000 RJ45 EtherModule   WS-X6148A-GE-TX   

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  0000.0000.0000 to 0000.0000.0000   1.1   7.2(1)       8.5(0.46)ROC Ok
  2  0000.0000.0000 to 0000.0000.0000   5.0   5.4(2)       8.5(0.46)ROC Ok
  3  0000.0000.0000 to 0000.0000.0000   4.1   6.3(1)       8.5(0.46)ROC Ok
  4  0000.0000.0000 to 0000.0000.0000   1.3   5.4(2)       8.5(0.46)ROC Ok
  5  0000.0000.0000 to 0000.0000.0000   6.1   5.4(2)       8.5(0.46)ROC Ok
  6  0000.0000.0000 to 0000.0000.0000   6.8   5.4(2)       8.5(0.46)ROC Ok
  7  0000.0000.0000 to 0000.0000.0000   4.4   8.1(3)       12.2(18)SXE4 Ok
  8  0000.0000.0000 to 0000.0000.0000   4.4   8.1(3)       12.2(18)SXE4 Ok
  9  0000.0000.0000 to 0000.0000.0000   2.1   5.4(2)       8.5(0.46)ROC Ok
13  0000.0000.0000 to 0000.0000.0000   4.1   Unknown      Unknown      PwrDown

Mod Sub-Module                  Model              Serial        Hw     Status
--- --------------------------- ------------------ ------------ ------- -------
  2 Inline Power Module         WS-F6K-PWR                       1.0    Ok
  5 Inline Power Module         WS-F6K-PWR                       1.0    Ok
  6 Inline Power Module         WS-F6K-PWR                       2.0    Ok
  7 Policy Feature Card 3       WS-F6K-PFC3B          2.1    Ok
  7 MSFC3 Daughterboard         WS-SUP720             2.3    Ok
  8 Policy Feature Card 3       WS-F6K-PFC3B           2.1    Ok
  8 MSFC3 Daughterboard         WS-SUP720              2.3    Ok
  9 Inline Power Module         WS-F6K-PWR                       1.0    Ok

Mod Online Diag Status
--- -------------------
  1 Pass
  2 Pass
  3 Pass
  4 Pass
  5 Pass
  6 Pass
  7 Pass
  8 Pass
  9 Pass
13 Unknown

show version
Cisco Internetwork Operating System Software
IOS (tm) s72033_rp Software (s72033_rp-IPSERVICESK9_WAN-M), Version 12.2(18)SXE4, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2005 by cisco Systems, Inc.
Compiled Fri 07-Oct-05 14:16 by kellmill
Image text-base: 0x40021040, data-base: 0x42B18000

ROM: System Bootstrap, Version 12.2(17r)S2, RELEASE SOFTWARE (fc1)
BOOTLDR: s72033_rp Software (s72033_rp-IPSERVICESK9_WAN-M), Version 12.2(18)SXE4, RELEASE SOFTWARE (fc2)

ThisIsMy6513 uptime is 1 year, 39 weeks, 1 day, 20 hours, 43 minutes
Time since ThisIsMy6513 switched to active is 1 year, 39 weeks, 1 day, 20 hours, 55 minutes
System returned to ROM by unknown reload cause - suspect boot_data[BOOT_COUNT] 0x0, BOOT_COUNT 0, BOOTDATA 19 (SP by power-on)
System restarted at 16:20:36 CDT Sat Sep 12 2009
System image file is "disk0:s72033-ipservicesk9_wan-mz.122-18.SXE4.bin"

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C6513 (R7000) processor (revision 1.0) with 458720K/65536K bytes of memory.
Processor board ID TBM06197374
SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache
Last reset from power-on
SuperLAT software (copyright 1990 by Meridian Technology Corp).
X.25 software, Version 3.0.0.
Bridging software.
TN3270 Emulation software.
40 Virtual Ethernet/IEEE 802.3 interfaces
192 FastEthernet/IEEE 802.3 interfaces
84 Gigabit Ethernet/IEEE 802.3 interfaces
1917K bytes of non-volatile configuration memory.
8192K bytes of packet buffer memory.

65536K bytes of Flash internal SIMM (Sector size 512K).
Configuration register is 0x2102

[stephenshaw]

Hi,

yes, these are the classic modules that will have the higher oversubscription ratios. These modules are not designed for Server connections and/or high speed WAN links and can result in some of the problems you have described.

In our case, we installed a WS-X6748-GE-TX module and moved the high speed MAN link and key servers onto this module and haven't experienced the issues we had previously.

It's one possibility that could improve performance for you.

Regards,

Steve