Cisco 3750 stack and recommend IOS version

[Tomasz Mowinski]

Hi

I have 8 switches (different models) configured in stack:

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
     1 28    WS-C3750G-24TS-1U  12.2(55)SE            C3750-IPSERVICESK9-M
     2 26    WS-C3750-24P       12.2(55)SE            C3750-IPSERVICESK9-M
     3 30    WS-C3750X-24P      12.2(55)SE            C3750E-UNIVERSALK9-M
     4 30    WS-C3750X-24P      12.2(55)SE            C3750E-UNIVERSALK9-M
*    5 12    WS-C3750G-12S      12.2(55)SE            C3750-IPSERVICESK9-M
     6 30    WS-C3750E-24PD     12.2(55)SE            C3750E-UNIVERSALK9-M
     7 30    WS-C3750E-24PD     12.2(55)SE            C3750E-UNIVERSALK9-M
     8 30    WS-C3750X-24P      12.2(55)SE            C3750E-UNIVERSALK9-M

Now I'm have IOS ver. 12.2(55)SE but from time to time I'm experiencing strange system freezing. It means that all systems connected to the stack are working very, very slow. It hapens ones, sometimes twice a month and takes 5 minutes. I didn't find anything strange in logs (neither on switch nor router). I would like to install other IOS version. Could you recomend me the most stable version for this stack (built from these different models) ?

And the secound question:  - Is it possible to copy SE1 manualy to every switch and reload one by one manualy. Will they join the stack and work fine after reload or maybe there will be a IOS mismatch ?

thank you in advance

Tomek

[hobbe]

Hi

First of all I would start with splitting the stack a part.

switch number 1,2 and 5 i would split into a separate stack.

That makes shure that you have a good speed in the stackwise data transfer (double what you have today)

it also makes shure that you can use the universal Image on the 3750E and X stack. that should work better.

that would be better for the stack I think

[Tomasz Mowinski]

Hi

Now all stack ports are up so the speed shoud be 32Gb/s

  Switch #    Port 1       Port 2
  --------    ------       ------
    1           Ok           Ok
    2           Ok           Ok
    3           Ok           Ok
    4           Ok           Ok
    5           Ok           Ok
    6           Ok           Ok
    7           Ok           Ok
    8           Ok           Ok

Why should I disconnect these 3 switches ? Cisco says that these switches can work together. Beside it is better for me to have them in one stack because of a lot of traffic between these switches.

best

Tomek

[hobbe]

Yes it is true that the switches can work together.

but there is a difference in working together and working together.

If fx you take the 3750x switches.

the three of them in a stack with none of the others will give you 64 gig stackwise+, you can do a stack that is redundant with powersupplies and powerstacking and with that you can tell the switches that IF there for any reason should be a power shortage shut down these ports then this switch and so on so you protect the most valuable assets.

the switches all can do local switching so when you need to send something over the data stack there is more than likely ample space for it.

all of the stacked switches share the exact same code and microcode.

this is a stack that works together.

Now we add the 3750E

well we still have the 64gb stackwise+ and we still have local switching but they can not make use of the powerstack, the good news is that the software is still the same so this is still a stack that works together.

Now we add the 3750G and the 3750 switch

This is where things change.

The stack now is no longer compatible with stackwise+ and have to change down to stackwise.

we are now down to 32gig instead of 64.

The 3750G  and the 3750 can not do local switching wich means that every packet that enters their port needs to be sent out on the stackwise.

so first we lost half the speed then we add alot of traffic to the stackwise.

now to the code, its the same version but it is not the same code. so there will be differences in how they process things. the microcode is most likely not the same at all.

This is a stack that "works".

But you have lost both features, speed and most likely stability.

Everytime you go to an older version ov the switches you loose something.

so I would start with breaking apart the stack and do etherchannel between the 2 stacks.

if that is not possible then I would start with getting rid of the universal image and go with the ip-services image for the 3750X and E.

does this mean that the 3750 and 3750G are bad switches ?

no it just means that newer switch are better.

Good luck

HTH

[john.wright]

It may be that there is a bug in this IOS.

I have a similar problem with my 3750-x statck as well. The cpu goes up to 100% at times and the whole stack becomes sluggish.

Check your cpu with the show proc cpu history and note if you get high cpu usgae from time to time.

I am running IOS Version 12.2(53)SE2.

And I opened a TAC case and this is what I was told by the engineer.

As discussed with Dipan, I would like to inform you that the IOS you are running is affected by a bug CSCtn42790

Bug Details -

3560X/3750X: Elevated CPU usage due to Hulc LED process

Symptom:

Hulc LED Process uses 15-30% CPU on Catalyst 3560X/3750X platforms.

Conditions:

The is seen in 12.2(53)SE releases or later.

Workaround:

This is an expected behavior and there is no workaround.

Further Problem Description:

The "Hulc LED" process does following tasks:

- Check Link status on every port

- If the switch supports POE, it checks to see if there is a Power Device (PD) detected

- Check the status of the transceiver

- Update Fan status

- Set Main LED and ports LEDs

- Update both Power Supplies and RPS

- Check on system temperature status

The fix of the bug is yet not available.

[Leo Laohoo]

1.  Your software image has "universal" and "IP services".   If you have the correct license then use the "IP Services" feature set.

2.  Split your stack in two.  I do not recommend a stack size of 7 and more.

[Tomasz Mowinski]

Thanks guys.

Below you can see the stack cpu history. I think it looks suspicious....

100       **                      **                       *
90       **                      **                  *  * *
80       **                      **                  *  * *
70       **                      **                  ** * *
60       ** *        *    *      **                  ** * **
50    *  **** ** *  ****  * * *  ****         *  *   **** ***
40 **********************************************************************
30 **********************************************************************
20 **********************************************************************
10 ######################################################################
   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
             0    5    0    5    0    5    0    5    0    5    0    5    0
                   CPU% per hour (last 72 hours)
                  * = maximum CPU%   # = average CPU%

I think that I'll try to change IOS to one future set (ip services) and then if this will not help I will try to replace older models with newest.  I realy hope that this issue it is not cause by the mentioned bug. In the mean time if this problem occure again I will open a TAC request.

Thank you all for your help.

Best

Tomek

[john.wright]

That cpu history looks just like mine does.

And my stack is only 4 switches.

[Tomasz Mowinski]

Hi

I have checked  and it looks that for 3750X switches is only ipbase and universal software. So there is no chanse to have ipservices on all switches.

So I have to open case in Cisco TAC.

best

Tomek

[oliver]

Hi , i have a question can some one help me out , i have got a 3750x 48 port swith but it is showing an boot loading error.

switch: boot

Loading "flash:/c3750e-universalk9-mz.122-53.SE2/c3750e-universalk9-mz.122-53.SE2.bin"...flash:/c3750

e-universalk9-mz.122-53.SE2/c3750e-universalk9-mz.122-53.SE2.bin: no such file or directory

Error loading "flash:/c3750e-universalk9-mz.122-53.SE2/c3750e-universalk9-mz.122-53.SE2.bin"

Interrupt within 5 seconds to abort boot process.

Boot process failed...

[Leo Laohoo]

Oliver,

I've already responded to your thread.