06-30-2005 11:09 PM - edited 03-02-2019 11:16 PM
Hi,
We are almost a week dealing with a very instable 6500.
What we see is that modules are undergoing a soft reset. When this reset fails the module has a faulty status. Resetting the module mostly solves the problem. But in some cases more then one module has this problem. Even the supervisor module is undergoing the same soft reset some time.
Here are the messages that appear in the logging.
1/07/2005,6:50:41,10.2.0.116,???,LOCAL,NOTICE,2005 Jul 01 06:47:58 %SYS-5-MOD_NOSCPPINGRESPONSE:Module 5 not responding... resetting module
1/07/2005,6:50:41,10.2.0.116,???,LOCAL7,NOTICE,2005 Jul 01 06:47:58 %SYS-5-MOD_RESET:Module 5 reset from Software
1/07/2005,6:50:43,10.2.0.116,???,LOCAL7,NOTICE,2005 Jul 01 06:48:00 %SYS-5-MOD_NOSCPPINGRESPONSE:Module 6 not responding... resetting module
1/07/2005,6:50:43,10.2.0.116,???,LOCAL7,NOTICE,2005 Jul 01 06:48:00 %SYS-5-MOD_RESET:Module 6 reset from Software
1/07/2005,6:53:36,10.2.0.116,???,LOCAL7,NOTICE,2005 Jul 01 06:50:52 %SYS-5-MOD_RMVDNLDSTOP:Download terminated for module 6. Module removed
1/07/2005,6:53:36,10.2.0.116,???,LOCAL7,NOTICE,2005 Jul 01 06:50:52 %SYS-5-MOD_DNLDFAIL:Download failed for module 6
1/07/2005,6:56:44,10.2.0.116,???,LOCAL7,EMERGENCY,2005 Jul 01 06:54:00 %SYS-0-MOD_NOSLCPRESPONSE:Module 1 SLCP not responding... resetting module
1/07/2005,6:56:44,10.2.0.116,???,LOCAL7,NOTICE,2005 Jul 01 06:54:00 %SYS-5-MOD_RESET:Module 1 reset from Software
What has been done:
Since module 5 always reboots first, this card had been replaced. Then we replaced the supervisor. Since still cards where rebooting we replaced the entire chassis. The power modules have also been replaced.
Anyone seen this problem before?
Many thanks!
06-30-2005 11:21 PM
Please paste the "show version" and "show module" ,"show power" output from the chassis.
Please see the link below and try the steps listed.
http://www.ciscotaccc.com/lanswitching/result
HTH,
-amit singh
06-30-2005 11:30 PM
SBRU_01> (enable) sh ver
WS-C6509 Software, Version NmpSW: 7.3(2)
Copyright (c) 1995-2002 by Cisco Systems
NMP S/W compiled on Aug 1 2002, 17:18:44
System Bootstrap Version: 5.2(1)
Hardware Version: 2.0 Model: WS-C6509 Serial #: SCA041201PZ
PS1 Module: WS-CAC-1300W Serial #: SON04301066
PS2 Module: WS-CAC-1300W Serial #: SON04290510
Mod Port Model Serial # Versions
--- ---- ------------------- ----------- --------------------------------------
1 2 WS-X6K-SUP1-2GE SAD03384603 Hw : 5.0
Fw : 5.2(1)
Fw1: 5.1(1)CSX
Sw : 7.3(2)
Sw1: 7.3(2)
WS-F6020A SAD03405664 Hw : 2.0
2 48 WS-X6248-TEL SAD04130FP9 Hw : 1.0
Fw : 4.2(0.24)VAI78
Sw : 7.3(2)
3 48 WS-X6248-TEL SAD0412056T Hw : 1.0
Fw : 4.2(0.24)VAI78
Sw : 7.3(2)
4 48 WS-X6248-TEL SAD04130FJH Hw : 1.0
Fw : 4.2(0.24)VAI78
Sw : 7.3(2)
5 48 WS-X6248A-TEL SAL05062YWP Hw : 3.0
Fw : 5.4(2)
Sw : 7.3(2)
6 48 WS-X6248A-TEL SAL05106KV4 Hw : 2.0
Fw : 5.4(2)
Sw : 7.3(2)
DRAM FLASH NVRAM
Module Total Used Free Total Used Free Total Used Free
------ ------- ------- ------- ------- ------- ------- ----- ----- -----
1 65408K 43089K 22319K 16384K 9993K 6391K 512K 377K 135K
Uptime is 0 day, 0 hour, 22 minutes
SBRU_01> (enable) sh mod
Mod Slot Ports Module-Type Model Sub Status
--- ---- ----- ------------------------- ------------------- --- --------
1 1 2 1000BaseX Supervisor WS-X6K-SUP1-2GE yes ok
2 2 48 10/100BaseTX Ethernet WS-X6248-TEL no ok
3 3 48 10/100BaseTX Ethernet WS-X6248-TEL no ok
4 4 48 10/100BaseTX Ethernet WS-X6248-TEL no ok
5 5 48 10/100BaseTX Ethernet WS-X6248A-TEL no other
6 6 48 10/100BaseTX Ethernet WS-X6248A-TEL no other
Mod Module-Name Serial-Num
--- -------------------- -----------
1 SAD03384603
2 SAD04130FP9
3 SAD0412056T
4 SAD04130FJH
5 SAL05062YWP
6 SAL05106KV4
Mod MAC-Address(es) Hw Fw Sw
--- -------------------------------------- ------ ---------- -----------------
1 00-30-80-f7-7a-c2 to 00-30-80-f7-7a-c3 5.0 5.2(1) 7.3(2)
00-30-80-f7-7a-c0 to 00-30-80-f7-7a-c1
00-13-60-48-63-00 to 00-13-60-48-66-ff
2 00-01-97-09-77-f0 to 00-01-97-09-78-1f 1.0 4.2(0.24)V 7.3(2)
3 00-01-97-10-c3-30 to 00-01-97-10-c3-5f 1.0 4.2(0.24)V 7.3(2)
4 00-01-97-09-78-e0 to 00-01-97-09-79-0f 1.0 4.2(0.24)V 7.3(2)
5 00-30-96-37-98-14 to 00-30-96-37-98-43 3.0 5.4(2) 7.3(2)
6 00-02-7e-c1-dc-8c to 00-02-7e-c1-dc-bb 2.0 5.4(2) 7.3(2)
Mod Sub-Type Sub-Model Sub-Serial Sub-Hw
--- ----------------------- ------------------- ----------- ------
1 L2 Switching Engine II WS-F6020A SAD03405664 2.0
06-30-2005 11:31 PM
PS1 Capacity: 1153.32 Watts (27.46 Amps @42V)
PS2 Capacity: 1153.32 Watts (27.46 Amps @42V)
PS Configuration : PS1 and PS2 in Redundant Configuration.
Total Power Available: 1153.32 Watts (27.46 Amps @42V)
Total Power Available for Line Card Usage: 1153.32 Watts (27.46 Amps @42V)
Total Power Drawn From the System: 636.30 Watts (15.15 Amps @42V)
Remaining Power in the System: 517.02 Watts (12.31 Amps @42V)
Default Inline Power allocation per port: 7.00 Watts (0.16 Amps @42V)
Slot power Requirement/Usage :
Slot Card Type PowerRequested PowerAllocated CardStatus
Watts A @42V Watts A @42V
---- ------------------- ------- ------ ------- ------ ----------
1 WS-X6K-SUP1-2GE 71.40 1.70 71.40 1.70 ok
2 WS-X6248-TEL 112.98 2.69 112.98 2.69 ok
3 WS-X6248-TEL 112.98 2.69 112.98 2.69 ok
4 WS-X6248-TEL 112.98 2.69 112.98 2.69 ok
5 WS-X6248A-TEL 112.98 2.69 112.98 2.69 other
6 WS-X6248A-TEL 112.98 2.69 112.98 2.69 other
06-30-2005 11:57 PM
Hi,
I checked the Hardware/Software and power matix for your chassis and everythign seems to be OK. Its a strange issue and only the module WS-X6248A-TEL are having the issue.
Did you try moving the modules to differnt slots say slot 7, 8 9 or swapping them with a working one. You said that you have swapped the chassis as well, do you have same problem in the new chasis as well.
Did you contact TAC for this.
regards,
-amit singh
07-01-2005 12:12 AM
Hello,
Hey many thanks for the checks!
Yes idd only the A modules seem to have this problem.. And the sup as well.
We've placed the blades in different unused slots. But with no result.
We have the same problems with the new chassis and new supervisor. But before more then 2 blades became faulty. The supervisor caused this.
But I have the idea that the blades become less frequent faulty.. But not always. Half an hour ago module 5 and 5 became faulty.
A little later al ports on blade 5 where in status "not connected". A show test 5 did not showed faults.. A reset solved the problem again.
We cannot open TAC cases, our Integrator is aware of this since 5 days. But no solution they provides could help us out...
Thanks
koen
07-05-2005 09:54 AM
This really calls for a TAC case.
Can you demand the integrator open one? I'd call anyway and provide the serial number. You may have some coverage.
Wish I could help you out her as I've never seen a 6500 have such low-level problems.
Also "show env" to see if you have cooling issues. But one would expect to see messages. What log message do we have when this occurs? 6500 can log locally or syslog depending on how you set it up.
07-06-2005 12:47 PM
A TAC case has been opened.
We do not have cooling issues.
Have a look in my next reply, I'll describe what we did till now.
Thanks for your response.
07-05-2005 10:15 AM
In the CAT OS you are running, a single faulty line card can cause issues like this. We have made enhancements to better handle this issue through the following bug
http://www.cisco.com/cgi-bin/bugtool/onebug.pl?bugid=CSCea38268
7.6(9) in 7.x has this enhancement but you will need 128MB DRAM for this. Downgrade to 6.4(17) if it as the features you need. Make sure you back up config if you are downgrading since configuration is lost during downgrade.
Also, make sure all the cards in the chassis are seated firmly and screws tightened.
Hope this helps
07-06-2005 01:03 PM
Hi,
A TAC case has been opened and I received the same feedback as you write in your reply.
Yesterday we downgraded to version 6-4-11, because for the version you also mention we need 128 and we only have 64MB.
Tonight one module showed up as the reason for all this trouble and was placed in disable by the switch. In the afternoon we replaced the module. 2 hours ago an other module went down. It was the module we replaced yesterday preventive. This module caused a lot of trouble the past days, that's why we decided to replaces this.
What has been replaced till now:
Chassis
Power supplies
Power cables
Supervisor
3 Telco blades.
We have 2 telco blades left that have not been replaced:
Yesterday we also checked the pins and screws, all seem to be fine.
We are almost dealing 2 weeks with this, and I'm running out of idea's..
Thanks for all feedback till now!
Regards
Koen Kips
SD WORX
Belgium
08-05-2005 01:22 AM
Hi, we had experienced the same issue on 4 core switches during 3 days.
"NOSCPPINGRESPONSE","Module X not responding"
These messages occured for all modules
Since the last supervisor software reset, the situation is stable.
We opened a case as well but we received the same response from the TAC :
Upgrade yours switches on 7.6(9) regarding the Bug CSCea38268.
We can't the supervisor have only 64M of Ram
Finally, did you find the solution ?
Regards
Philippe
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide