09-04-2013 01:48 AM - edited 03-07-2019 03:17 PM
Good day,
In our campus enviroment we have Cisco 6506 VSS Sup 2T as a core switch.
Each of our Cisco 6506 switches contain:
1) 6904-40G with 4xFourX adapter
2) empty
3) 6848-sfp
4) empty
5) Sup 2T
6) empty
Sometimes we had a problem: on all an odd or an even interfaces, interface state become "notconnect" in the interface card 6848-sfp.
On the other side there is access switches 2960, we see the blocking issues it is UDLD err-disable.
Sh/noSH didn't help.
We made some test to check connectivity, made a blocked interface in the 6500 as L3 interface and from the other side put 2960 blocked interface to the access mode and create an SVI on the 2960, to ping and check L3 connectivity between them. There was not any replies from each of side. But interfaces counters increments while we pinging. The only way to return interfaces to the normal state is reload an interface card.
Somebody faced with this problem? Any ideas about a cause?
Best regards,
Almaz
09-04-2013 04:01 AM
Hi,
Do we see any error logs in 6500 ? Please provide the same
Also provide the output of
- show diagnostic result mod 3 detail
- show log from the module using
-remote login mod 3
-show log
Regards
Partha
09-04-2013 04:36 AM
09-04-2013 12:19 PM
Could anybody describe the cause of the problem below?
16:36:10 | 10.240.0.128 | Critical | 10.240.0.128 | 16714: MSK: Bad refcount in datagram_done, ptr=F88A38C, count=0 | SYS-SW1_DFC1-2-BADSHARE | |
16:36:10 | 10.240.0.128 | Debug | 10.240.0.128 | 16723: -Traceback= 8F5DDD4z 90AD9B8z 94155F8z 942326Cz 9423974z 9423AC4z 9415DA8z 94AFF28z 93CFE3Cz 93CFE3Cz 94AFFD8z 94B78B4z 94B4F2Cz | ||
16:36:10 | 10.240.0.128 | Critical | 10.240.0.128 | 16722: MSK: Bad refcount in datagram_done, ptr=FB5AF5C, count=0 | SYS-SW1_DFC1-2-BADSHARE | |
16:36:10 | 10.240.0.128 | Debug | 10.240.0.128 | 16721: -Traceback= 8F5DDD4z 90AD9B8z 94155F8z 942326Cz 9423974z 9423AC4z 9415DA8z 94AFF28z 93CFE3Cz 93CFE3Cz 94AFFD8z 94B78B4z 94B4F2Cz | ||
16:36:10 | 10.240.0.128 | Critical | 10.240.0.128 | 16724: MSK: Bad refcount in datagram_done, ptr=FBA7CCC, count=0 | SYS-SW1_DFC1-2-BADSHARE | |
16:36:10 | 10.240.0.128 | Critical | 10.240.0.128 | 16718: MSK: Bad refcount in datagram_done, ptr=FB81DDC, count=0 | SYS-SW1_DFC1-2-BADSHARE | |
16:36:10 | 10.240.0.128 | Critical | 10.240.0.128 | 16716: MSK: Bad refcount in datagram_done, ptr=FB3EFF0, count=0 | SYS-SW1_DFC1-2-BADSHARE | |
16:36:10 | 10.240.0.128 | Debug | 10.240.0.128 | 16715: -Traceback= 8F5DDD4z 90AD9B8z 94155F8z 942326Cz 9423974z 9423AC4z 9415DA8z 94AFF28z 93CFE3Cz 93CFE3Cz 94AFFD8z 94B6C08z 94B4F2Cz | ||
16:36:09 | 10.240.0.128 | Error | 10.240.0.128 | 16712: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:09 | 10.240.0.128 | Error | 10.240.0.128 | 16711: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:09 | 10.240.0.128 | Error | 10.240.0.128 | 16713: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:09 | 10.240.0.128 | Error | 10.240.0.128 | 16710: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16709: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16704: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16707: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16708: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16706: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16705: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16703: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16701: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16699: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16702: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16698: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16700: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16697: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16695: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16696: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:08 | 10.240.0.128 | Error | 10.240.0.128 | 16694: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16689: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16693: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16688: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16690: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16687: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16686: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16683: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16684: MSK: Received an IPC message from unknown slot 35 | XDR-3-RX_UNKNOWN_PEER | |
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16685: -Traceback= 46CC708z 4CB0CB8z 5E5F42Cz 5E5F470z 5E5F7E8z 5E5FE20z 5E60418z 5E60B9Cz 4C965ECz 4C96A74z 4C9A738z 4C9AB24z 4C863D8z 4C9490Cz 804EDD4z 804AAACz | ||
16:36:07 | 10.240.0.128 | Error | 10.240.0.128 | 16680: .MSK: Te2/1/20: Link down | VSLP-SW2_STBY-3-VSLP_LMP_FAIL_REASON | |
16:36:06 | 10.240.0.128 | Informational | 10.240.0.128 | 16679: MSK: Message not sent to slot 33/0 (33) because of IPC error timeout. Disabling linecard. (Expected during linecard OIR or system reloads) | XDR-6-XDRIPCNOTIFY | |
16:36:04 | 10.240.0.128 | Informational | 10.240.0.128 | 16675: MSK: DualActive: DUAL ACTIVE CAPAILITY LOST | HA_EM-6-LOG | |
16:36:04 | 10.240.0.128 | Error | 10.240.0.128 | 16672: -Traceback= 5873DECz 5874D8Cz 5870940z 5872B38z 8E6F9A8z 5877788z 5BDB1E4z 5BDBD04z 5BDAF08z 5C17B10z 5C17C60z 5C1C124z 5C0E504z 5C1F4D4z 5C22210z 501E280z | ||
16:36:04 | 10.240.0.128 | Error | 10.240.0.128 | 16673: MSK: Failed to send cstate message to standby for GigabitEthernet2/3/18, rc=6 | NETWORK_RF_API-3-FAILSENDMSGTOSTDBY | |
16:36:04 | 10.240.0.128 | Error | 10.240.0.128 | 16674: -Traceback= 64FD060z 64FE8A0z 4805314z 4802644z 501E280z 5017B34z | ||
16:36:04 | 10.240.0.128 | Error | 10.240.0.128 | 16671: MSK: Checkpointing message transmission failure (queue flush) | CHKPT-3-TRANSMIT_FAILED | |
16:33:54 | 10.240.0.128 | Informational | 10.240.0.128 | 16652: MSK: list shahidy_in denied udp 192.168.103.251(138) -> 192.168.103.255(138), 1 packet | SEC-6-IPACCESSLOGP |
09-04-2013 10:36 PM
Hi,
As per the logs provided it looks there was some VSS issue since we are seeing the logs for blocked interface from both switch 1 and switch 2 ie not specific to module on 1 switch. I would like to know the following
1. What is the version of IOS running in 6500?
2. Where are the above error message collected from ? Is it from 6500 or 2960 ? I believe these error messages appeared during the time issue happened?
3. Please provide the output of show diagnostic result mod 5 detail ( Active superisor Switch 1 ) ?
Regards
Partha
09-06-2013 02:20 AM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide