ā05-01-2024 01:49 PM
We have more than 40 cisco switches in the network that has lost IP connectivity via the management IP. All ports, access, and network trunks, are up and passing data and voice traffic but we cannot ping some time all the access switches via the management IP. It has been working fine from many years but last few weeks it has gone up/down a few times. All the switches are reachable from core switch and it will be reachable from the local network once we ping the any management IP of the switch. I do not understand why it is happening.
ā05-01-2024 10:46 PM
- Could be a bug in the software version they are running , upgrade to latest advisory , especially if currently on an older version ,
M.
ā05-02-2024 06:26 AM
The OP mentions 40 switches in the network. I am not clear whether the problem is impacting all of the switches or impacting some of the switches. Can you clarify?
The OP says it has been working many years, so the suggestion to upgrade code might be a good idea.
Is it possible that there have been some changes in the network which might be causing this problem?
Are there some switches that you can access while they are experiencing the problem? If so when the problem is occurring please access the switch(es) and post the output of show interface status and show arp.
ā05-02-2024 08:48 AM
ASW01#sh int status
Port Name Status Vlan Duplex Speed Type
Gi1/0/1 "VTC" notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/2 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/3 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/4 "RESERVED FOR AP c notconnect 33 auto auto 10/100/1000BaseTX
Gi1/0/5 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/6 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/7 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/8 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/9 connected 11 a-full a-100 10/100/1000BaseTX
Gi1/0/10 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/11 connected 21 a-full a-1000 10/100/1000BaseTX
Port Name Status Vlan Duplex Speed Type
Gi1/0/12 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/13 connected 11 a-full a-100 10/100/1000BaseTX
Gi1/0/14 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/15 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/16 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/17 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/18 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/19 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/20 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/21 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/22 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/23 notconnect 11 auto auto 10/100/1000BaseTX
Port Name Status Vlan Duplex Speed Type
Gi1/0/24 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/25 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/26 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/27 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/28 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/29 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/30 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/31 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/32 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/33 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/34 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/35 "Test-Switch" notconnect 1 auto auto 10/100/1000BaseTX
Port Name Status Vlan Duplex Speed Type
Gi1/0/36 "Test-AP" notconnect 313 auto auto 10/100/1000BaseTX
Gi1/0/37 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/38 connected 11 a-full a-1000 10/100/1000BaseTX
Gi1/0/39 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/40 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/41 disabled 33 auto auto 10/100/1000BaseTX
Gi1/0/42 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/43 notconnect 11 auto auto 10/100/1000BaseTX
Gi1/0/44 connected trunk a-full a-100 10/100/1000BaseTX
Gi1/0/45 "AP-FTS" connected 33 a-full a-1000 10/100/1000BaseTX
Gi1/0/46 "Switch-Sanjeev" connected trunk a-full a-100 10/100/1000BaseTX
Gi1/0/47 "Switch-Diwan" connected trunk a-full a-100 10/100/1000BaseTX
Port Name Status Vlan Duplex Speed Type
Gi1/0/48 "UPLink To UNPOL" connected trunk a-full a-1000 10/100/1000BaseTX
Gi1/0/49 "UPLink To Core" connected trunk a-full a-1000 1000BaseLX SFP
Gi1/0/50 "UPLink To Core" notconnect 1 auto auto Not Present
Gi1/0/51 "UPLink To Core" notconnect 1 auto auto Not Present
Gi1/0/52 "UPLink To Core" notconnect 1 auto auto Not Present
Fa0 disabled routed auto auto 10/100BaseTX
ASW01#
ASW01#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.155.11.121 48 bc26.c71e.91bf ARPA Vlan35
Internet 10.155.35.1 45 bc26.c71e.91bf ARPA Vlan35
Internet 10.155.35.20 - 94d4.69c7.9741 ARPA Vlan35
Internet 10.155.35.27 237 00d6.febd.5c43 ARPA Vlan35
Internet 10.155.35.30 105 a478.06c9.cbd9 ARPA Vlan35
Internet 10.155.35.31 74 00d6.fe8d.a7c1 ARPA Vlan35
Internet 10.155.35.38 16 84eb.ef1d.b059 ARPA Vlan35
ā05-02-2024 11:52 AM
Thank you for the output that I requested. In that output it looks like the ports on this switch are connected to vlan 11. Is that correct?
The show arp shows addresses in vlan 35 (including this which I believe would be the address of this switch Internet 10.155.35.20 - 94d4.69c7.9741 ARPA Vlan35, is this correct?) Can you confirm that the gateway address for this switch is one of the addresses in the output, and if so which one? And can you confirm that this switch was experiencing the problem when you gathered the output?
And would you post the output of these commands:
show ip interface brief
show ip route
ā05-02-2024 01:57 PM
Hello,
ASW01 #sh ip int brie
Interface IP-Address OK? Method Status Protocol
Vlan1 unassigned YES NVRAM administratively down down
Vlan11 unassigned YES unset up up
Vlan35 10.155.35.20 YES NVRAM up up
FastEthernet0 unassigned YES NVRAM administratively down down
GigabitEthernet1/0/1 unassigned YES unset down down
GigabitEthernet1/0/2 unassigned YES unset down down
GigabitEthernet1/0/3 unassigned YES unset up up
GigabitEthernet1/0/4 unassigned YES unset down down
GigabitEthernet1/0/5 unassigned YES unset up up
GigabitEthernet1/0/6 unassigned YES unset up up
GigabitEthernet1/0/7 unassigned YES unset up up
GigabitEthernet1/0/8 unassigned YES unset down down
GigabitEthernet1/0/9 unassigned YES unset up up
GigabitEthernet1/0/10 unassigned YES unset up up
GigabitEthernet1/0/11 unassigned YES unset up up
GigabitEthernet1/0/12 unassigned YES unset down down
GigabitEthernet1/0/13 unassigned YES unset up up
GigabitEthernet1/0/14 unassigned YES unset up up
GigabitEthernet1/0/15 unassigned YES unset down down
GigabitEthernet1/0/16 unassigned YES unset up up
GigabitEthernet1/0/17 unassigned YES unset down down
GigabitEthernet1/0/18 unassigned YES unset up up
GigabitEthernet1/0/19 unassigned YES unset up up
GigabitEthernet1/0/20 unassigned YES unset down down
GigabitEthernet1/0/21 unassigned YES unset down down
GigabitEthernet1/0/22 unassigned YES unset up up
GigabitEthernet1/0/23 unassigned YES unset down down
GigabitEthernet1/0/24 unassigned YES unset up up
GigabitEthernet1/0/25 unassigned YES unset up up
GigabitEthernet1/0/26 unassigned YES unset up up
GigabitEthernet1/0/27 unassigned YES unset up up
GigabitEthernet1/0/28 unassigned YES unset down down
GigabitEthernet1/0/29 unassigned YES unset up up
GigabitEthernet1/0/30 unassigned YES unset down down
GigabitEthernet1/0/31 unassigned YES unset up up
GigabitEthernet1/0/32 unassigned YES unset down down
GigabitEthernet1/0/33 unassigned YES unset up up
GigabitEthernet1/0/34 unassigned YES unset up up
GigabitEthernet1/0/35 unassigned YES unset down down
GigabitEthernet1/0/36 unassigned YES unset down down
GigabitEthernet1/0/37 unassigned YES unset down down
GigabitEthernet1/0/38 unassigned YES unset up up
GigabitEthernet1/0/39 unassigned YES unset down down
GigabitEthernet1/0/40 unassigned YES unset down down
GigabitEthernet1/0/41 unassigned YES unset administratively down down
GigabitEthernet1/0/42 unassigned YES unset down down
GigabitEthernet1/0/43 unassigned YES unset down down
GigabitEthernet1/0/44 unassigned YES unset up up
GigabitEthernet1/0/45 unassigned YES unset up up
GigabitEthernet1/0/46 unassigned YES unset up up
GigabitEthernet1/0/47 unassigned YES unset up up
GigabitEthernet1/0/48 unassigned YES unset up up
GigabitEthernet1/0/49 unassigned YES unset up up
GigabitEthernet1/0/50 unassigned YES unset down down
GigabitEthernet1/0/51 unassigned YES unset down down
GigabitEthernet1/0/52 unassigned YES unset down down
ASW01#sh ip route
Default gateway is 10.155.35.1
Host Gateway Last Use Total Uses Interface
ICMP redirect cache is empty
ASW01#
ā05-02-2024 02:27 PM
Thank you for the additional information. Now that we know the IP of the gateway I see something interesting. Looking into the arp table I see these entries:
Internet 10.155.11.121 48 bc26.c71e.91bf ARPA Vlan35
Internet 10.155.35.1 45 bc26.c71e.91bf ARPA Vlan35
Why are there 2 devices in vlan 35 with the same mac address? Is it possible that 10.155.11.121 was a recent addition/change?
ā05-02-2024 12:13 PM
I'm wondering if some newly occurring broadcast flood? It's the type of thing that might hit multiple devices control plane, at the same time.
ā05-02-2024 03:16 PM
Internet 10.155.11.1 - bc26.c71e.91bf ARPA Vlan11
Internet 10.155.21.1 - bc26.c71e.91bf ARPA Vlan21
Internet 10.155.33.1 - bc26.c71e.91bf ARPA Vlan33
Internet 10.155.34.1 - bc26.c71e.91bf ARPA Vlan34
Internet 10.155.35.1 - bc26.c71e.91bf ARPA Vlan35
Internet 10.155.62.1 - bc26.c71e.91bf ARPA Vlan62
Internet 10.155.114.1 - bc26.c71e.91bf ARPA Vlan114
# I have checked one of the distribution switch I found same entry here Internet 10.155.11.121 74 bc26.c71e.91bf ARPA Vlan35
DSW#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.155.11.121 74 bc26.c71e.91bf ARPA Vlan35
Internet 10.155.35.1 34 bc26.c71e.91bf ARPA Vlan35
Internet 10.155.35.10 - a478.06c9.b8d9 ARPA Vlan35
Internet 10.155.35.19 66 94d4.69c7.e141 ARPA Vlan35
Internet 10.155.35.20 67 94d4.69c7.9741 ARPA Vlan35
Internet 10.155.35.50 89 cc5a.538f.4941 ARPA Vlan35
DSW#
ā05-02-2024 03:30 PM
I am not clear what is going on here and why/how multiple devices are having the same mac address. I do know that having multiple devices in the same vlan using the same mac address can cause problems. And I believe this is your problem.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide