Unable to access switch on management vlan

[roark-michael]

I got an alert form solarwinds stating that switch is down. i treid to ping it and could not ping it. I logged into the distrobution stack that the switch was attached to and was able to ping and access the switch. I have 20 switches hanging off of the distrobution stack, i can reach all of them but this one specific switch(10.82.142.213). I did a stare and compare from a switch that i can access and the one that i cannot. All the configs are the same. the only differance is the ip address. I changed the ip address on the switch that i could not access to an ip address that i could access. still unable to get to the switch unless I try from the distro switch. I can't ping the .213 switch from the core either, I sent the ping packets with the source id as .142. I spoke with the site IT admin and he said that there was no changes made to thier enviroment. Attacked is the show run from the .213 switch. The PC's connected to the switch can access everything, there is no issue with the useers, just seems to be the mangement access to the switch. Any ideas why i can't get to the switch unless I'm connected to the distro switch?

   Access                Distribution           Core
10.82.142.213 -> 10.82.142.200 -> 10.82.142.1

Vlan 142

Ping from Dist 10.82.142.200 -> 10.82.142.213 Good

Ping from Core 10.82.142.1 -> 10.82.142.213  Unsucseccful

[Jon Marshall]

Michael

Are the distribution switches acting as L2 only ? It looks like they are as your core switch also has an IP from the 10.82.142.x subnet.

Which switch actually routes for that subnet ?

When you ping from the dist switch what does the arp table show ?

And what does the core switch arp table show for the ping ?

Jon

[roark-michael]

Jon,

Are the distribution switches acting as L2 only ? It looks like they are as your core switch also has an IP from the 10.82.142.x subnet. Yes, the distro switches are layer 2 and the core is doing the routing

Which switch actually routes for that subnet ? The Core switch is doing the routing

When you ping from the dist switch what does the arp table show ? TICA-Stack-SW01#sho arp | in 10.82.142.213

Internet 10.82.142.213         52   0c27.243c.08c1 ARPA   Vlan142

And what does the core switch arp table show for the ping ? TACGSW01#sho arp | in 10.82.142.213

Internet 10.82.142.213         53   0c27.243c.08c1 ARPA   Vlan142

[Jon Marshall]

Can the 2960 switch ping 10.82.142.1 ?

Jon

[roark-michael]

Can the 2960 switch ping 10.82.142.1 ?

No, i can not ping the core. I can ping other switches that are connected to the distro switch from .213  and swithces that are connected to the distro switch can ping 10.82.142.213.

[Jon Marshall]

Michael

What does the 2960 show in it's arp table for 10.182.142.1 ?

Also can you post the config of the trunk port on the distro switch that connects to the 2960 that isn't working.

Jon

[roark-michael]

What does the 2960 show in it's arp table for 10.182.142.1 ?

TICA-SW-M14#sho arp | in 10.82.142.1

Internet  10.82.142.1             0   0014.6a21.294c  ARPA   Vlan142

Also can you post the config of the trunk port on the distro switch that connects to the 2960 that isn't working.

TICA-Stack-SW01#sho run int g 1/0/7
Building configuration...

Current configuration : 282 bytes
!
interface GigabitEthernet1/0/7
description **Uplink to TICA-SW-M14**
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 5,6,40,140-143,333
switchport mode trunk
srr-queue bandwidth share 1 30 35 5
priority-queue out
mls qos trust cos
auto qos trust
end

[Jon Marshall]

Michael

Can't see anything obviously wrong with anything.

Using the mac address of the SVI on the 2960 that isnt working can you check the cam table on the core switch make sure the mac address is pointing out of the right trunk link ie. the one connected to the distro switch that the 2960 is connected to.

Jon

[roark-michael]

Jon,

I'm stummped as well, i looked at configs from a switch that I can access and the one that i cannot at both ends, everything looks identical except for IP.

Switch that I cannot connect to mac address

TICA-SW-M14#sho int vlan 142

Vlan142 is up, line protocol is up

  Hardware is EtherSVI, address is 0c27.243c.08c1

Core Switch Arp entry

TACGSW01#sho arp | in 10.82.142.213

Internet  10.82.142.213          39   0c27.243c.08c1  ARPA   Vlan142

TACGSW01#

[Jon Marshall · Accepted solution]

Michael

Can you check the mac address table to make sure the core switch is using the right link ie. not the arp table the actual cam table ie.

the mac address table entry will show you which port the core switch thinks the 2960 SVI mac is reachable on. Is this the correct port ie. does it lead to the distro switch which the 2960 connects to ?

Jon

[roark-michael]

Jon,

I'm not seeing a mac entry on th ecore switch  for the .213 switch that is not working.

[roark-michael]

Jon,

I'm seeing the mac address now in the core, it is pointed to the correct port. Still unable access the switch.

TACGSW01#sho mac address-table | in 08c1

142    0c27.243c.08c1    DYNAMIC     Po1

[bgroves]

Might be worth a quick check of arp for the ip of the switch having issues ( 10.82.142.213 ) in the L3 for this subnet.

Make sure the router for the subnet and the switches are all in agreement as to the mac of 10.82.142.213 before following the mac hop by hop as Jon suggested.

The fact it was working and suddenly stopped would lead me to check current running config of the access and core switches against archived config from when it was working.

Have you checked the switch via the console port for any errors ?

Can you ssh/telnet whatever you allow into the switch from one of the other in the same subnet ? (assuming your vty acl's allow this of course)

It's always possible the access switch has lost it's mind

Brian

[roark-michael]

Brian,

All switches agree on the MAC address of the switch that is not working from th e sho arp command.

I do not see any errors on any of the ports on the bad switch.

I can access the switch via telnet from any other switch that is on the same subnet attached to the distro switch. I cannot access it from the core switch.

[Jon Marshall]

Michael

I'm assuming this is not the case but the core switch vlan 142 is using the same subnet mask ?

Jon