arp entry blocks ping and snmp authentication

[gavinr98]

Having a strange issue on one of my 3750 switches and hoping someone might be able shed some light on whats happening. We are enforcing NAC on our network and we are having an issue where one of the switches stops accepting snmp authentication (only from this device). When testing the authentication from the NAC management we get an authentication failure. When I log into the switch, and try and ping the NAC device, I dont get a response. When I clear the arp entry from the switch, I am able to ping the NAC device again, but only for a few minutes, and then the same thing happens again. Anyone have any ideas on what might cause this?

[Richard Burts]

I am not sure what the issue is here and suggest this approach to troubleshoot it:

1) when the problem is happening do show arp and record the arp entry in question.

2) clear the arp entries

3) ping the NAC device

4) show arp and record the arp entry in question

5) wait till the problem starts again

6) show arp and record the arp entry in question

7) post the outputs

It will be interesting to see what is the original arp entry when you are having the problem, to see what is the arp entry when it is working, and to see if the original arp entry comes back when it is not working or is a different entry.

Once we have this information we will evaluate what is our next step.

HTH

Rick

[gavinr98]

Here are the outputs

Cannot Ping
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.1.1.137 0 0050.5689.7d60 ARPA Vlan1

Cleared ARP cache and can ping
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.1.1.137 0 0050.5689.7d60 ARPA Vlan1

After 30 seconds, cannot ping
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.1.1.137 0 0050.5689.7d60 ARPA Vlan1

[Richard Burts]

Thanks for the output. This is surprising. I expected to see a change in the arp entry. But this output shows exactly the same entry each time. So we need to look further for the cause of the issue. Can you tell us about the topology of the network? Is 10.1.1.137 locally connected on this switch? Or does the IP packet go through some other device(s) to get to that address? Is layer 3 routing enabled on this switch or is it a layer 2 only switch?

HTH

Rick

[gavinr98]

This is a Layer 3 switch with routing enabled. The Server is not directly connected to this switch, we have a 20MB fiber link to the building where this switch is. We have several other switches in the same location and do not have this issue. I am starting to think that there may be an issue with the IOS on this switch as I also get disconnected occasionally for no reason, other times I have to ssh a few times before I can access the switch.  

[Richard Burts]

Thanks for the additional information. Can you tell us whether the mac address shown in the arp entry is the address of the NAC device, or is the address of some intermediate device/next hop?

HTH

Rick

[gavinr98]

Yes, it is the same MAC.