Ping not working even if ARP entry exist

[bassomarco1998]

Hello,

i'm experiencing an odd behavior from a C9500-24YC stack (SVL) running 17.9.4a firmware version. The stack acts as a gateway for VLAN 1 (192.168.10.17). When i try to ping another client within the same VLAN the ping does not work. Below you can find the test i've done.

CS-9500#sh arp vl 1 | i .121
Internet  192.168.10.121          0   0050.5697.6817  ARPA   Vlan1
CS-9500#ping 192.168.10.121 so vl1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.121, timeout is 2 seconds:
Packet sent with a source address of 192.168.10.17 
.....
Success rate is 0 percent (0/5)
CS-9500#clear arp-cache 192.168.10.121
CS-9500#ping 192.168.10.121
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.121, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

It seems that even when the ARP entry is present, the ping still fails. I attempted to remove the specific ARP entry and then pinged again, which resolved the issue. I also tried rebooting the stack, but that did not fix the problem.

Do you know if it is some sort of bug?

[Giuseppe Larosa]

Hello @bassomarco1998 ,

try to find out where the client MAC address is learned. The SVI interface is a logical host on the master switch.

check with

show switch

who is the master.

find what port the MAC address is learned with

show mac address-table address < MAC-address>

is the hosts with problems connected to a port on the non master switch  member ?

Check also what happens after the clear of ARP entry is the switch learning the same MAC address in the ARP table ?

Hope to help

Giuseppe

[bassomarco1998]

Hello Giuseppe, thanks for your reply.

The network schema is the following

The mac address is learned from the port-channel that has been configured towards DC1 (as you can see from the output below):

CS-9500#sh arp | i 192.168.10.121
Internet  192.168.10.121          0   0050.5697.6817  ARPA   Vlan1
CS-9500#sh mac address-table address 0050.5697.6817
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
   1    0050.5697.6817    DYNAMIC     Po21
Total Mac Addresses for this criterion: 1
CS-9500#

The client (.121) is attacched to an access interface of DC1.

If i try to clear the specific ARP entry, the same mac-address appear again:

CS-9500#sh arp | i 192.168.10.121
Internet  192.168.10.121          0   0050.5697.6817  ARPA   Vlan1
CS-9500#clear ip arp 192.168.10.121
CS-9500#sh arp | i 192.168.10.121  
Internet  192.168.10.121          0   0050.5697.6817  ARPA   Vlan1
CS-9500#

[Elliot Dierksen]

Is the client in question a windows machine that belongs to a domain? If so, the problem is most likely windows firewall. If you have learned the mac address and a proper arp entry exists then all the right stuff seems to be there.

[bassomarco1998]

I've tested with various types of clients, so I don't think the issue is client-related. Additionally, if a client does not respond to the initial ICMP echo requests, it will not respond later, even if the ARP entry is cleared.

[Giuseppe Larosa]

Hello @bassomarco1998 ,

can I ask what kind of device is DC1 ?

The two Cat 9500 in SVL pair have a multi chassis ethertchannel to DC1.

Check with the commands suggested by @gtrejoor what member link is used to reach the client.

the network diagram show other switches connected to the two Cat9500 SVL pair . Are there issues also for clients connected to them ?

Edit:

I see from network diagram DC1 is a J9850A 5404RZI2. A modular HP Aruba switch.

Hope to help

Giuseppe

[shambhu.kumar]

Hello,

Can you check

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvm45417.

It is similar kind of issue but IOS-XE is different.

Regards

Shambhu Kumar

[bassomarco1998]

Thanks for your reply. 

Although the issue seems to be similar, as you mentioned the firmware version is different. I don't think it is something related to that bug, but i'm not sure.

[MHM Cisco World]

9500 don't support vss!

Also are the host connect to both device or only one device?

Check VSL status it can issue here 

MHM

[bassomarco1998]

Yep, sorry i meant StackWise Virtual. Btw the client is attacched to another switch. You can refer to the schema i posted within the reply to Giuseppe.

[MHM Cisco World]

share below

show stackwise-virtual link <<- for both 9500
show port-channel summary <<- for all SW
show etherchannel summary <<- for all SW

MHM

[gtrejoor]

Hi bassomarco1998 

When the issue is present, Did you make a packet capture and confirm the ICMP request is leaving from the switch and if the ICMP response is coming through the switch? it would help if you had first isolated which the device isn't responding, also when you clear ARP entry, it doesn't mean that the issue originated on the switch, you force the ARP resolution again to occur, and that is when the other device responds.

I suggest the following:

- Collect packet capture on the Inside/outside interface and verify the ICMP flows

- Collect the show IP cef <IP destination> to confirm the switch knows how to forward the traffic (broken & working scenario).

- Enable debugging "debug IP icmp" and look at the logs and see if the "inject" packet (ICMP request packet generated by CPU) leaves the switch.

Regards,