Random packet Drops

[Manindersinghnegi]

Hi All,

I am trying to ping destination 12.12.12.12 , but every time 50% packet loss is happening.

between source and destination, we have Palo Alto Firewall, Can someone suggest how to figure out whether this routing or Firewall issue.

ICMP response is also not coming for all the packets.

RTR21#ping 12.12.12.12 source 11.11.11.11 repeat 4
Type escape sequence to abort.
Sending 4, 100-byte ICMP Echos to 12.12.12.12, timeout is 2 seconds:
Packet sent with a source address of 11.11.11.11
.!
.Jan 23 13:23:21: ICMP: echo reply rcvd, src 12.12.12.12, dst 11.11.11.11, topology BASE, dscp 0 topoid 0.!
Success rate is 50 percent (2/4), round-trip min/avg/max = 1/1/1 ms
RTR21#
.Jan 23 13:23:23: ICMP: echo reply rcvd, src 12.12.12.12, dst 11.11.11.11, topology BASE, dscp 0 topoid 0
RTR21#
.Jan 23 13:25:07: ICMP: echo reply sent, src 10.133.202.12, dst 10.37.57.173, topology BASE, dscp 0 topoid 1

[M02@rt37]

Hello @Manindersinghnegi 

Perform packet captures on both sides of the Palo Alto firewall (if possible) to analyze the flow of ICMP packets. This can help identify where the packet loss is occurring.

[Manindersinghnegi]

Right now Routers are in DC and not possible to send someone there  to connect Laptop to perform packet capture.

If there any debug or other way to figure out?

When I am bypassing the Firewall then no packet drops are happening.

When Firewall is bypass I am receiving reply for all 4 ICMP request.

RTR21#ping 12.12.12.12 source 40.40.40.40 repeat 4
Type escape sequence to abort.
Sending 4, 100-byte ICMP Echos to 12.12.12.12, timeout is 2 seconds:
Packet sent with a source address of 40.40.40.40
!!!!
Success rate is 100 percent (4/4), round-trip min/avg/max = 1/1/1 ms
RTR21#
.Jan 24 09:15:51: ICMP: echo reply rcvd, src 12.12.12.12, dst 40.40.40.40, topology BASE, dscp 0 topoid 0
.Jan 24 09:15:51: ICMP: echo reply rcvd, src 12.12.12.12, dst 40.40.40.40, topology BASE, dscp 0 topoid 0
.Jan 24 09:15:51: ICMP: echo reply rcvd, src 12.12.12.12, dst 40.40.40.40, topology BASE, dscp 0 topoid 0
.Jan 24 09:15:51: ICMP: echo reply rcvd, src 12.12.12.12, dst 40.40.40.40, topology BASE, dscp 0 topoid 0

When traffic is going through Firewall then only response is coming for 2 IMCP packets.

RTR21#ping 12.12.12.12 source 11.11.11.11 repeat 4
Type escape sequence to abort.
Sending 4, 100-byte ICMP Echos to 12.12.12.12, timeout is 2 seconds:
Packet sent with a source address of 11.11.11.11
.!
.Jan 24 09:19:31: ICMP: echo reply rcvd, src 12.12.12.12, dst 11.11.11.11, topology BASE, dscp 0 topoid 0.!
Success rate is 50 percent (2/4), round-trip min/avg/max = 1/1/1 ms
RTR21#
.Jan 24 09:19:33: ICMP: echo reply rcvd, src 12.12.12.12, dst 11.11.11.11, topology BASE, dscp 0 topoid 0

[MHM Cisco World]

This ratio 50 success 50 failed meaning one thing 

You have asymmetric traffic 

To check this 

Do traceroute and see if same hop appear with three * or there multi hops appear 

MHM

[Georg Pauwen]

Hello,

does that 50% packet loss also occur when you do an extended ping with different timeout values ? I somewhere remember that the PA times out earlier than Cisco (1,5 seconds as far as I recall). Try 1 second:

RTR21#ping
Protocol [ip]:
Target IP address: 12.12.12.12
Repeat count [5]:
Datagram size [100]:
--> Timeout in seconds [2]: 1

[Elliot Dierksen]

It could also mean that the Palo has some ICMP rate limiting in place. Windows ping has some delay in it between packets, but the IOS CLI ping does not. A good way to check that would be to do CLI pings with a count of 1. Do those always succeed? If not, then @MHM Cisco World could be right about asymmetric routing.