Tracert on Cisco 4331

[wOOlf9999]

Hello Community!

I'm noticing an interesting behaviour of Windows standard "tracert" command: when I simultaneously run several tracerts to the same host in the internet I get some random timeouts on different steps of trace. I run Cisco 4331 with IOS-XE 16.6.7. On 16.9.3 the issue is the same. ZBFW is in place. When I remove zone membership for interfaces tracert works ok: all running tracerts show pretty much the same results. At least no random timeouts noticed. Is it the expected thing or something can be tuned up in configuration? Here's my config:

### Interfaces config: ###
interface GigabitEthernet0/0/0
 description ### WAN ###
 ip address XX.XX.XX.XX 255.255.255.248
 ip nat outside
 ip access-group isp3-list in
 zone-member security zone-outside
 ip tcp adjust-mss 1380
 negotiation auto
 crypto map vpn-map
interface GigabitEthernet0/0/1
 description ### LAN ###
 ip flow monitor nf-meter-ingress input
 ip flow monitor nf-meter-egress output
 ip address 172.16.6.3 255.255.255.0
 ip nat inside
 zone-member security zone-inside
 negotiation auto
 service-policy output policy-lan-qos

### NAT: ###
ip nat inside source route-map nonat-isp3-r-map interface GigabitEthernet0/0/0 overload
route-map nonat-isp3-r-map permit 10
 match ip address nat-list
 match interface GigabitEthernet0/0/0
ip access-list extended nat-list
 permit ip 172.16.0.0 0.0.7.255 any

### Zone pairs: ###
zone security zone-outside
zone security zone-inside
zone-pair security pair-inside-to-outside source zone-inside destination zone-outside
 service-policy type inspect policy-inside-to-outside
zone-pair security pair-outside-to-inside source zone-outside destination zone-inside
 service-policy type inspect policy-outside-to-inside

### Policies: ###
policy-map type inspect policy-inside-to-outside
 class type inspect class-inside-to-outside
  inspect
 class class-default
  drop log
policy-map type inspect policy-outside-to-inside
 class type inspect class-icmp-unreachable
  pass
 class class-default
  drop log

### Class maps: ###
class-map type inspect match-any class-inside-to-outside
 match protocol dns
 match protocol icmp
 match protocol ntp
 match protocol ftp
 match protocol sip
 match protocol tcp
 match protocol udp
class-map type inspect match-any class-icmp-unreachable
 match access-group name icmp-unreachable-list

### ACLs: ###
ip access-list extended isp3-list
 remark === WAN in ACL ===
 deny   tcp any object-group host-rtr-outside3 eq telnet
 deny   tcp any object-group host-rtr-outside3 eq 22
 deny   tcp any object-group host-rtr-outside3 eq www
 deny   tcp any object-group host-rtr-outside3 eq smtp
 deny   tcp any object-group host-rtr-outside3 eq 443
 deny   udp any any eq snmp
 deny   ip 127.0.0.0 0.255.255.255 any
 deny   ip 192.168.0.0 0.0.255.255 any
 deny   ip 172.16.0.0 0.15.255.255 any
 deny   ip 10.0.0.0 0.255.255.255 any
 deny   ip 0.0.0.0 0.255.255.255 any
 deny   ip 169.254.0.0 0.0.255.255 any
 deny   ip 192.0.2.0 0.0.0.255 any
 deny   ip 224.0.0.0 15.255.255.255 any
 deny   ip 240.0.0.0 15.255.255.255 any
 permit udp object-group group-vpn-peers eq isakmp object-group host-rtr-outside3 eq isakmp
 permit esp object-group group-vpn-peers object-group host-rtr-outside3
 deny   udp any eq isakmp any eq isakmp
 deny   esp any any
 permit ip any any
ip access-list extended icmp-unreachable-list
 permit icmp any object-group group-loc-nets host-unreachable
 permit icmp any object-group group-loc-nets ttl-exceeded
 permit icmp any object-group group-loc-isp ttl-exceeded

object-group network group-msk-nets
 description Local LANs
 172.16.6.0 255.255.255.0

object-group network group-msk-isp
 description External IPs
 host XX.XX.XX.XX

[Rob Ingram]

Hi,
Traceroute usually uses UDP Probes and ICMP replies, but Windows OS uses icmp echo-request probes. Try adding echo-reply to your ACL "icmp-unreachable-list" and try again. You are already logging on class-default, what is the output of the logs? that should identify what has been dropped.

[wOOlf9999]

Same thing - random timeouts.

Tracert uses ttl-exceeded messages to determine hops along the packets path so echo-replies wouln't help here...

And logging doesn't help either because when there's only one tracert running all replies are being received without timeouts. If it was due to firewall policies there were no replies at all...

I'm attaching the outputs for both multi/single tracerts. Three tracerts were run with 1 second gap (time to switch between CMD windows and hit Enter).

[Rob Ingram]

Looks potentially like rate limiting the responses on the upstream devices. If no ZBFW do they definately never timeout?

[wOOlf9999]

It's been tested on two ISPs.

Contacted with Cisco TAC, will wait if they have some insights for this...