Introduction
Traceroute is useful tool to check path across IP networks. Let’s review how traceroute works on Cisco routers. We have a good document which explains traceroute command for Cisco routers -
“The traceroute command is used to discover the routes that packets actually take when traveling to their destination. The device (for example, a router or a PC) sends out a sequence of User Datagram Protocol (UDP) datagrams to an invalid port address at the remote host.
Three datagrams are sent, each with a Time-To-Live (TTL) field value set to one. The TTL value of 1 causes the datagram to "timeout" as soon as it hits the first router in the path; this router then responds with an ICMP Time Exceeded Message (TEM) indicating that the datagram has expired.
Another three UDP messages are now sent, each with the TTL value set to 2, which causes the second router to return ICMP TEMs. This process continues until the packets actually reach the other destination. Since these datagrams are trying to access an invalid port at the destination host, ICMP Port Unreachable Messages are returned, indicating an unreachable port; this event signals the Traceroute program that it is finished.“
Statement
Traceroute, sourced from Cisco devices, uses UDP packets with unreachable port by default to get “icmp port unreachable” packet from designated router to check the path connectivity. The packet will be punted according to the rib but replied directly with the IP address of input interface due to the unreachable UDP port. As distinct from UDP probe, ICMP traceroute (is not used by Cisco devices) uses ICMP packet to get the final destination ICMP reply to check the path connectivity. It will be punted and then processed by ICMP process, so the packet will be replied with the final destination of the traceroute as source ip address just like common ICMP echo process does.
Let see how it works if we trace to loopback IP-address or secondary IP-address of an interface.
Demonstration
There are IOS 15.1(4)M12a and IOS-XE 03.16.04b.S in the routers.
Let’s take two direct connected routers and trace loopback IP-address or secondary IP-address each other.
Trace 2.2.2.2 and 10.2.23.2 from R1.
Trace 2.2.2.2 and 10.2.23.2 from R1. R1#traceroute 10.2.23.2 Type escape sequence to abort. Tracing the route to 10.2.23.2 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.2 0 msec * 0 msec R1#traceroute 2.2.2.2 Type escape sequence to abort. Tracing the route to 2.2.2.2 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.2 0 msec * 0 msec
As you see we have reply sources from incoming interface of R2. Let’s review this packet in Wireshark.
Pay attention headers original packet with Dst 10.2.23.2 encapsulated to ICMP and is sent back but outer IP header has Src: 10.0.12.2.
Let’s trace R1 from R2 and compare behavior.
R2#traceroute 1.1.1.1 Type escape sequence to abort. Tracing the route to 1.1.1.1 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.1 1 msec * 1 msec R2#traceroute 10.1.10.1 Type escape sequence to abort. Tracing the route to 10.1.10.1 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.1 1 msec * 1 msec
We have the same behavior.
Let’s take third router and make R1 reachable for R2 through R3 but R1 still uses direct connection to reach R2.
Let’s trace loopback IP-address or secondary IP-address of R2 but use Loopback0 of R1 as source. R1’s loopback (1.1.1.1) is reachable on R2 through R3.
R2#sh ip cef 1.1.1.1 0.0.0.0/0 nexthop 10.0.23.3 GigabitEthernet0/0/0.200
However, on R1 2.2.2.2 and 10.2.23.2 are reachable through direct connection to R2.
R1#sh ip cef 2.2.2.2 0.0.0.0/0 nexthop 10.0.12.2 GigabitEthernet0/0.100 R1#sh ip cef 10.2.23.2 0.0.0.0/0 nexthop 10.0.12.2 GigabitEthernet0/0.100
Thus we have different incoming (10.0.12.2) and outgoing (10.0.23.2) interfaces for the trace on R2.
R1#traceroute 10.2.23.2 source 1.1.1.1 Type escape sequence to abort. Tracing the route to 10.2.23.2 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.2 0 msec * 0 msec R1#traceroute 2.2.2.2 source 1.1.1.1 Type escape sequence to abort. Tracing the route to 2.2.2.2 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.2 0 msec * 0 msec
As expected we have reply from 10.0.12.2. Let’s see debug ip packet from R2.
IP: tableid=0, s=1.1.1.1 (GigabitEthernet0/0/0.100), d=10.2.23.2 (GigabitEthernet0/0/0.200), routed via RIB ICMP: dst (10.2.23.2) port unreachable sent to 1.1.1.1 IP: tableid=0, s=10.0.12.2 (local), d=1.1.1.1 (GigabitEthernet0/0/0.200), routed via FIB IP: s=10.0.12.2 (local), d=1.1.1.1 (GigabitEthernet0/0/0.200), len 56, sending IP: tableid=0, s=1.1.1.1 (GigabitEthernet0/0/0.100), d=2.2.2.2 (Loopback0), routed via RIB ICMP: dst (2.2.2.2) port unreachable sent to 1.1.1.1 IP: tableid=0, s=10.0.12.2 (local), d=1.1.1.1 (GigabitEthernet0/0/0.200), routed via FIB IP: s=10.0.12.2 (local), d=1.1.1.1 (GigabitEthernet0/0/0.200), len 56, sending
The debug just shows it one more time – R2 uses IP address of incoming interface for reply.
At the end let’s review debug of R1 to compare IOS and IOS-XE behavior. I just changed routing according to red arrow on the picture below. R2 can reach IP addresses of R1 directly bur R1 does through R3.
R2#traceroute 1.1.1.1 source 2.2.2.2 Type escape sequence to abort. Tracing the route to 1.1.1.1 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.1 2 msec * 2 msec R2#traceroute 10.1.13.1 source 2.2.2.2 Type escape sequence to abort. Tracing the route to 10.1.13.1 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.12.1 2 msec * 2 msec
And debug ip packet from R1.
IP: s=2.2.2.2 (GigabitEthernet0/0.100), d=1.1.1.1, len 28, rcvd 0 IP: tableid=0, s=2.2.2.2 (GigabitEthernet0/0.100), d=1.1.1.1 (Loopback0), routed via RIB ICMP: dst (1.1.1.1) port unreachable sent to 2.2.2.2 IP: s=10.0.12.1 (local), d=2.2.2.2 (GigabitEthernet0/0.300), len 56, sending IP: s=10.0.12.1 (local), d=2.2.2.2 (GigabitEthernet0/0.300), len 56, sending full packet IP: s=2.2.2.2 (GigabitEthernet0/0.100), d=10.1.13.1, len 28, rcvd 0 IP: tableid=0, s=2.2.2.2 (GigabitEthernet0/0.100), d=10.1.13.1 (GigabitEthernet0/0.300), routed via RIB ICMP: dst (10.1.13.1) port unreachable sent to 2.2.2.2 IP: s=10.0.12.1 (local), d=2.2.2.2 (GigabitEthernet0/0.300), len 56, sending IP: s=10.0.12.1 (local), d=2.2.2.2 (GigabitEthernet0/0.300), len 56, sending full packet
As we make sure we have the same behavior. Cisco IOS and IOS-XE routers send reply with the IP-address of input interface due to the unreachable UDP port as against ICMP probe which get reply with the final destination of the traceroute as source IP-address.
IOS IOS-XE Traceroute