Solved: ICMP unreachables not sent when Fragmentation required, and DF flag set (Type 3, code 4 ICMP message...

jakelomax · ‎02-12-2013

I have a problem with a router not sending out ICMP unreachables when it receives a frame which exceeds the MTU and the do not fragment flag is set.

Cisco IOS Software, s2t54 Software (s2t54-IPSERVICESK9-M), Version 15.0(1)SY2, RELEASE SOFTWARE (fc4)

Mod Ports Card Type Model

--- ----- -------------------------------------- ------------------

1 48 CEF720 48 port 1000mb SFP WS-X6748-SFP

5 5 Supervisor Engine 2T 10GE w/ CTS VS-SUP2T-10G

interface Tunnel0

ip address 1.1.2.17 255.255.255.248

no ip redirects

ip tcp adjust-mss 1436

tunnel source 1.1.2.1

tunnel destination 1.1.2.9

tunnel path-mtu-discovery

end

---------------------

Ping across tunnel:

#ping

Protocol [ip]:

Target IP address: 1.9.8.17

Repeat count [5]:

Datagram size [100]:

Timeout in seconds [2]:

Extended commands [n]: y

Source address or interface: loopback0

Type of service [0]:

Set DF bit in IP header? [no]: yes

Validate reply data? [no]:

Data pattern [0xABCD]:

Loose, Strict, Record, Timestamp, Verbose[none]: v

Loose, Strict, Record, Timestamp, Verbose[V]:

Sweep range of sizes [n]: y

Sweep min size [36]: 1350

Sweep max size [18024]: 1550

Sweep interval [1]: 5

Type escape sequence to abort.

Sending 205, [1350..1550]-byte ICMP Echos to 1.9.8.17, timeout is 2 seconds:

Packet sent with a source address of 1.4.4.88

Packet sent with the DF bit set

Request 0 timed out (size 1350)

Reply to request 1 (16 ms) (size 1355)

Reply to request 2 (12 ms) (size 1360)

Reply to request 3 (16 ms) (size 1365)

Reply to request 4 (16 ms) (size 1370)

Reply to request 5 (16 ms) (size 1375)

Reply to request 6 (16 ms) (size 1380)

Reply to request 7 (16 ms) (size 1385)

Reply to request 8 (12 ms) (size 1390)

Reply to request 9 (16 ms) (size 1395)

Reply to request 10 (16 ms) (size 1400)

Reply to request 11 (16 ms) (size 1405)

Reply to request 12 (16 ms) (size 1410)

Reply to request 13 (16 ms) (size 1415)

Reply to request 14 (12 ms) (size 1420)

Reply to request 15 (16 ms) (size 1425)

Reply to request 16 (16 ms) (size 1430)

Reply to request 17 (16 ms) (size 1435)

Reply to request 18 (16 ms) (size 1440)

Reply to request 19 (16 ms) (size 1445)

Reply to request 20 (12 ms) (size 1450)

Reply to request 21 (16 ms) (size 1455)

Reply to request 22 (16 ms) (size 1460)

Reply to request 23 (16 ms) (size 1465)

Reply to request 24 (16 ms) (size 1470)

Reply to request 25 (16 ms) (size 1475)

Request 26 timed out (size 1480)

Request 27 timed out (size 1485)

Request 28 timed out (size 1490)

Request 29 timed out (size 1495)

Request 30 timed out (size 1500)

Success rate is 80 percent (25/31), round-trip min/avg/max = 12/15/16 ms

#sho run | inc unreachable

class-map match-any class-copp-icmp-redirect-unreachable

class class-copp-icmp-redirect-unreachable

#sho ip icmp rate-limit

DF bit unreachables All other unreachables

Interval (millisecond) 500 500

Log threshold (packet) 1000 1000

Log interval (millisecond) 60000 60000

Interface # DF bit unreachables # All other unreachables

--------- --------------------- ------------------------

GigabitEthernet1/17 0 0

Loopback0 0 0

Loopback1 0 0

Loopback224 0 0

Tunnel0 0 0

Tunnel1 0 0

Tunnel2 0 0

Vlan601 0 0

Vlan603 0 0

Vlan605 0 0

Vlan607 0 35

Vlan608 0 99

Vlan611 0 0

Vlan613 0 0

Vlan616 0 0

Vlan617 0 0

Vlan621 0 0

Vlan623 0 0

Vlan625 0 0

Vlan628 0 0

Vlan654 0 0

Vlan667 0 0

Vlan669 0 0

Vlan671 0 0

Vlan672 0 0

Greatest number of unreachables on Vlan608

I'm perplexed, and need this feature for PMTUD to work correctly on the network. Any ideas please? Thanks

pankudin · ‎02-25-2013

fixed in 15.0(1)SY3 and 15.1(1)SY

View solution in original post

jakelomax · ‎02-13-2013

Here is a little bit more info, if anyone finds it useful?

#sho ip int t0

Tunnel0 is up, line protocol is up

Internet address is 1.1.2.17/29

Broadcast address is 255.255.255.255

Address determined by non-volatile memory

MTU is 1476 bytes

Helper address is not set

Directed broadcast forwarding is disabled

Outgoing access list is not set

Inbound access list is not set

Proxy ARP is enabled

Local Proxy ARP is disabled

Security level is default

Split horizon is enabled

ICMP redirects are never sent

ICMP unreachables are always sent

ICMP mask replies are never sent

IP fast switching is enabled

IP CEF switching is enabled

IP CEF switching turbo vector

IP Null turbo vector

Associated unicast routing topologies:

Topology "base", operation state is UP

IP multicast fast switching is enabled

IP multicast distributed fast switching is disabled

IP route-cache flags are Fast, CEF

Router Discovery is disabled

IP output packet accounting is disabled

IP access violation accounting is disabled

TCP/IP header compression is disabled

RTP/IP header compression is disabled

Probe proxy name replies are disabled

Policy routing is disabled

Network address translation is disabled

BGP Policy Mapping is disabled

Input features: CEF Packet Capture, MCI Check, TCP Adjust MSS

Output features: IP Post Routing Processing, TCP Adjust MSS, HW Shortcut Installation

Post encapsulation features: MTU Processing, IP Protocol Output Counter, IP Sendself Check, HW Shortcut Installation, CEF Packet Capture

WCCP Redirect outbound is disabled

WCCP Redirect inbound is disabled

WCCP Redirect exclude is disabled

#sho ip int g1/17

GigabitEthernet1/17 is up, line protocol is up

Internet address is 1.1.2.1/30

Broadcast address is 255.255.255.255

Address determined by non-volatile memory

MTU is 1500 bytes

Helper address is not set

Directed broadcast forwarding is disabled

Outgoing access list is not set

Inbound access list is not set

Proxy ARP is enabled

Local Proxy ARP is disabled

Security level is default

Split horizon is enabled

ICMP redirects are always sent

ICMP unreachables are always sent

ICMP mask replies are never sent

IP fast switching is enabled

IP CEF switching is enabled

IP CEF switching turbo vector

IP Null turbo vector

Associated unicast routing topologies:

Topology "base", operation state is UP

IP multicast fast switching is enabled

IP multicast distributed fast switching is disabled

IP route-cache flags are Fast, CEF

Router Discovery is disabled

IP output packet accounting is disabled

IP access violation accounting is disabled

TCP/IP header compression is disabled

RTP/IP header compression is disabled

Probe proxy name replies are disabled

Policy routing is disabled

Network address translation is disabled

BGP Policy Mapping is disabled

Input features: CEF Packet Capture, MCI Check

Output features: IP Post Routing Processing, HW Shortcut Installation

Post encapsulation features: MTU Processing, IP Protocol Output Counter, IP Sendself Check, HW Shortcut Installation, CEF Packet Capture

WCCP Redirect outbound is disabled

WCCP Redirect inbound is disabled

WCCP Redirect exclude is disabled

Peter Paluch · ‎02-13-2013

Hello,

If you want to test whether ICMP Fragmentation Needed messages are being sent from a router as a response to an oversized packet, do not send oversized packets from that router itself. That router is not going to send ICMP Unreachables to itself. Instead, you should send oversized packets from a different device that just routes through your router you want to test.

Best regards,

Peter

jakelomax · ‎02-13-2013

Hi Peter, Thanks for your input. I have already tried that and got a similar response - ie, it looks like they are not sent by the router.

jakelomax · ‎02-13-2013

For clarity I should also mention I've captured the packets in/out of the router and don't see any unreachables being sent when I send frames too large for the MTU on the link with the DF flag set.

Peter Paluch · ‎02-13-2013

Hello Jake,

The ICMP Fragmentation Needed will be sent when a packet with DF set arrives to a router and should be sent out a different interface whose MTU is smaller than the packet's size. Note that the packet must first be accepted, i.e. its size must not be larger than the incoming interface's MTU. What you did, if I understand you correctly, was basically a MTU mismatch on a single link. These MTU mismatches on a single link are not handled by ICMP; in fact, they are considered a grave misconfiguration.

What you need to do is to send a packet that will be accepted by the router on the incoming inteface but that is too large for the outgoing interface where the router tries to forward it through.

Best regards,

Peter

jakelomax · ‎02-13-2013

Hi Peter,

I'm not sure you understood correctly at all?

I'm trying to send a packet that is too large for the MTU on a tunnel, and I'm expecting to get an ICMP unreachable (type 3, code 4) back, but this is not happening and the packets are dropped even though the interfaces are configured to send ICMP unreachables, which seems strange. There is no misocnfiguration as far as I can tell, and one end of the tunnel config is shown above. The other end is configured similarly (except for the platform and IOS) and the same test from the other side of the tunnel produces ICMP unreachables.

This issue only affects A-symetric traffic as I have configured the mss tcp value to a relavant size for the IP mtu on the link (as you can see from the config). So PMTUD is not required in this instance, but for a-symetric traffic the mss is only rewritten in the header in one direction so PMTUD is required. For PMTUD to work we require the router to send ICMP unreachables which it is not doing.

I hope this clarifies the issue.

Peter Paluch · ‎02-13-2013

Hi,

I admit I am having troubles understading where the oversized packets are originated and on which interface are they supposed to be dropped, along with ICMP unreachables being generated as a result. Please understand that you know your network topology and the environment; I do not - so far, I only know that you are pinging from somewhere to somewhere else and you expect ICMP unreachables to somehow be generated at some point in the network. I am struggling to understand your topology; there is not much information about it in your posts so far.

Do you believe you could draw a quick topology sketch that indicates the position of the sender of the oversized packets and the interface whose MTU is lower?

Best regards,

Peter

jakelomax · ‎02-14-2013

Here you go Peter, obviously the IP addresses are invalid. Quite simple really.

Peter Paluch · ‎02-14-2013

Hello,

Thank you. This really helps.

This may be a blind shot but notice that on R1, your tunnel is not configured with ip mtu 1476 command while R2 is. Although the MTU on GRE tunnels should be decreased automatically, the default setting differs between different IOS versions. Can you try configuring this command on the R1's tunnel please and repeating your experiment?

Thank you!

Best regards,

Peter

jakelomax · ‎02-15-2013

Hi Paul, Unfortunately I tried that day one, and the behavior in this version of IOS is to not display that command in the config. The command tunnel path-mtu-discovery effectively enables MTU negotiation so the interface always should have the correct MTU, as can be seen from the below output (which was also posted above).

#sho ip int t0

Tunnel0 is up, line protocol is up

Internet address is 1.1.2.17/29

Broadcast address is 255.255.255.255

Address determined by non-volatile memory

MTU is 1476 bytes

Helper address is not set

Directed broadcast forwarding is disabled

Outgoing access list is not set

Inbound access list is not set

Proxy ARP is enabled

Local Proxy ARP is disabled

Security level is default

Split horizon is enabled

ICMP redirects are never sent

ICMP unreachables are always sent

ICMP mask replies are never sent

IP fast switching is enabled

IP CEF switching is enabled

IP CEF switching turbo vector

IP Null turbo vector

Associated unicast routing topologies:

Topology "base", operation state is UP

IP multicast fast switching is enabled

IP multicast distributed fast switching is disabled

IP route-cache flags are Fast, CEF

Router Discovery is disabled

IP output packet accounting is disabled

IP access violation accounting is disabled

TCP/IP header compression is disabled

RTP/IP header compression is disabled

Probe proxy name replies are disabled

Policy routing is disabled

Network address translation is disabled

BGP Policy Mapping is disabled

Input features: CEF Packet Capture, MCI Check, TCP Adjust MSS

Output features: IP Post Routing Processing, TCP Adjust MSS, HW Shortcut Installation

Post encapsulation features: MTU Processing, IP Protocol Output Counter, IP Sendself Check, HW Shortcut Installation, CEF Packet Capture

WCCP Redirect outbound is disabled

WCCP Redirect inbound is disabled

WCCP Redirect exclude is disabled

amritami · ‎07-21-2016

Hello Jake,

you need to configure "ip unreachables" on the egress L3 interface towards the host.

Thanks.

pankudin · ‎02-25-2013

fixed in 15.0(1)SY3 and 15.1(1)SY

ICMP unreachables not sent when Fragmentation required, and DF flag set (Type 3, code 4 ICMP message)