cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
358
Views
0
Helpful
1
Replies
Highlighted
Beginner

Nexus 9300 tunnel feature fragmentation problem

Hi everybody!

Hope any of you had the same problem.

We are using the tunnel feature in NX-OS. When the packets are bigger than the tunnel MTU and fragmentation is needed, the IP fragments created by the switch have different ID at an IP level and cannot be reassembled by receiving node.

 

We have multiple Nexus 93180YC-EX in different locations and with different versions: 7.0(3)I7(3) & 9.2(2).

 

They all seem to be having the problem.

 

This is my test environment:

In NX-OS:

 

feature tunnel

interface Tunnel2
ip address 172.16.1.1/30
tunnel source Vlan212
tunnel destination 172.23.1.1

mtu 1000
no shutdown


ip route 1.1.1.1/32 Tunnel2


I then go to a L2 access switch routing through the previous NX-OS switch
Access Switch#ping 1.1.1.1 size 2000 repeat 1

 

The tunnel in this test terminates at a server running Wireshark (IP 172.23.1.1)

I can see the capture with 3 packets not necessarily in order. The 1st and 2nd packets have IP offset of 0 & 976; the ID is 0x80f0. 

The last packet has an IP offset of 1480; the ID is 0x00f0!!!

One bit is wrong! instead of 0x80f0 it is 0x00f0. The 3 fragments cannot be reassembled by the end recipient.

 

Any ideas or similar experiences?

 

1 REPLY 1
Highlighted
Cisco Employee

I've tried the following topology & configuration below and I'm not seeing my IP IDs or Fragment Offsets differ between each hop:

 

Screen Shot 2019-03-11 at 2.43.52 PM.png

 

Here are the ICMP Requests from N9K-3 to N9K-5 as they get fragmented:

 

############################
#### SENDING N9K, N9K-3 ####
############################

These come out fragmented themselves as I'm sending with a size of 4000 and the SVI here has default MTU (1500):

N9K-3# ethanalyzer local interface inband display-filter ip limit-c 0 Capturing on inband 2019-03-11 13:29:32.354015 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=0, ID=ee69) 2019-03-11 13:29:32.354104 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=1480, ID=ee69) 2019-03-11 13:29:32.354160 172.23.2.1 -> 1.1.1.1 ICMP Echo (ping) request ########################## ##### FRAGMENTING N9K #### ########################## N9K-4# ethanalyzer local interface inband display-filter ip limit-c 0 Original packet arriving fragmented as sent by N9K-3: Capturing on inband 2019-03-11 18:59:31.678237 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=0, ID=ee69) 2019-03-11 18:59:31.678309 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=1480, ID=ee69) 2019-03-11 18:59:31.678344 172.23.2.1 -> 1.1.1.1 ICMP Echo (ping) request Fragmenting further in order to send out of Tunnel2 (MTU of 1000):
2019-03-11 18:59:31.678991 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=0, ID=ee69) 2019-03-11 18:59:31.679228 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=976, ID=ee69) 2019-03-11 18:59:31.679298 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=1480, ID=ee69) 2019-03-11 18:59:31.679364 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=2456, ID=ee69) 2019-03-11 18:59:31.679446 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=2960, ID=ee69) 2019-03-11 18:59:31.679529 172.23.2.1 -> 1.1.1.1 ICMP Echo (ping) request ########################## ##### RECEIVING N9K-5 #### ########################## N9K-5# ethanalyzer local interface inband display-filter ip limit-c 0 Capturing on inband 2019-03-11 18:05:30.499740 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=0, ID=ee69) 2019-03-11 18:05:30.499925 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=976, ID=ee69) 2019-03-11 18:05:30.500004 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=1480, ID=ee69) 2019-03-11 18:05:30.500056 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=2456, ID=ee69) 2019-03-11 18:05:30.500153 172.23.2.1 -> 1.1.1.1 IP Fragmented IP protocol (proto=ICMP 0x01, off=2960, ID=ee69) 2019-03-11 18:05:30.500214 172.23.2.1 -> 1.1.1.1 ICMP Echo (ping) request Notice all offsets above line up with the second half of the "fragmenting N9Ks" output.

I'll unicast you to get specific configurations / setup in the event something differs and I can gladly test again -- We can circle back to the forum with the "answer" at the end.

 

Thank you,

- Andrea, CCIE #56739 R&S
Content for Community-Ad