Fragment Table Overflow Disconnects Internet

mustafa.chapal · ‎11-24-2016

I have a Cisco 3925e router running a IOS Version 15.4(3)M5 and it is being used as an edge router.

The internet stops working and the input rate gets choked on two outbound interface GE0/1 and GE0/2. Whenever this happens I see the following errors in the log:

081425: Nov 24 22:11:11.389 Karachi: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/2: the fragment table has reached its maximum threshold 64

081426: Nov 24 22:11:41.398 Karachi: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/1: the fragment table has reached its maximum threshold 64

081427: Nov 24 22:12:11.407 Karachi: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/1: the fragment table has reached its maximum threshold 64

081428: Nov 24 22:13:15.669 Karachi: %IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/2: the fragment table has reached its maximum threshold 64

Even after adding this command "ip virtual-reassembly in max-reassemblies 64" on the outbound interfaces, the problem is still there.

Please guide me how to solve this issue.

Thanks

Jose Fonseca · ‎11-24-2016

Hello Mustafa,

Please include the configuration for interfaces Gi0/1 and Gi0/2, just to confirm.

That messages is telling you that the maximum amount of datagrams that can be reassembled at any given time has reached the maximum permitted by these interfaces. The default is 16, and that number can be increased up to 1024.

Router(config-if)#ip virtual-reassembly in max-reassemblies ? 
  <1-1024>  Number of datagrams that can be reassembled at a time

I will wait for the current configuration just to double check.

Warm regards,

mustafa.chapal · ‎11-24-2016

Hi,

interface GigabitEthernet0/1

ip address x.x.x.x 255.255.255.252

ip access-group inside in

no ip redirects

no ip unreachables

no ip proxy-arp

ip flow ingress

ip virtual-reassembly in max-reassemblies 64

ip verify unicast reverse-path

ip tcp adjust-mss 1460

load-interval 30

duplex auto

speed auto

no cdp enable

no mop enabled

!

interface GigabitEthernet0/2

ip address x.x.x.x 255.255.255.252

ip access-group inside in

no ip redirects

no ip unreachables

no ip proxy-arp

ip flow ingress

ip virtual-reassembly in max-reassemblies 64

ip verify unicast reverse-path

ip tcp adjust-mss 1460

load-interval 30

duplex auto

speed auto

no cdp enable

no mop enabled

(config)#interface GigabitEthernet0/1

(config-if)#ip virtual-reassembly in max-reassemblies ?

<1-1024> Number of datagrams that can be reassembled at a time

Jose Fonseca · ‎11-24-2016

Hello Mustafa,

If we increased the value to the maximum value, the router can use a very high amount of resources trying to reassembly the packets. Please take in consideration the following information:

http://www.cisco.com/c/en/us/td/docs/ios/sec_data_plane/configuration/guide/12_4/sec_data_plane_12_4_book/sec_virt_frag_reassm.pdf

As it mentioned in the file, you need to understand if the reassembly packets are valid, or part of fake packets due to an overflow attack.

I would recommend to increase the value to 128, and turn on the following debug:

debug ip virtual-reassembly

With this, you can confirm if the packets are from valid sources, or in the other hand you can block them to avoid this situation.

mustafa.chapal · ‎11-24-2016

Hi Josefon,

I have increased the value to 128.

If the issue still happens and the packets are fake causing an overflow attack, how can I prevent this buffer overflow attack?

Jose Fonseca · ‎11-24-2016

You can always block the packets based on the source with an ACL. So you will make sure that fake packets from that particular source, won't cause this issue further.

For instance:

conf t
ip access-list extended fake-packets
deny ip host x.x.x.x any
permit ip any any

If you have different source, you must include them in the ACL, of course.

Please let me know if you have any further questions or concerns. Also and just to confirm 128 value address the issue?

mustafa.chapal · ‎11-24-2016

Hi,

Now the logs are full of the following VFR statuses:

_VFR: Trying to Coalesce in CEF path; pak:0x22BB3B14, Newpak:0x212EB604

IP_VFR: pak incomplete cpak-offset:0, cpak-len:1456, flag: 1

IP_VFR: dgrm incomplete, returning...

IP_VFR: fragment 0x22BB3B14 (sa:180.149.216.9, da:103.44.223.218, id:16342, offset:1456, len:1456) in fast path...

IP_VFR: updated Reinject handle for pak:0x22BB3B14

IP_VFR: Frag 0x22BB3B14 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22BB3B14 has particles; Created newpak 0x2130881C

IP_VFR: Getting hwidb for Pak 0x22BB3B14 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22BB3B14, Newpak:0x2130881C

IP_VFR: cpak-offset:0, cpak-len:1456, npak-offset:1456

IP_VFR: pak incomplete cpak-offset:1456, cpak-len:1456, flag: 1

IP_VFR: dgrm incomplete, returning...

IP_VFR: fragment 0x22BB3B14 (sa:180.149.216.9, da:103.44.223.218, id:16342, offset:2912, len:1093) in fast path...

IP_VFR: updated Reinject handle for pak:0x22BB3B14

IP_VFR: Frag 0x22BB3B14 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22BB3B14 has particles; Created newpak 0x2130399C

IP_VFR: Getting hwidb for Pak 0x22BB3B14 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22BB3B14, Newpak:0x2130399C

IP_VFR: cpak-offset:0, cpak-len:1456, npak-offset:1456

IP_VFR: cpak-offset:1456, cpak-len:1456, npak-offset:2912

IP_VFR: dgrm complete, switching the frags.

IP_VFR: switching fragment (sa:180.149.216.9, da:103.44.223.218, id:16342, offset:0, len:1456)

IP_VFR: Trying to Reinject frag 0x212EB604

IP_VFR: Frag 0x212EB604 hit Ok

IP_VFR: switching fragment (sa:180.149.216.9, da:103.44.223.218, id:16342, offset:1456, len:1456)

IP_VFR: Trying to Reinject frag 0x2130881C

IP_VFR: pak_subblock_free - pak 0x2130881C

IP_VFR: Frag 0x2130881C hit Ok

IP_VFR: switching fragment (sa:180.149.216.9, da:103.44.223.218, id:16342, offset:2912, len:1093)

IP_VFR: Trying to Reinject frag 0x2130399C

IP_VFR: pak_subblock_free - pak 0x2130399C

IP_VFR: Frag 0x2130399C hit Ok

IP_VFR: all fragments have been switched.

IP_VFR: pak_subblock_free - pak 0x212EB604

IP_VFR: deleted frag state for sa:180.149.216.9, da:103.44.223.218, id:16342

IP_VFR: fragment 0x22BB3B14 (sa:180.149.216.9, da:103.44.223.218, id:16346, offset:0, len:1456) in fast path...

IP_VFR: created frag state for pak:0x22BB3B14, sa:180.149.216.9, da:103.44.223.218, id:16346...

IP_VFR: created Reinject handle for pak:0x22BB3B14

IP_VFR: Frag 0x22BB3B14 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22BB3B14 has particles; Created newpak 0x2132CB0C

IP_VFR: Getting hwidb for Pak 0x22BB3B14 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22BB3B14, Newpak:0x2132CB0C

IP_VFR: pak incomplete cpak-offset:0, cpak-len:1456, flag: 1

IP_VFR: dgrm incomplete, returning...

IP_VFR: fragment 0x22BB3B14 (sa:180.149.216.9, da:103.44.223.218, id:16346, offset:1456, len:1456) in fast path...

IP_VFR: updated Reinject handle for pak:0x22BB3B14

IP_VFR: Frag 0x22BB3B14 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22BB3B14 has particles; Created newpak 0x213133D4

IP_VFR: Getting hwidb for Pak 0x22BB3B14 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22BB3B14, Newpak:0x213133D4

IP_VFR: cpak-offset:0, cpak-len:1456, npak-offset:1456

IP_VFR: pak incomplete cpak-offset:1456, cpak-len:1456, flag: 1

IP_VFR: dgrm incomplete, returning...

IP_VFR: fragment 0x22BB3B14 (sa:180.149.216.9, da:103.44.223.218, id:16346, offset:2912, len:1093) in fast path...

IP_VFR: updated Reinject handle for pak:0x22BB3B14

IP_VFR: Frag 0x22BB3B14 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22BB3B14 has particles; Created newpak 0x212DA82C

IP_VFR: Getting hwidb for Pak 0x22BB3B14 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22BB3B14, Newpak:0x212DA82C

IP_VFR: cpak-offset:0, cpak-len:1456, npak-offset:1456

IP_VFR: cpak-offset:1456, cpak-len:1456, npak-offset:2912

IP_VFR: dgrm complete, switching the frags.

IP_VFR: switching fragment (sa:180.149.216.9, da:103.44.223.218, id:16346, offset:0, len:1456)

IP_VFR: Trying to Reinject frag 0x2132CB0C

IP_VFR: Frag 0x2132CB0C hit Ok

IP_VFR: switching fragment (sa:180.149.216.9, da:103.44.223.218, id:16346, offset:1456, len:1456)

IP_VFR: Trying to Reinject frag 0x213133D4

IP_VFR: pak_subblock_free - pak 0x213133D4

IP_VFR: Frag 0x213133D4 hit Ok

IP_VFR: switching fragment (sa:180.149.216.9, da:103.44.223.218, id:16346, offset:2912, len:1093)

IP_VFR: Trying to Reinject frag 0x212DA82C

IP_VFR: pak_subblock_free - pak 0x212DA82C

IP_VFR: Frag 0x212DA82C hit Ok

IP_VFR: all fragments have been switched.

IP_VFR: pak_subblock_free - pak 0x2132CB0C

Georg Pauwen · ‎11-24-2016

Hello,

there is a source and destination IP address in the logs you posted.

Source: 180.149.216.9 --> this is registered to DELTANET-PK (Karachi, Pakistan)

Destination: 103.44.223.218 --> this is registered to CHAPAL (Karachi, Pakistan)

Do these sound familiar ?

Either way, I pinged both addresses, and the lowest MTU at which packets were not fragmented was 1452.

Try and configure 'ip mtu 1452' on both interfaces:

interface GigabitEthernet0/1

ip mtu 1452

interface GigabitEthernet0/2

ip mtu 1452

mustafa.chapal · ‎11-25-2016

Hi,

The destination is our network. I tested the MTU and I have got the following results:

Sending 32 bytes to 103.44.223.218 <- not fragmented

Sending 750 bytes to 103.44.223.218 <- not fragmented

Sending 1125 bytes to 103.44.223.218 <- not fragmented

Sending 1313 bytes to 103.44.223.218 <- not fragmented

Sending 1407 bytes to 103.44.223.218 <- not fragmented

Sending 1454 bytes to 103.44.223.218 <- not fragmented

Sending 1478 bytes to 103.44.223.218 <- FRAGMENTED!

Sending 1466 bytes to 103.44.223.218 <- not fragmented

Sending 1472 bytes to 103.44.223.218 <- not fragmented

Sending 1475 bytes to 103.44.223.218 <- FRAGMENTED!

Sending 1473 bytes to 103.44.223.218 <- FRAGMENTED!

Sending 1472 bytes to 103.44.223.218 <- not fragmented

From the tests we did, we can assume that 1472 bytes is the largest unfragmented packet size. The MTU size would be 1500, made up from 1472 payload and 28 ICMP/IP Headers and payload information.

The maximum MTU size for 103.44.223.218 is: 1500

Georg Pauwen · ‎11-25-2016

Hello,

it probably depends on where you test from. I got 1452. Either way, you wanto to get rid of the log messages, so try 1452 on both interfaces and see what it does.

mustafa.chapal · ‎12-04-2016

Using this command ip virtual-reassembly in max-reassemblies 1024 or lowering the MTU did not fix the outside interface choking problem.

Below are the logs after running the command debug ip virtual-assembly

IP_VFR: created Reinject handle for pak:0x22091708

IP_VFR: Frag 0x22091708 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22091708 has particles; Created newpak 0x215BBB70

IP_VFR: Getting hwidb for Pak 0x22091708 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22091708, Newpak:0x215BBB70

IP_VFR: pak incomplete cpak-offset:0, cpak-len:1480, flag: 1

IP_VFR: dgrm incomplete, returning...

IP_VFR: fragment 0x22091708 (sa:63.151.67.235, da:103.76.3.10, id:14820, offset:1480, len:1480) in fast path...

IP_VFR: updated Reinject handle for pak:0x22091708

IP_VFR: Frag 0x22091708 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22091708 has particles; Created newpak 0x215EE010

IP_VFR: Getting hwidb for Pak 0x22091708 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22091708, Newpak:0x215EE010

IP_VFR: cpak-offset:0, cpak-len:1480, npak-offset:1480

IP_VFR: pak incomplete cpak-offset:1480, cpak-len:1480, flag: 1

IP_VFR: dgrm incomplete, returning...

IP_VFR: fragment 0x22091708 (sa:63.151.67.235, da:103.76.3.10, id:14820, offset:2960, len:1040) in fast path...

IP_VFR: updated Reinject handle for pak:0x22091708

IP_VFR: Frag 0x22091708 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22091708 has particles; Created newpak 0x215F50E8

IP_VFR: Getting hwidb for Pak 0x22091708 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22091708, Newpak:0x215F50E8

IP_VFR: cpak-offset:0, cpak-len:1480, npak-offset:1480

IP_VFR: cpak-offset:1480, cpak-len:1480, npak-offset:2960

IP_VFR: dgrm complete, switching the frags.

IP_VFR: switching fragment (sa:63.151.67.235, da:103.76.3.10, id:14820, offset:0, len:1480)

IP_VFR: Trying to Reinject frag 0x215BBB70

IP_VFR: Frag 0x215BBB70 hit Ok

IP_VFR: switching fragment (sa:63.151.67.235, da:103.76.3.10, id:14820, offset:1480, len:1480)

IP_VFR: Trying to Reinject frag 0x215EE010

IP_VFR: pak_subblock_free - pak 0x215EE010

IP_VFR: Frag 0x215EE010 hit Ok

IP_VFR: switching fragment (sa:63.151.67.235, da:103.76.3.10, id:14820, offset:2960, len:1040)

IP_VFR: Trying to Reinject frag 0x215F50E8

IP_VFR: pak_subblock_free - pak 0x215F50E8

IP_VFR: Frag 0x215F50E8 hit Ok

IP_VFR: all fragments have been switched.

IP_VFR: pak_subblock_free - pak 0x215BBB70

IP_VFR: deleted frag state for sa:63.151.67.235, da:103.76.3.10, id:14820

IP_VFR: fragment 0x22091708 (sa:62.48.185.186, da:103.76.3.10, id:17352, offset:2960, len:1040) in fast path...

IP_VFR: created frag state for pak:0x22091708, sa:62.48.185.186, da:103.76.3.10, id:17352...

IP_VFR: created Reinject handle for pak:0x22091708

IP_VFR: Frag 0x22091708 seen in Fast path while processing frag_state

IP_VFR: Pak 0x22091708 has particles; Created newpak 0x215C0ED8

IP_VFR: Getting hwidb for Pak 0x22091708 in Ingress

IP_VFR: Trying to Coalesce in CEF path; pak:0x22091708, Newpak:0x215C0ED8

IP_VFR: dgrm incomplete, returning...

IP_VFR: fragment 0x22091708 (sa:62.202.12.3, da:103.76.3.10, id:7580, offset:2944, len:1116) in fast path...

IP_VFR: fragment 0x22091708 (sa:63.231.252.29, da:103.76.3.10, id:31306, offset:1472, len:1472) in fast path...

IP_VFR: fragment 0x22091708 (sa:64.32.122.200, da:103.76.3.10, id:17779, offset:0, len:1472) in fast path...

IP_VFR: fragment 0x22091708 (sa:63.140.119.242, da:103.76.3.10, id:1279, offset:0, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:63.140.119.242, da:103.76.3.10, id:1279, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.176.87.59, da:103.76.3.10, id:49227, offset:2960, len:1040) in fast path...

IP_VFR: fragment 0x22091708 (sa:63.231.252.29, da:103.76.3.10, id:31306, offset:2944, len:1056) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.162.207.9, da:103.76.3.10, id:55673, offset:0, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.162.207.9, da:103.76.3.10, id:55673, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.107.177.73, da:103.76.3.10, id:61438, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.107.177.73, da:103.76.3.10, id:61438, offset:2960, len:1068) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.161.134.85, da:103.76.3.10, id:19274, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.161.134.85, da:103.76.3.10, id:19274, offset:0, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.12.166.219, da:103.76.3.10, id:12405, offset:2944, len:1056) in fast path...

IP_VFR: fragment 0x22091708 (sa:65.183.9.118, da:103.76.3.10, id:27842, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:65.34.233.96, da:103.76.3.10, id:3257, offset:0, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.153.135.57, da:103.76.3.10, id:42684, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:65.34.233.96, da:103.76.3.10, id:3257, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.153.135.57, da:103.76.3.10, id:42684, offset:2960, len:1068) in fast path...

IP_VFR: fragment 0x22091708 (sa:64.17.27.48, da:103.76.3.10, id:12879, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:64.17.27.48, da:103.76.3.10, id:12879, offset:2960, len:1121) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.183.1.244, da:103.76.3.10, id:23171, offset:2960, len:1114) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.210.152.123, da:103.76.3.10, id:31693, offset:0, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.210.152.123, da:103.76.3.10, id:31693, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.210.152.123, da:103.76.3.10, id:31694, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:69.165.131.188, da:103.76.3.10, id:23973, offset:0, len:1472) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.38.131.221, da:103.76.3.10, id:6991, offset:2960, len:1040) in fast path...

IP_VFR: fragment 0x22091708 (sa:69.165.131.188, da:103.76.3.10, id:23973, offset:1472, len:1472) in fast path...

IP_VFR: fragment 0x22091708 (sa:69.165.131.188, da:103.76.3.10, id:23973, offset:2944, len:1056) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.153.201.130, da:103.76.3.10, id:31521, offset:0, len:1480) in fast path...

n:1480)

IP_VFR: Trying to Reinject frag 0x215F3860

IP_VFR: pak_subblock_free - pak 0x215F3860

IP_VFR: Frag 0x215F3860 hit Ok

IP_VFR: switching fragment (sa:61.91.183.154, da:103.76.3.10, id:30055, offset:2960, len:1040)

IP_VFR: Trying to Reinject frag 0x21618520229, da:103.76.3.10, id:29497, offset:1480, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.146.70.229, da:103.76.3.10, id:29497, offset:2960, len:1040) in fast path...

IP_VFR: fragment 0x22091708 (sa:62.146.70.229, da:103.76.3.10, id:29497, offset:0, len:1480) in fast path...

IP_VFR: fragment 0x22091708 (sa:72.143.100.162, da:103.76.3.10, id:26113,

Below is the output of ip cache flow:

SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts

Gi0/1 83.223.202.222 Null 103.76.3.10 11 0035 7230 1

Gi0/1 85.95.241.132 Gi0/3.203 103.76.3.10 11 0035 4C3B 1

Gi0/3.119 103.44.223.230 Gi0/1 119.254.210.112 11 C5CC 0035 1

Gi0/1 83.31.2.130 Null 103.76.3.10 11 3258 79D7 1

Gi0/1 81.136.227.216 Gi0/3.203 103.76.3.10 11 0035 A36D 1

Gi0/1 83.169.241.46 Gi0/3.203 103.76.3.10 01 0000 0303 1

Gi0/1 81.88.224.79 Null 103.76.3.10 11 0035 D6E3 1

Gi0/1 81.95.255.78 Null 103.76.3.10 11 0035 8A44 1

Gi0/1 84.201.31.138 Gi0/3.203 103.76.3.10 11 0035 8354 1

Gi0/1 82.69.85.65 Null 103.76.3.10 11 0035 C238 1

Gi0/1 83.102.172.103 Gi0/3.203 103.76.3.10 11 0035 499A 1

Gi0/1 85.94.178.198 Null 103.76.3.10 11 0035 ED52 1

Gi0/1 82.119.108.86 Null 103.76.3.10 11 0035 6279 1

Gi0/1 83.211.83.193 Null 103.76.3.10 11 0035 070D 1

mustafa.chapal · ‎11-26-2016

Hi,

The error 'the fragment table has reached its maximum threshold' has been resolved by increasing the value of ip virtual-reassembly in max-reassemblies 128.

But I am still facing this serious problem of the outside interfaces GE0/1 and GE0/2 input rate being congested. During this problem, I see the input and output rate of the inside interface GE0/3 is normal.

Please take a look at the interfaces below during the problem and let me know if anything seems abnormal.

show interfaces

GigabitEthernet0/0 is up, line protocol is up

Hardware is iGbE, address is xxxx.xxxx.xxxx (bia xxxx.xxxx.xxxx)

Internet address is x.x.x.x/30

MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full Duplex, 1Gbps, media type is ZX

output flow-control is unsupported, input flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:19, output 00:00:08, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/103 (size/max/drops/flushes); Total output drops: 156

Queueing strategy: fifo

Output queue: 0/40 (size/max)

30 second input rate 0 bits/sec, 0 packets/sec

30 second output rate 0 bits/sec, 0 packets/sec

165465809 packets input, 96241093108 bytes, 0 no buffer

Received 18457 broadcasts (0 IP multicasts)

0 runts, 0 giants, 0 throttles

174754 input errors, 0 CRC, 0 frame, 174754 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

68565882 packets output, 33575024181 bytes, 0 underruns

0 output errors, 0 collisions, 3 interface resets

0 unknown protocol drops

0 babbles, 0 late collision, 0 deferred

5 lost carrier, 0 no carrier, 0 pause output

0 output buffer failures, 0 output buffers swapped out

GigabitEthernet0/1 is up, line protocol is up

Hardware is iGbE, address is xxxx.xxxx.xxxx (bia xxxx.xxxx.xxxx)

Internet address is x.x.x.x/30

MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 5/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full Duplex, 1Gbps, media type is RJ45

output flow-control is unsupported, input flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:05, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/214 (size/max/drops/flushes); Total output drops: 531

Queueing strategy: fifo

Output queue: 0/40 (size/max)

30 second input rate 20715000 bits/sec, 6564 packets/sec

30 second output rate 761000 bits/sec, 771 packets/sec

889317237 packets input, 129610395025 bytes, 0 no buffer

Received 232 broadcasts (0 IP multicasts)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

586391384 packets output, 102746524682 bytes, 0 underruns

0 output errors, 0 collisions, 1 interface resets

0 unknown protocol drops

0 babbles, 0 late collision, 0 deferred

4 lost carrier, 0 no carrier, 0 pause output

0 output buffer failures, 0 output buffers swapped out

GigabitEthernet0/2 is up, line protocol is up

Hardware is iGbE, address is xxxx.xxxx.xxxx (bia xxxx.xxxx.xxxx)

Internet address is x.x.x.x/30

MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 9/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full Duplex, 1Gbps, media type is RJ45

output flow-control is unsupported, input flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 411

Queueing strategy: fifo

Output queue: 0/40 (size/max)

30 second input rate 38852000 bits/sec, 11030 packets/sec

30 second output rate 749000 bits/sec, 750 packets/sec

480446100 packets input, 152092730337 bytes, 0 no buffer

Received 18030 broadcasts (0 IP multicasts)

0 runts, 0 giants, 0 throttles

943355 input errors, 0 CRC, 0 frame, 943355 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

527813427 packets output, 87786740725 bytes, 0 underruns

0 output errors, 0 collisions, 1 interface resets

0 unknown protocol drops

0 babbles, 0 late collision, 0 deferred

2 lost carrier, 0 no carrier, 0 pause output

0 output buffer failures, 0 output buffers swapped out

GigabitEthernet0/3 is up, line protocol is up

Hardware is iGbE, address is xxxx.xxxx.xxxx (bia xxxx.xxxx.xxxx)

MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set

Keepalive set (10 sec)

Full Duplex, 1Gbps, media type is RJ45

output flow-control is XON, input flow-control is XON

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 58586783

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 4021000 bits/sec, 2915 packets/sec

5 minute output rate 3882000 bits/sec, 3134 packets/sec

1208651283 packets input, 244947338943 bytes, 0 no buffer

Received 36516 broadcasts (0 IP multicasts)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 2276 pause input

1398699197 packets output, 369932695510 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

120 unknown protocol drops

Georg Pauwen · ‎11-24-2016

Hello,

try and set the 'ip virtual-reassembly in max-reassemblies' value to the highest possible number, 512 or even 1024, and check if the messages disappear.

mustafa.chapal · ‎11-24-2016

Hi,

Will setting this command 'ip virtual-reassembly in max-reassemblies 1024' have any major performance impact on the router processor or memory?

Georg Pauwen · ‎11-24-2016

Hello,

according to Cisco:

-->Performance Impact

VFR causes a performance impact on the basis of functions such as packet copying, fragment validation, and fragment reorder. This performance impact varies depending on the number of concurrent IP datagrams that are being reassembled<--

The real problem is an MTU mismatch which causes the fragmentation. Try to ping the source of your traffic with the -df bit set, and lower the size until the message 'Packet needs to be fragmented but DF set' disappears:

ping x.x.x.x size 1500 df-bit