can not ping packet size larger than 9200 over IPSec vpn on ASR 1002

[cciesec2011]

I have an existing site-2-site VPN between a Cisco 2621 router (IOS 12.3) and Cisco 1841 (IOS 12.3) and I can ping packet size of 17000 over the IPSec tunnel without any issue:

c2621#ping 192.168.230.254 source f0/1 repeat 20 size 17000

Type escape sequence to abort.
Sending 20, 17000-byte ICMP Echos to 192.168.230.254, timeout is 2 seconds:
Packet sent with a source address of 192.168.208.254
!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (20/20), round-trip min/avg/max = 144/146/148 ms
c2621#

I replaced the Cisco 2621 with a more powerful ASR 1002 running IOS version asr1000rp1-adventerprisek9.03.01.00.S.150-1.S.bin.  However, I can not ping packet size larger than 9200 over the IPSec tunnel:

Feb 24 02:42:52.362: %IOSXE-3-PLATFORM: F0: cpp_cp: QFP:00 Thread:015 TS:00000015834854465792
%IPSEC-3-PKT_TOO_BIG: IPSec Packet size 10072 larger than maximum supported size 9216 hence dropping it.........
Success rate is 0 percent (0/10)
asr1002#

Why is not working?  Basically the more expensive ASR router can not perform the same task as the old Cisco 2621 router.

Anyone know why?

[Marcin Latosiewicz]

Hi,

I saw you already opened SR 616946789 for this.

Just a question regarding how impacting this excercise is. Even with jumbo frames you will not be able to send more than 9000 bytes of data...so in the end you will end up with fragmentation regardless of scenario.

Honestly I'm not sure why there is this problem if no df-bit is set in header

And on regular basis software and hardware platforms should behave the same.

That being said is there a real world application for this excercise, call it professional curiosity :-)

Marcin

[cciesec2011]

"And on regular basis software and hardware platforms should behave the same."

The key word is "should" but in reality it does not

I have an application that will be sending large multicast packets, >9200, over the VPNs and it is not working.

Something very interesting, because it did not work, I decided to reboot the ASR and after that I was able to send packet >9200 over the VPNs

I spoke to TAC and the engineer told me that she does not know what the issue is.  Searching Cisco internal database does not show any errors or information of the errors I provided.  Basically, she told me that there is nothing she can do.  Next time, if it happens, go ahead and open a priority 1 case so that cisco TAC engineer can webex in and run debug commands but other than that, I am hitting a brick wall.

The case was opened as a Priority 3 case and it took TAC like two days to get back to me with no useful information, and by the time she could get back

to me, the issue is already gone.  It would have been helpful if TAC is more responsive.

This makes me very "nervous" when I roll this out into production because it may stop working for no reason and no explaination.

[Marcin Latosiewicz]

Gotcha,

If you run into those problems again first thing you should check is:

sh pl ha qf ac fe ipsec data drop
sh plat hard qfp act stat drop | e _0_

That's the first thing TAC engineer should check anyway.

Marcin

[cciesec2011]

asr1002#sh pl ha qf ac fe ipsec data drop

------------------------------------------------------------------------

Drop Type  Name                                     Packets

------------------------------------------------------------------------

        4  IN_US_V4_PKT_SA_NOT_FOUND_SPI                              15

       25  IN_INFRA_V4_PKT_LEN_TOO_BIG                                37

asr1002#sh plat hard qfp act stat drop | e _0_----------------------------------------------------------------Global Drop Stats                         Packets        Octets---------------------------------------------------------------IpsecInput                                     54        544644Ipv4Martian                                    16          1324Ipv4NoRoute                                     2           130UnconfiguredIpv4Fia                            12           744

asr1002#

Mar  1 09:57:02.422: %IOSXE-3-PLATFORM: F0: cpp_cp: QFP:00 Thread:020 TS:00000029146589775578 %IPSEC-3-PKT_TOO_BIG: IPSec Packet size 10072 larger than maximum supported size 9216 hence dropping it

asr1002#

[Marcin Latosiewicz]

Awesome,

So it comes back after a time. Maybe a phase 2 or phase 1 rekey?

IN_INFRA_V4_PKT_LEN_TOO_BIG - seems to be the reason. 

In my opionion this could use some investigation by development.

Marcin

[cciesec2011]

it came back after I reboot the ASR1002.  Traffics flow fine between the VPN tunnel. except traffics > 9200.

In other words, the expensive ASR can not perform the job like the obsolete 2621 but reliable non-supported router

I will follow up with TAC this morning.  Thanks.