Hello all,
I have following setup. 2 main sites (Site A and Site B) (VPN concentrator) and several remote sites.
Each remote site have a GRE tunnel with 2 ipsec tunnels to each main site.
On one of the remotes site we got compalins and we found out packets drops on the ipsec tunnel between this remote site and one on the main site (Let say site A), while ipsec tunnel to site B is clean.
Ping between site A and remote site on the internet (using public ip address) is ok with no packet drops. So it is not ISP issue. my ping test over ipsec tunnel is with several packet size starting 60 bytes and I always see packet drops.
Here is the configs (Partial)
Site A: 7204vxr 12.4(24)T2
===================
interface Tunnel10
ip address 173.32.8.2 255.255.255.0
no ip redirects
ip flow ingress
ip nhrp authentication XXXXXX
ip nhrp map multicast dynamic
ip nhrp map 173.32.8.1 203.14.112.189
ip nhrp map multicast 203.14.112.189
ip nhrp network-id 10
ip nhrp holdtime 600
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 10
tunnel protection ipsec profile DMVPN_PROF
interface GigabitEthernet0/1
description ISP 1
ip address 89.3.11.244 255.255.255.248
ip flow ingress
ip nat inside
ip virtual-reassembly
ip route-cache policy
ip policy route-map clear-df
duplex full
speed 1000
media-type rj45
negotiation auto
Site B: 7204vxr 12.4(24)T2
===================
interface Tunnel10
ip address 173.32.8.1 255.255.255.0
no ip redirects
ip flow ingress
ip nhrp authentication XXXXX
ip nhrp map multicast dynamic
ip nhrp network-id 10
ip nhrp holdtime 600
tunnel source GigabitEthernet0/1
tunnel mode gre multipoint
tunnel key 10
tunnel protection ipsec profile DMVPN_PROF
interface GigabitEthernet0/1
description INTERNET ISP2
ip address 89.3.18.116 255.255.255.248
ip flow ingress
duplex full
speed 1000
media-type rj45
negotiation auto
Remote site: 1841 12.4(15)T7
=====================
interface Tunnel3
description Connection to VPN ConC
bandwidth 256
ip address 173.32.8.87 255.255.255.0
no ip redirects
ip mtu 1300
ip nhrp authentication XXXXXX
ip nhrp map 173.33.8.1 89.3.18.116
ip nhrp map multicast 89.3.18.116
ip nhrp map 173.33.8.2 89.3.11.244
ip nhrp map multicast 89.3.11.244
ip nhrp network-id 3
ip nhrp holdtime 600
ip nhrp nhs 173.33.8.1
ip nhrp nhs 173.33.8.2
tunnel source FastEthernet0/0
tunnel mode gre multipoint
tunnel key 3
tunnel path-mtu-discovery
tunnel protection ipsec profile DMVPN_PROF
tunnel bandwidth transmit 128
tunnel bandwidth receive 256
interface FastEthernet0/0
description Connection to ISP 3
bandwidth 256
ip address 87.45.27.130 255.255.248.0
speed 100
full-duplex
All was working fine until few days ago.
There is many other remote site with simlar setup working fine with no drops.
When I look to the VPN concentrator with drops Site A, I am seeing:
SiteA#show pas isa interface
VPN Acceleration Module Version II+ in slot : 3
Statistics for Hardware VPN Module since the last clear
of counters 3974 seconds ago
74969347 packets in 74969346 packets out
41510632572 bytes in 41136609103 bytes out
18864 paks/sec in 18864 paks/sec out
83561 Kbits/sec in 82808 Kbits/sec out
0 pkts compressed 0 pkts not compressed
0 bytes before compress 0 bytes after compress
1.0:1 compression ratio 1.0:1 overall
43708 commands out 43708 commands acknowledged
Last 5 minutes:
5735788 packets in 5735786 packets out
19119 paks/sec in 19119 paks/sec out
88939410 bits/sec in 88086550 bits/sec out
Errors:
ppq full errors : 513 ppq rx errors : 11 <<<<<<<<<<<<<<<< ???
cmdq full errors : 0 cmdq rx errors : 0
ppq down errors : 0 cmdq down errors : 0
no buffer : 0 replay errors : 47800 <<<<<<<<<<<<<<< ???
dest overflow : 0 authentication errors : 17
Other error : 0 Raw Input Underrun : 11
IPSEC Unsupported Option: 0 IPV4 Header Length : 0
ESP Pad Length : 11 IPSEC Decompression : 0
AH ESP seq mismatch : 0 AH Header Length : 0
AH ICV Incorrect : 0 IPCOMP CPI Mismatch : 0
IPSEC ESP Modulo : 0 Unexpected IPV6 Extensio: 0
Unexpected Protocol : 0 Dest Buf overflow : 0
IPSEC Pkt is fragment : 0 IPSEC Pkt src count : 0
Invalid IP Version : 0 Unwrappable : 0
SSL Output overrun : 0 SSL Decompress failure : 0
SSL BAD Decomp History : 0 SSL Version Mismatch : 0
SSL Input overrun : 0 SSL Conn Modulo : 0
SSL Input Underrun : 0 SSL Connection closed : 0
SSL Unrecognised content: 0 SSL record header length: 0
PPTP Duplicate packet : 0 PPTP Exceed max missed p: 0
RNG self test fail : 0 DF Bit set : 0
Hash Miscompare : 0 Unwrappable object : 0
Missing attribute : 0 Invalid attrribute value: 0
Bad Attribute : 0 Verification Fail : 0
Decrypt Failure : 0 Invalid Packet : 0
Invalid Key : 0 Input Overrun : 0
Input Underrun : 0 Output buffer overrun : 0
Bad handle value : 0 Invalid parameter : 0
Bad function code : 0 Out of handles : 0
Access denied : 0 Out of memory : 0
NR overflow : 0 pkts dropped : 541
Warnings:
sessions_expired : 0 packets_fragmented : 0
general : 0 compress_bypassed : 0
HSP details:
hsp_operations : 254044712 hsp_sessions : 799
When I clear counters I still see "ppq full errors" and "replay errors" increasing on Site A while it is clean with Site B.
Any one knows what those counters mean and if they are related to problem I am seeing?
Thanks in advance.
