VPN tunnel send errors

[Steve Coady]

Hello

I have an VPN with satellite office. User's are reporting Session drops with my server @ DC. User's are able to reconnect to session and pick up where they left off. That makes it seem as if the server did not realize they were not still connected. Can anyone provide some expert technical guidance on the puzzling issue?

What could be causing these errors?

Router# 891

crypto isakmp policy 1
 encr 3des
 authentication pre-share
 group 2
crypto isakmp key 6 !#@$#^$%@(@*#) address (Remote Peer IP)
!
!
crypto ipsec transform-set MY esp-3des esp-sha-hmac
!
crypto map MY_VPN 10 ipsec-isakmp
 set peer (Remote Peer IP)
 set transform-set MY
 match address ACL_MY_VPN

interface FastEthernet8
 description OUTSIDE THEIRS internet
 ip address (Ip address) 255.255.255.128
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 duplex auto
 speed auto
 crypto map MY_VPN

ip access-list extended ACL_MY_VPN
 permit ip 10.x.220.0 0.0.0.255 any
 permit ip 10.x.221.0 0.0.0.255 any
 permit ip 10.x.222.0 0.0.0.255 any

ROUTER891#sh crypto ipsec sa

interface: FastEthernet8
    Crypto map tag: MY_VPN, local Peer addr (ip address x.x.x.x)

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (10.x.220.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer (ip address z.z.z.z) port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 887637, #pkts encrypt: 887637, #pkts digest: 887637
    #pkts decaps: 992370, #pkts decrypt: 992370, #pkts verify: 992370
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 8, #recv errors 0

     local crypto Peer endpt.: (ip address x.x.x.x), remote crypto Peer endpt.: (ip address z.z.z.z)
     path mtu 1500, ip mtu 1500, ip mtu idb FastEthernet8
     current outbound spi: 0xED21C8D0(3978414288)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0x3FC8FC31(1070136369)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 37, flow_id: Onboard VPN:37, sibling_flags 80000046, crypto map: MY_VPN
        sa timing: remaining key lifetime (k/sec): (4580137/3089)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0xED21C8D0(3978414288)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 38, flow_id: Onboard VPN:38, sibling_flags 80000046, crypto map: MY_VPN
        sa timing: remaining key lifetime (k/sec): (4580098/3089)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (10.x.221.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer (ip address z.z.z.z) port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 18753258, #pkts encrypt: 18753258, #pkts digest: 18753258
    #pkts decaps: 16815594, #pkts decrypt: 16815594, #pkts verify: 16815594
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 3438, #recv errors 0

     local crypto Peer endpt.: (ip address x.x.x.x), remote crypto Peer endpt.: (ip address z.z.z.z)
     path mtu 1500, ip mtu 1500, ip mtu idb FastEthernet8
     current outbound spi: 0x58A07B0(92932016)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0xAE72C947(2926758215)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 41, flow_id: Onboard VPN:41, sibling_flags 80000046, crypto map: MY_VPN
        sa timing: remaining key lifetime (k/sec): (4398843/3128)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0x58A07B0(92932016)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 42, flow_id: Onboard VPN:42, sibling_flags 80000046, crypto map: MY_VPN
        sa timing: remaining key lifetime (k/sec): (4398881/3128)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (10.x.222.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer (ip address z.z.z.z) port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 3500166, #pkts encrypt: 3500166, #pkts digest: 3500166
    #pkts decaps: 3654295, #pkts decrypt: 3654295, #pkts verify: 3654295
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 1565, #recv errors 0

     local crypto Peer endpt.: (ip address x.x.x.x), remote crypto Peer endpt.: (ip address z.z.z.z)
     path mtu 1500, ip mtu 1500, ip mtu idb FastEthernet8
     current outbound spi: 0xDE39F6D3(3728340691)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0xDFC9D904(3754547460)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 39, flow_id: Onboard VPN:39, sibling_flags 80000046, crypto map: MY_VPN
        sa timing: remaining key lifetime (k/sec): (4547977/3107)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0xDE39F6D3(3728340691)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 40, flow_id: Onboard VPN:40, sibling_flags 80000046, crypto map: MY_VPN
        sa timing: remaining key lifetime (k/sec): (4547976/3107)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:
ROUTER# 891

[Marcin Latosiewicz · Accepted solution]

Send errors increase when traffic was matched to a policy but IPsec SA was not yet up. I'm not aware of any other circumstance. 

It's a long shot but it's possible that there is phase 2 rekey not taking place or not taking place fast enough which causes some traffic to be dropped while new IPsec SA is being negotiated. This could explain what the users are seeing, but ... it's a long shot. 

I would look for other reasons first. 

[Steve Coady]

Marcin

Thank you for the response.

I will keep looking. However, is there a command to use to check on whether this phase 2 rekey is having troubles?

One other detail is this particular user is able to work on her pc fine for approx 1 hours. Work starts at 10:00 am and trouble usually starts around 11:00 am and the re-appears approx every 20 mins.

[Steve Coady]

One other detail

I do not see any receive or send errors on other side of tunnel

[Marcin Latosiewicz · Accepted solution]

Steve, 

If bulk of traffic is coming from this side of the tunnel it would be expected.

However you do raise a valid point - timing. By default IPsec SA lifetime in seconds is 1 hour. However another way for IPsec SA to expire is too much data sent (there's also a kilobyte lifetime value). 

"crypto logging session" would be a good way to start monitoring the tunnel. 

For the rest I would also setup some SLAs/tracking to see what is or is not affected during the problem. 

M.

[Steve Coady]

Marcin

Thank you for the guidance.