09-02-2015 09:58 AM - edited 02-21-2020 08:26 PM
I have a network where the client router cannot pass encrypted traffic. I see the SAs form, the DMVPN tunnel is up and I am getting encrypt on one side. And here is where it is confusing, if I remove the transform-set everything is fine.
Home side
crypto isakmp policy 1
encr aes 256
authentication pre-share
crypto isakmp key password address 0.0.0.0
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 30 20 periodic
crypto isakmp diagnose error 10
!
crypto ipsec transform-set LTE ah-sha-hmac esp-aes 256
mode transport
!
crypto ipsec profile LTE
set transform-set LTE
!
interface Tunnel910
ip address 99.0.0.1 255.255.255.0
no ip redirects
ip nbar protocol-discovery
ip flow ingress
ip nhrp authentication TTPS
ip nhrp map multicast dynamic
ip nhrp network-id 2
ip nhrp holdtime 360
ip virtual-reassembly in
load-interval 30
qos pre-classify
tunnel source GigabitEthernet0/0
tunnel mode gre multipoint
tunnel key 1
tunnel protection ipsec profile LTE
Remote Side
crypto isakmp policy 1
encr aes 256
authentication pre-share
crypto isakmp key password address 0.0.0.0
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 30 20 periodic
crypto isakmp diagnose error 10
!
!
crypto ipsec transform-set LTE ah-sha-hmac esp-aes 256
mode transport
!
crypto ipsec profile LTE
set transform-set LTE
!
interface Tunnel910
ip address 99.0.0.2 255.255.255.0
ip nhrp authentication TTPS
ip nhrp map multicast 172.30.1.1
ip nhrp map 99.0.0.1 172.30.1.1
ip nhrp network-id 2
ip nhrp holdtime 360
ip nhrp nhs 99.0.0.1
ip tcp adjust-mss 1250
qos pre-classify
tunnel source Ethernet1/0
tunnel destination 172.30.1.1
tunnel key 1
tunnel protection ipsec profile LTE
The tunnels are up/up on both sides, and when I ping the home side of the tunnel from the remote I get encrypts:
Lynx-2E#ca
Crypto Engine Connections
ID Type Algorithm Encrypt Decrypt LastSeqN IP-Address
31 IPsec SHA+AES256 0 0 0 192.168.0.1
32 IPsec SHA+AES256 1 0 0 192.168.0.1
1005 IKE SHA+AES256 0 0 0 192.168.0.1
Lynx-2E#p 99.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 99.0.0.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
Lynx-2E#ca
Crypto Engine Connections
ID Type Algorithm Encrypt Decrypt LastSeqN IP-Address
31 IPsec SHA+AES256 0 0 0 192.168.0.1
32 IPsec SHA+AES256 6 0 0 192.168.0.1
1005 IKE SHA+AES256 0 0 0 192.168.0.1
But the home side gets an encapsulation failed when I try to ping the remote tunnel IP:
home #p 99.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 99.0.0.2, timeout is 2 seconds:
IP: s=99.0.0.1 (local), d=99.0.0.2 (Tunnel910), len 100, sending
IP: s=99.0.0.1 (local), d=99.0.0.2 (Tunnel910), len 100, output feature, Common Flow Table(27), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: s=99.0.0.1 (local), d=99.0.0.2 (Tunnel910), len 100, output feature, Stateful Inspection(28), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: s=99.0.0.1 (local), d=99.0.0.2 (Tunnel910), len 100, output feature, QoS Preclassification(71), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: s=99.0.0.1 (local), d=99.0.0.2 (Tunnel910), len 100, output feature, Post-Ingress-NetFlow(74), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: s=99.0.0.1 (local), d=99.0.0.2 (Tunnel910), len 100, encapsulation failed.
IP: s=99.0.0.1 (local), d=99.0.0.2 (Tunnel910), len 100, sending
No encrypts:
ca
Crypto Engine Connections
ID Type Algorithm Encrypt Decrypt LastSeqN IP-Address
39 IPsec SHA+AES256 0 0 0 172.30.1.1
40 IPsec SHA+AES256 0 0 0 172.30.1.1
9005 IKE SHA+AES256 0 0 0 172.30.1.1
And with no transform set all is well:
B12_3925(config)#in tu910
B12_3925(config-if)#no tunnel protection ipsec profile LTE
B12_3925(config-if)#
*Sep 2 17:09:26: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is OFF
B12_3925#p 99.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 99.0.0.2, timeout is 2 seconds:
!!!!!
If I put the transform set back on it looks like the home is actually seeing the packets from the remote side:
remote: ping 99.0.0.1
home:
s=192.168.102.130 (GigabitEthernet0/0 [remote tunnel outside IP), d=172.30.1.1 (GigabitEthernet0/0), len 124, rcvd 3
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1, len 124, stop process pak for forus packet
IP: s=172.30.1.1 (local), d=192.168.102.130 (GigabitEthernet0/0), len 124, se
B12_3925#nding
IP: s=172.30.1.1 (local), d=192.168.102.130 (GigabitEthernet0/0), len 124, output feature, Post-Ingress-NetFlow(74), rtype 1, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: s=172.30.1.1 (local), d=192.168.102.130 (GigabitEthernet0/0), len 124, sending full packet
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1, len 188, input feature, MCI Check(94), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: tableid=0, s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1 (GigabitEthernet0/0), routed via RIB
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1 (GigabitEthernet0/0), len 188, rcvd 3
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1, len 188, stop process pak for forus packet
B12_3925#
B12_3925#ca
Crypto Engine Connections
ID Type Algorithm Encrypt Decrypt LastSeqN IP-Address
41 IPsec SHA+AES256 0 0 0 172.30.1.1
42 IPsec SHA+AES256 0 0 0 172.30.1.1
9007 IKE SHA+AES256 0 0 0 172.30.1.1
Any idea why the IPSec does not work?
Full configs, crypto sa and network topo are attached.
Thank you
09-02-2015 10:16 AM
Not sure about the platforms you're running on, but not all platforms support nesting AH and ESP.
Might want to try with just ESP and/or AES-GCM, vide :
http://www.cisco.com/web/about/security/intelligence/nextgen_crypto.html
09-02-2015 10:31 AM
Thanks for the suggestion, I am running the 5921 software router and an ISR G2 3925. The crypto worked across wired, it is just across the cellular link that is causing me issues.
09-02-2015 11:56 AM
Could be some weird padding/fragmentation problem in transit.
" if I remove the transform-set everything is fine" <--- you mean when you remove crypto from DMVPN or use a different transform-set?
09-02-2015 12:00 PM
Correct, if I remove the crypto from the DMVPN.
You are right about the weird padding/frag. If I put my DMVPN inside another tunnel the crypto works! I know that a tunnel in a tunnel is a horrible idea, I was testing a theory about an issue with the underlying infrastructure. I control the eNB and LTE EPC, so it is all my set up.
09-02-2015 12:27 PM
Do you mind trying using pure ESP for test?
You also might also use tunnel path mtu discovery ... to see what happens rather than alleviating anything (do we start receiving ICMP unreachables?)
You can also try to see if you're able to get some specific packet sizes to pass through:
You can calculate from here https://cway.cisco.com/tools/ipsec-overhead-calc/ipsec-overhead-calc.html
Taking a step back I'd like to see, if we're getting encap here and decap on remote end.
The encap failed you indicate in debugs is typically happening when resolution did not work (ARP, NHRP etc), among other scenarios.
09-03-2015 10:04 AM
No joy with esp only:
crypto ipsec transform-set esp-only esp-aes 256
mode transport
!
interface Tunnel800
description VTI in a GRE tunnel (ugly)
ip address 8.0.0.2 255.255.255.0
tunnel source 192.168.0.1
tunnel destination 172.30.1.1
tunnel path-mtu-discovery
tunnel protection ipsec profile esp-only
Lynx-2E#ca
Crypto Engine Connections
ID Type Algorithm Encrypt Decrypt LastSeqN IP-Address
1 IPsec AES256 0 9 0 192.168.0.1
2 IPsec AES256 19 0 0 192.168.0.1
1003 IKE SHA+AES256 0 0 0 192.168.0.1
Lynx-2E#ping 8.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.0.0.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
Lynx-2E#ca
Crypto Engine Connections
ID Type Algorithm Encrypt Decrypt LastSeqN IP-Address
1 IPsec AES256 0 11 0 192.168.0.1
2 IPsec AES256 26 0 0 192.168.0.1
1003 IKE SHA+AES256 0 0 0 192.168.0.1
It looks like it is working, and I see packets on the both sides (below). but no response to pings.
B12_3925#
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1, len 164, input feature, MCI Check(94), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: tableid=0, s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1 (GigabitEthernet0/0), routed via RIB
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1 (GigabitEthernet0/0), len 164, rcvd 3
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1, len 164, stop process pak for forus packet
B12_3925#
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1, len 164, input feature, MCI Check(94), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
IP: tableid=0, s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1 (GigabitEthernet0/0), routed via RIB
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1 (GigabitEthernet0/0), len 164, rcvd 3
IP: s=192.168.102.130 (GigabitEthernet0/0), d=172.30.1.1, len 164, stop process pak for forus packet
B12_3925#
Lynx-2E#
*Sep 3 17:03:07: IP: s=192.16.1.1 (Tunnel800), d=192.16.2.2, len 100, input feature, MCI Check(108), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
*Sep 3 17:03:07: IP: tableid=0, s=192.16.1.1 (Tunnel800), d=192.16.2.2 (Ethernet0/1), routed via RIB
*Sep 3 17:03:07: IP: s=192.16.1.1 (Tunnel800), d=192.16.2.2, len 100, rcvd 4
*Sep 3 17:03:07: IP: s=192.16.1.1 (Tunnel800), d=192.16.2.2, len 100, stop process pak for forus packet
*Sep 3 17:03:07: IP: s=192.16.1.1 (Tunnel800), d=192.16.2.2, len 100, enqueue feature, TCP Adjust MSS(5), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
And if I remove the IPSec it all works:
interface Tunnel800
description VTI in a GRE tunnel (ugly)
ip address 8.0.0.2 255.255.255.0
tunnel source 192.168.0.1
tunnel destination 172.30.1.1
tunnel path-mtu-discovery
tunnel protection ipsec profile esp-only
end
Lynx-2E#c
Enter configuration commands, one per line. End with CNTL/Z.
Lynx-2E(config)#in tu800
Lynx-2E(config-if)#no tunnel protection ipsec profile esp-only
Lynx-2E(config-if)#^Z
Lynx-2E#p
*Sep 3 17:04:01: %SYS-5-CONFIG_I: Configured from console by console
Lynx-2E#p 8.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/36/48 ms
Lynx-2E#p 192.16.1.1 sou 192.16.2.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.16.1.1, timeout is 2 seconds:
Packet sent with a source address of 192.16.2.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 31/40/48 ms
Lynx-2E#
09-03-2015 10:53 AM
Some more info. I am clearly encrypting packets on both sides, and the client (right side of drawing) is decrypting packets from the home side. But the home side is not seeing or decrypting packets from the remote.
home sending to remote:
p 192.16.2.2 sou 192.16.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.16.2.2, timeout is 2 seconds:
Packet sent with a source address of 192.16.1.1
Before encryption:
3CA22C50: 4500007C 02A50000 E..|.%..
3CA22C60: FF2FE463 AC1E0101 C0A86682 00000800 ./dc,...@(f.....
3CA22C70: 45000064 00780000 FF0137FD C0100101 E..d.x....7}@...
3CA22C80: C0100202 0800A7DD @.....'] ...
After encryption:
3CA23610: 4500009C 02A50000 E....%..
3CA23620: FF11E461 AC1E0101 C0A86682 11941194 ..da,...@(f.....
3CA23630: 00880000 39D5AF86 0000002D 00000800 ....9U/....-....
3CA23640: 45000064 00780000 E..d.x.. ...
post_crypto_ip_encrypt: Data just encrypted, 156 bytes
Process switched encrypted packet
Punt packet to process switch.
Remote getting the packet:
*Sep 3 17:50:53: post_crypto_ip_decrypt: Data just decrypted, 124 bytes
*Sep 3 17:50:53: crypto_tun_post_decrypt_switch: oce_les_inline returned 1 with tun_decap_oce 3FA5EE08. punting PS.
*Sep 3 17:50:53: PostDecrypt::crypto_tun: calling process switching
*Sep 3 17:50:53: Punt packet to process switch
*Sep 3 17:50:53: Before encryption:
other way (remote to home):
Remote is encrypting the packets:
p 192.16.1.1 sou 192.16.2.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.16.1.1, timeout is 2 seconds:
Packet sent with a source address of 192.16.2.2
*Sep 3 17:51:50: Before encryption:
41F8F060: 4500007C 02C60000 E..|.F..
41F8F070: FF2F4AC4 C0A80001 AC1E0101 00000800 ./JD@(..,.......
41F8F080: 45000064 003C0000 FF013839 C0100202 E..d.<....89@...
41F8F090: C0100101 080099EC @......l ...
*Sep 3 17:51:50: After encryption:
41FAAD90: 4500009C 02C60000 FF114AC2 C0A80001 E....F....JB@(..
41FAADA0: AC1E0101 11941194 00880000 B6034179 ,...........6.Ay
41FAADB0: 0000003C 00000800 45000064 003C0000 ...<....E..d.<..
41FAADC0: ...
*Sep 3 17:51:50: post_crypto_ip_encrypt: Data just encrypted, 156 bytes
*Sep 3 17:51:50: Process switched encrypted packet
*Sep 3 17:51:50: Punt packet to process switch.
*Sep 3 17:51:52: Before encryption:
home side sees nothing.
09-04-2015 12:41 AM
You might want to try EPC to check if you're seeing those packets ingressing/egressing correctly ... but if you do ... "Dear ISP"?
http://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/ios-embedded-packet-capture/116045-productconfig-epc-00.html
https://supportforums.cisco.com/document/139686/configuration-example-embedded-packet-capture-cisco-ios-and-ios-xe
09-03-2015 09:53 AM
99.0.0.1 and 99.0.0.2 are your public ip addresses. Am i correct?
after you remove your ipsec profile, the crypto is disabled; hence pings from one end to 99.0.0.2 are going to go out through internet.
can you check your DMVPN config once again.
the tunnel interface ip would be some random private ip address on both the ends in the same subnet and the tunnel destination and source would be your public ip addresses.
09-03-2015 10:09 AM
99.x.x.x are public IPs, but this is a stand alone network with no access to the Internet.
I have a static route to the far side of each router that uses the tunnel far IP as the destination, it does not change when I remove the crypto.
I am using VTIs now. I was testing with VTI and DMVPN, but the VTI is just easier to set up quickly.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide