Nat Using Loopback and Routing Across ACL Based IPSEC Tunnel

mhillab · ‎06-14-2023

Hi Team....Thanks in advance for your expertise

Goal:

Nat internal traffic from a smtp server 10.0.0.15 routed to a Loopback0 on my CSR1000V for NAT and then down the tunnel to the remote SMTP server. The remote server must be able to initiate the same connection in reverse with the target as the Loopback IP.

Diagram
PartnerSMTPServer(70.31.121.254) <-> PartnerASA(100.31.119.10) <-----Policy/ACL/CryptoMap Based IPSEC Tunnel--------> MyCSR8000V-GE1(90.127.39.157) <-> Loopback0(50.30.2.52) <-> GE4(10.0.0.5) <-> MySMTPServer(10.0.0.15)

My Router Configs:

crypto ikev2 proposal GLR_IKEV2_PROP1
encryption aes-cbc-256
integrity sha256
group 14
!
crypto ikev2 policy GLR_IKEV2_POLICY
match fvrf any
proposal GLR_IKEV2_PROP1
!
!
crypto ikev2 keyring GLR_KEYRING
peer GLR
address 100.31.119.10
pre-shared-key local xxxxxxxxxxxxxxxxxxxx
pre-shared-key remote xxxxxxxxxxxxxxxxxx
!
!
crypto ikev2 profile GLR_IKEV2_PROFILE
match identity remote address 100.31.119.10 255.255.255.255
authentication remote pre-share
authentication local pre-share
keyring local GLR_KEYRING
dpd 20 10 on-demand
!
!
crypto ipsec transform-set GLR_TSET esp-aes 256 esp-sha256-hmac
mode tunnel
!
!
crypto map cmap 10 ipsec-isakmp
set peer 100.31.119.10
set security-association lifetime seconds 1800
set transform-set GLR_TSET
set ikev2-profile GLR_IKEV2_PROFILE
match address GLR_CRYPTO_ACL
!
!
interface Loopback0
description GLR Public Target (NAT)
ip address 50.30.2.52 255.255.255.255
ip nat outside
!
!
!
interface GigabitEthernet1
ip address dhcp
negotiation auto
ipv6 address dhcp
ipv6 enable
ipv6 nd autoconfig default-route
no mop enabled
no mop sysid
crypto map cmap
!
interface GigabitEthernet4
ip address dhcp
ip nat inside
negotiation auto
no mop enabled
no mop sysid
!
ip nat inside source static 10.0.0.15 interface Loopback0
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 10.0.200.1
!
!
ip access-list extended GLR_CRYPTO_ACL
10 permit ip host 50.30.2.52 host 70.31.121.254 log
!
!

CSR8000V Notes:
-Cisco CSR8000V hosted in AWS
-GE01 has a Elastic Public IP attached - 90.127.39.157 and is the target for the partners VPN. AWS handles the nat to public IP
-Cisco CSR8000V does not support NAT on the same interface using a Crypto MAP (stated in documentation and by TAC) and causes performance issues in virtual appliances
-The CSR does not route any internal traffic to the internet other than its own traffic to licensing servers, ie no overload/PAT. Strictly used for tunneling SMTP

Partner Notes
-IPSEC VPN must use CryptoMap per partner requirements and will not support VTI or DMVPN
-Server IPs must be public and not private ips routed over the tunnel. Was hoping to use a Loopback address to NAT and not apply NAT on GE1 as this has the CMAP applied

VPN Status
-I can bring the VPN up when doing an extended ping from the Loopback0 IP as the source to create interesting traffic

MySMTPServer Notes:
Has a route to the partner server via 10.0.0.5
Can ping GE4 of MyCSR8000V 10.0.0.5
Traceroute to 70.31.121.254 hits 10.0.0.5 then stops

The issue is that I cannot get the traffic from MySMTPServer(10.0.0.15) to NAT through the Loopback0 IP. Nat statistics show all zeros.

I am clearly missing something or misunderstanding how NAT works on a Loopback especially when wanting it to route across an ACL based IPSEC tunnel. I injected the Loopback so I can apply "ip nat outside" to that interface instead of GE1 as the CSR does not support NAT and CMAP on the same interface. I thought this would be simple...

Any help is super appreciated.

MHM Cisco World · ‎06-14-2023

NATing is prefect

interface GigabitEthernet4
ip address dhcp
ip nat inside
ip policy route-map MHM
!
route-map MHM permit 10
match address 100
set ip next-hop LO-IP
!
ip access-list extended 100
permit ip host <local Host before NATing> host <remote Host>

mhillab · ‎06-14-2023

Thank you much for the info and super quick response

This seems to make sense....

I added the configs you suggested (see below for new full configs with suggested changes)

From the local SMTP server, when I ping the remote SMTP server, it replies, but it seems GE4 or Loopback0 is what is replying. (.277 ms)

The new configs do not generate "interesting traffic" to open the tunnel (it was down). I also tried opening the tunnel with extended ping using the source as Looback0 (tunnel came up) but no NAT statistics or routing over the tunnel.

New Configs:

crypto ikev2 proposal GLR_IKEV2_PROP1
encryption aes-cbc-256
integrity sha256
group 14
!
crypto ikev2 policy GLR_IKEV2_POLICY
match fvrf any
proposal GLR_IKEV2_PROP1
!
!
crypto ikev2 keyring GLR_KEYRING
peer GLR
address 100.31.119.10
pre-shared-key local xxxxxxxxxxxxxxxxxxxx
pre-shared-key remote xxxxxxxxxxxxxxxxxx
!
!
crypto ikev2 profile GLR_IKEV2_PROFILE
match identity remote address 100.31.119.10 255.255.255.255
authentication remote pre-share
authentication local pre-share
keyring local GLR_KEYRING
dpd 20 10 on-demand
!
!
crypto ipsec transform-set GLR_TSET esp-aes 256 esp-sha256-hmac
mode tunnel
!
!
crypto map cmap 10 ipsec-isakmp
set peer 100.31.119.10
set security-association lifetime seconds 1800
set transform-set GLR_TSET
set ikev2-profile GLR_IKEV2_PROFILE
match address GLR_CRYPTO_ACL
!
!
interface Loopback0
description GLR Public Target (NAT)
ip address 50.30.2.52 255.255.255.255
ip nat outside
!
!
!
interface GigabitEthernet1
ip address dhcp
negotiation auto
ipv6 address dhcp
ipv6 enable
ipv6 nd autoconfig default-route
no mop enabled
no mop sysid
crypto map cmap
!
interface GigabitEthernet4
ip address dhcp
ip nat inside
ip policy route-map GLR_NAT_RMAP
negotiation auto
no mop enabled
no mop sysid
!
ip nat inside source static 10.0.0.15 interface Loopback0
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 10.0.200.1
!
!
ip access-list extended GLR_CRYPTO_ACL
10 permit ip host 50.30.2.52 host 70.31.121.254 log
!
!
ip access-list extended 100
10 permit ip host 10.0.0.15 host 70.31.121.254 log
!
route-map GLR_NAT_RMAP permit 10
match ip address 100
set ip next-hop 50.30.2.52

sh ip nat translations and statistics:

Pro Inside global Inside local Outside local Outside global
--- 3.30.2.52 10.0.0.15 --- ---
Total number of translations: 1

Total active translations: 0 (0 static, 0 dynamic; 0 extended)
Outside interfaces:
Loopback0
Inside interfaces:
GigabitEthernet4
Hits: 0 Misses: 0
Expired translations: 0
Dynamic mappings:

Thank you again for all of your expertise....

MHM Cisco World · ‎06-14-2023

Pro Inside global Inside local Outside local Outside global
--- 3.30.2.52 10.0.0.15 --- ---

50.30.2.52 <<-
there is different in IP, this is typo ?

mhillab · ‎06-14-2023

I scrubbed the IPs from the outputs for various reasons, missed that one. Busted. 50.30.2.52 is really 3.30.2.52 throughout the whole config, but hiding that is really not important.

mhillab · ‎06-14-2023

Updated configs here , no IP scrubbing. They are not publicly accessible anyway.

crypto ikev2 proposal GLR_IKEV2_PROP1
encryption aes-cbc-256
integrity sha256
group 14
!
crypto ikev2 policy GLR_IKEV2_POLICY
match fvrf any
proposal GLR_IKEV2_PROP1
!
!
crypto ikev2 keyring GLR_KEYRING
peer GLR
address 100.31.119.10
pre-shared-key local xxxxxxxxxxxxxxxxxxxx
pre-shared-key remote xxxxxxxxxxxxxxxxxx
!
!
crypto ikev2 profile GLR_IKEV2_PROFILE
match identity remote address 100.31.119.10 255.255.255.255
authentication remote pre-share
authentication local pre-share
keyring local GLR_KEYRING
dpd 20 10 on-demand
!
!
crypto ipsec transform-set GLR_TSET esp-aes 256 esp-sha256-hmac
mode tunnel
!
!
crypto map cmap 10 ipsec-isakmp
set peer 100.31.119.10
set security-association lifetime seconds 1800
set transform-set GLR_TSET
set ikev2-profile GLR_IKEV2_PROFILE
match address GLR_CRYPTO_ACL
!
!
interface Loopback0
description GLR Public Target (NAT)
ip address 3.30.2.52 255.255.255.255
ip nat outside
!
!
!
interface GigabitEthernet1
ip address dhcp
negotiation auto
ipv6 address dhcp
ipv6 enable
ipv6 nd autoconfig default-route
no mop enabled
no mop sysid
crypto map cmap
!
interface GigabitEthernet4
ip address dhcp
ip nat inside
ip policy route-map GLR_NAT_RMAP
negotiation auto
no mop enabled
no mop sysid
!
ip nat inside source static 10.0.0.15 interface Loopback0
ip route 0.0.0.0 0.0.0.0 GigabitEthernet1 10.0.200.1
!
!
ip access-list extended GLR_CRYPTO_ACL
10 permit ip host 3.30.2.52 host 70.31.121.254 log
!
!
ip access-list extended 100
10 permit ip host 10.0.0.15 host 70.31.121.254 log
!
route-map GLR_NAT_RMAP permit 10
match ip address 100
set ip next-hop 3.30.2.52

MHM Cisco World · ‎06-14-2023

Debup ip policy <<- share this

mhillab · ‎06-15-2023

Hello.

I enabled debug as suggested but sh log comes up empty for this.

The RMAP does not seem to apply. I did some testing and adding a config like "ip local policy route-map GLR_NAT_RMAP" does start generating debug logs, but the logs are forwarding Goddady Trustpoint )default SSL cert to the loopback, so obviously a rework or different approach would be needed.

Not sure if ip policy debugging is possible with a route-map applied to a nat statement or an interface. I think you need to enable the policy globally as i did in my tests. Currently running Cisco IOS XE Software, Version 17.06.05

MHM Cisco World · ‎06-15-2023

set ip next-hop 3.30.2.52 <<- instead of using IP use interface loopback
remove log from the ACL of route-map

Note:- make double check ip you use in the acl of route-map
do this change and check again

mhillab · ‎06-15-2023

Thank you. I made the suggested changes.

Relevant configs here:

interface GigabitEthernet4
ip address dhcp
ip nat inside
ip policy route-map GLR_NAT_RMAP
negotiation auto
no mop enabled
no mop sysid

ip nat inside source static 10.0.0.15 interface Loopback0

ip access-list extended 100
10 permit ip host 10.0.0.15 host 70.31.121.254

route-map GLR_NAT_RMAP permit 10
match ip address 100
set interface Loopback0

Tried to create some interesting traffic from the local server, but nothing interesting in the logs and NAT hits are still zero.

I did have a few other debugs turned on

Packet Infra debugs:

Ip Address Port
------------------------------------------------------|----------

Route-map:
Routemap related IPC debugging is on
Routemap related API debugging is on
Generic IP:
IP NAT debugging is on for access list NAT

Policy Routing:
Policy routing debugging is on

Here is what is in the sh logs when I made the changes requested.

Jun 15 18:30:05.243: PBR CP: SET INTERFACE ADD
interface: Loopback0

Jun 15 18:30:05.243: PBR CP: Not a P2P interface - Not used for PBR!

Jun 15 18:31:07.525: IPNAT: push route-map command for GLR_NAT_RMAP seq 10, type 9
Jun 15 18:31:07.526: PBR CP: SET INTERFACE ADD
interface: Loopback0

Jun 15 18:31:07.526: PBR CP: Not a P2P interface - Not used for PBR!

Jun 15 18:31:07.526: PBR Control Plane Notification: PBR_CP_INVALID - shut off all

Jun 15 18:31:07.526: Policy NextHop Inquiry: GLR_NAT_RMAP seq: 10, type: INVALID SW_OBJ_TYPE: 0, SW_HANDLE: 0x0

Jun 15 18:31:07.526: PBR CP Notification sent: Type:INVALID, UNKNOWNSW_OBJ_TYPE: 0, SW_HANDLE: 0x0

Jun 15 18:31:07.526: PR-RP: Set GigabitEthernet4 policy_routemap=GLR_NAT_RMAP; cached_map=GLR_NAT_RMAP
Jun 15 18:31:10.236: %SYS-5-CONFIG_I: Configured from console by ec2-user on vty0 (50.206.66.85)
Jun 15 18:31:14.693: %SYS-6-PRIVCFG_ENCRYPT_SUCCESS: Successfully encrypted private config file
Jun 15 18:33:35.780: NAT: Dynamic PAT mapping id 0 add via config
Jun 15 18:33:35.780: NAT: Mapping address 3.30.2.52 for interface Loopback0
Jun 15 18:33:35.780: NAT: Entry assigned id 9
Jun 15 18:33:35.780: NAT: Mapping assigned id 6
Jun 15 18:33:35.780: ipnat_add_dynamic_pat_cfg: id 6, flag 5
Jun 15 18:38:04.695: PR-RP: Set GigabitEthernet4 policy_routemap=GLR_NAT_RMAP; cached_map=GLR_NAT_RMAP
Jun 15 18:39:08.564: PR-RP: Set GigabitEthernet4 policy_routemap=GLR_NAT_RMAP; cached_map=GLR_NAT_RMAP

MHM Cisco World · ‎06-15-2023

Router#show rn
Router#show ru
Router#show running-config
Building configuration...

Current configuration : 1698 bytes
!
! Last configuration change at 23:28:49 UTC Thu Jun 15 2023
!
version 16.8
service timestamps debug datetime msec
service timestamps log datetime msec
platform qfp utilization monitor load 80
no platform punt-keepalive disable-kernel-core
platform console serial
!
hostname Router
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
!
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
license udi pid CSR1000V sn 9Q9T2TFH562
no license smart enable
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
!
!
!
redundancy
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
!
interface GigabitEthernet1
ip address 100.0.0.1 255.255.255.0
ip nat outside
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet2
no ip address
shutdown
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
no ip address
shutdown
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet4
ip address 10.0.0.1 255.255.255.0
ip nat inside
ip policy route-map MHM
negotiation auto
no mop enabled
no mop sysid
!
ip nat inside source list 1 interface Loopback0 overload
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
!
!
access-list 1 permit 10.0.0.0 0.0.0.255
access-list 100 permit ip 10.0.0.0 0.0.0.255 any
!
!
route-map MHM permit 10
match ip address 100
set interface Loopback0
!
!
!
control-plane
!
!
!
!
!
!
line con 0
stopbits 1
line vty 0 4
login

I run lab and as you can see the traffic is NATing to LO after I redirect traffic to LO via PBR