Global to VRF NAT Issue

[ebng]

TLDR: When I try to NAT from the global routing instance to a VRF using an IP address in the same subnet as the router, the router doesn't reply to ARP requests for the NAT'd IP.

I'm having some issues with one-to-one static NAT entries when going from the global routing instance to a VRF and could use some help.  I originally had this issue with an ISR 44551-X, but have since tried testing it out in the lab using a CSR1000V and have the same issues.  I've tried to keep things pretty simple.  I have a server with an IP address of 100.19.0.52 attached to interface GigabitEthernet2 on my CSR1000V (call it R1).  Interface GigabitEthernet2 is part of the global routing instance and is my inside NAT interface:

interface GigabitEthernet2
ip address 100.19.0.33 255.255.255.224
ip nat inside
negotiation auto
no mop enabled
no mop sysid

Interface GigabitEthernet3 is in the EXTERNAL vrf and is my outside NAT interface:

interface GigabitEthernet3
ip vrf forwarding EXTERNAL
ip address 200.168.1.1 255.255.254.0
ip nat outside
negotiation auto
no mop enabled
no mop sysid

I want my internal server to be able to reach equipment in the same subnet as GigabitEthernet3, so I've created the following static route:

ip route 200.168.0.0 255.255.254.0 GigabitEthernet3

I want that traffic from that server to NAT to an IP address also in the same subnet as GigabitEthernet3 so I have the following statement:

ip nat inside source static 100.19.0.52 200.168.1.190

Now, GigabitEthernet3 is connected to another CSR1000V router (we'll call it R2) with an IP address of 200.168.1.252.  When I try to ping R2 from my server, I can see the ICMP request make it to R2.  R2, in turn, sends an ARP request for 200.168.1.252, but receives no response from R1.  After reading another post, I tried putting in the following on R1 to attempt to resolve the issue:

arp vrf OUTSIDE 200.168.1.190 <MAC address of GigabitEthernet2> arpa alias

When I did this, R1 would respond to ARP requests from R2 for 200.168.1.190, allowing R2 to send ICMP replies.  Unfortunately, those replies did not make it back to the server so I removed the command.  As one final test, I removed the nat command from above and replaced it with the following:

ip route 200.168.0.0 255.255.254.0 GigabitEthernet3

When I do that, my server can now ping R2, but this doesn't exactly solve my problem.  At this point, I'm fairly stuck and could use some assistance.   I'm attaching my complete config in case that sheds light on anything.  I appreciate any help I can get on this.  Thanks in advance.

[paul driver]

Hello
Is there any reason why you cannot add GigabitEthernet2 to VRF OUTSIDE because your natting on the public address anyway as such hiding 10.10.100.0/24?

[ebng]

Exactly.

[Georg Pauwen]

Hello,

typically, for VRF to Global, you would have to add the VRF to the NAT statement:

ip nat inside source static 100.19.0.52 200.168.1.190 vrf OUTSIDE

[ebng]

When I issue that statement, the traffic doesn't NAT out at all and see ICMP requests coming from 100.19.0.52 arriving at R2.

[paul driver]

Hello

Revisiting this again ive noticed that you have your default route pointing towards a different VRF then your natted outside vrf interface and that your internal lan is in neither of these, As such your static routing is incorrect because the global route table where your lan users reside are not aware of the other networks in either vrf and vice versa.

So can you confirm do you wish for your lan users to route via the MGT vrf but have a a specific static nat entry towards the vrf OUTSIDE .

[ebng]

Ya, I'm doing exactly what I want to be doing here.  This is an incredibly simplified representation of my actual environment so I can narrow down this one issue.  I hesitated to even include the complete config for my lab setup as I figured it would cause some confusion, but knew someone would ask for it anyway.  The MGMT VRF is simply for management of R1. 

[ebng]

To add one more piece of data here, if I drop the OUTSIDE VRF altogether and have both interfaces Gig2 and Gig3 in the main routing instance, this works just fine and R1 will reply to ARP requests for 200.168.1.190. 

[Georg Pauwen]

Hello,

just one thing: in the configuration you have posted, you have the below:

ip vrf EXTERNAL
rd 7683763:7

!

interface GigabitEthernet3
ip vrf forwarding OUTSIDE
ip address 200.168.1.1 255.255.254.0
ip nat outside
negotiation auto
no mop enabled
no mop sysid

Did you edit this, or did you leave the VRF definition for 'OUTSIDE' out of the configuration you posted ?

[ebng]

Whoops, that was my attempt to obscure aspects of the configuration, but, evidently, couldn't keep things straight and just made things extra confusing.  EXTERNAL==OUTSIDE.  Attaching an updated config.

[Georg Pauwen]

Hello,

try the config with the lines marked in bold:

version 16.9
service timestamps debug datetime msec
service timestamps log datetime msec
platform qfp utilization monitor load 80
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname CSR1000V-NAT
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
ip vrf EXTERNAL
rd 7683763:7
!
ip vrf MGMT
rd 38783783:1
!
login on-success log
!
subscriber templating
!
multilink bundle-name authenticated
!
license udi pid CSR1000V sn <redacted>
license boot level ax
no license smart enable
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
username admin privilege 15 secret 5 <redacted>
!
redundancy
!
interface GigabitEthernet1
ip vrf forwarding MGMT
ip address 10.10.100.32 255.255.255.0
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet2
ip address 100.19.0.33 255.255.255.224
ip nat inside
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
ip vrf forwarding OUTSIDE
ip address 200.168.1.1 255.255.254.0
ip nat outside
--> ip policy route-map PBR_VRF
negotiation auto
no mop enabled
no mop sysid
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip nat inside source static 100.19.0.52 200.168.1.190
--> ip route 0.0.0.0 0.0.0.0 GigabitEthernet3 200.168.1.2
ip route vrf MGMT 0.0.0.0 0.0.0.0 10.10.100.1
!
--> ip prefix-list PREFIX seq 5 permit 100.19.0.32/27
!
--> route-map PBR_VRF permit 10
--> match ip address prefix-list PREFIX
--> set global

[ebng]

It doesn't like it when I apply that route-map to the interface:

CSR1000V-NAT(config)#int gig3
CSR1000V-NAT(config-if)#ip policy route-map PBR_VRF
% Can not apply route-map PBR_VRF to this interface
if match clause with prefix-list is configured

Also, I'm happy to apply that default route for testing purposes, but it wouldn't be my intent to send all traffic out Gig3.  In this case, there isn't even an 200.168.1.2 to send the traffic to, but guessing you may have meant 200.168.1.252. 

[Georg Pauwen]

Hello,

try an access list instead of a prefix list:

version 16.9
service timestamps debug datetime msec
service timestamps log datetime msec
platform qfp utilization monitor load 80
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname CSR1000V-NAT
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
ip vrf EXTERNAL
rd 7683763:7
!
ip vrf MGMT
rd 38783783:1
!
login on-success log
!
subscriber templating
!
multilink bundle-name authenticated
!
license udi pid CSR1000V sn <redacted>
license boot level ax
no license smart enable
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
username admin privilege 15 secret 5 <redacted>
!
redundancy
!
interface GigabitEthernet1
ip vrf forwarding MGMT
ip address 10.10.100.32 255.255.255.0
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet2
ip address 100.19.0.33 255.255.255.224
ip nat inside
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
ip vrf forwarding OUTSIDE
ip address 200.168.1.1 255.255.254.0
ip nat outside
--> ip policy route-map PBR_VRF
negotiation auto
no mop enabled
no mop sysid
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip nat inside source static 100.19.0.52 200.168.1.190
--> ip route 0.0.0.0 0.0.0.0 GigabitEthernet3 200.168.1.2
ip route vrf MGMT 0.0.0.0 0.0.0.0 10.10.100.1
!
--> access-list 1 permit 100.19.0.32 0.0.0.31
!
--> route-map PBR_VRF permit 10
--> match ip address 1
--> set global

[ebng]

That took, but the results are the same.  R2 receives the ICMP request from the server's NAT'd IP and then sends an ARP request for said IP, but receives no response.

[paul driver]

Hello

---

@ebng wrote:

To add one more piece of data here, if I drop the OUTSIDE VRF altogether and have both interfaces Gig2 and Gig3 in the main routing instance, this works just fine and R1 will reply to ARP requests for 200.168.1.190

The above worked because both interfaces were in the same route tables.
As i stated before the reason it isn't working now is due to the rtr not knowing how to route between the global and vrf's route tables, because your static routing is incorrect, now you have confirmed this is how you would like it to be and dont wont the lan users to be in the same vrf then you need to tell the rtr how to route between the two route tables.

Add the following and test again

ip route 200.168.1.190 255.255.255.255 100.19.0.34
ip route vrf EXTERNAL 100.19.0.32 255.255.255.224 GigabitEthernet2 100.19.0.34 global