ISSUE WITH NAT ON A STICK using Loopback

[JUANNN]

Hello,

I am doing both on CML and on hardware the following configuration:

The goal is to configure NAT on the NAT_ROUTER using only 1 physical interface (G1.10), which will be on the INSIDE.

I have tried it with 2 subinterfaces, one as inside and one as outside, and it worked.

However, the next step is to get it to work using only 1 subinterface. For this, I created a Loopback interface and set it as the OUTSIDE:

Since the traffic from the users (from HOST1 which is in vlan 10.0.0.0 /24, and HOST1 has an interface VLAN 10 emulating a host with IP address 10.0.0.2 /24) gets routed before NATed when it goes through G1.10, I configued PBR to match that traffic (as you can see in the G1.10 screenshot) and place it on the loopback interface, which is on the OUTSIDE (next hop set by PBR is in network directly connected to lo0, 10.0.1.0 /30), so that way it gets NATed before routed:

Once my traffic with source 10.0.0.2 is placed in the loopback 0 interface on the OUTSIDE, NAT takes place and applies the following:

The source of my traffic from HOST 1 now should be 192.168.1.3. The destination is (for example) 100.100.100.100 (INET_SERV). The NAT router then routes traffic using the default route pointing to CMTS:

which is successfully installed in the RIB (in case someone was doubting). Once the traffic gets to CMTS, the rest of the path to INET_SERV is fine.

On the way back to HOST1, traffic from CMTS coming from INET_SERV has a source of 100.100.100.100 and a destination of 192.168.1.3. It gets to G1.10, but again matches PBR and gets placed on the loopback 0 interface:

Once traffic gets placed to the loopback0 interface, again is NATed before routed,. Since traffic has now a source of 100.100.100.100 and destination of 192.168.1.3, it should match the following NAT rule:

and the destination IP address of traffic should be translated to 10.0.0.2, which is HOST1 IP address, and finally traffic should get routed back to HOST1.

However, I might be doing something wrong or misunderstanding NAT or PBR, because traffic from HOST1 to INET SERV is reaching INET SERV, but NAT is not translating traffic in the NAT router. On the packet captures that I have done in the link between CMTS and INET_SERV, I see the ICMP request with source 10.0.0.2 and destination 100.100.100.100, which means that traffic goes through NAT router without getting translated. Traffic the cannot make it back since INET_SERV does not have a route to 10.0.0.0 /24. I know I could configure NAT on CMTS or a route to 10.0.0.0 /24 on INET_SERV to solve this, but the goal is to solve it from the NAT router.

I have tried this on hardware (CISCO 3650 port switch as L2SW and C1121X routers) and software (CML2.7.2) and no success. I have been stuck with this countless hours for many days.

Anyone knows what could be wrong, or if I am misunderstanding anything?

Thanks, any help is appreciated,

Juan

[paul driver]

 @JUANNN  please use the cfg i provided that is based in your OP 
can you confirm if you tried using the set ip nexthop 1.1.1.1 instead of the set interface loopback x  in the Route-map - it may not work 

[JUANNN]

Hello Paul

I used your cfg but it didn't work. I will try your cfg again, and also on hardware. 

[paul driver]

Hello
fyi i have just managed to lab this up in EVE from my own sanity and it did work  accordingly so it must be your CML software negating it 

[JUANNN]

Hello,

Things are looking almost perfect now.

I have followed the Cisco cfgs and CML now shows me NAT translations done correctly, PBR working properly even with a tunnel interface and packets from Host1 reaching CMTS lo0 ip add (6.6.6.6) and CMTS does NOT have a route to 10.0.0.0/24.

NAT ROUTER cfgs

Here are the packet captures that prove that NAT is working. Note that 10.0.0.2 is Host1 and 10.0.0.1 is NAT router G1 primary IP:

But like I said, is almost perfect. The problem, as you can see, is that I am missing the last ICMP reply that is supposed to be 

- source: 6.6.6.6

- dest: 10.0.0.2

So there is still a little issue in NAT router when returning traffic from CMTS comes back. Any thougths? I will try using loopback instead of tunnel, and check ACLs.

[MHM Cisco World]

Friend 

Now you missing 

Ip nat outside under LO

Add it if not work' then sorry I dont have time to continue troubleshoot issue.

Thanks for understanding

MHM

[JUANNN · Accepted solution]

There it is my friends, the end of this painful but interesting LAB. This was the missing command that Cisco does not include in their configs, and is the key to understand how PBR works with NAT: (from what I can deduct): 

1. Traffic from Host1 with source 10.0.0.2 and dest 6.6.6.6 enters NATrouter via G1, INSIDE

2. PBR goes before Routing Table Routing and before NAT on the INSIDE, so traffic is routed to interface tunnel 100, which is on the OUTSIDE. On that trip inside-outside, traffic source gets translated by NAT from 10.0.0.2 to 192.168.2.2. 

3. Once in the OUTSIDE on interface tunnel 100, traffic gets routed using the default route to CMTS, and reaches 6.6.6.6. This means that even if NAT attempts to act in the OUSTIDE-INSIDE trip (from int tunnel to G1) it would NOT matter because the ACL 10 does not match the traffic with source 192.168.1.2 destination 6.6.6.6.

4. Traffic returns from CMTS to NATrouter, traffic has source 6.6.6.6 dest 192.168.2.2. Enters G1 INSIDE, PBR takes place (ACL 102 matches also this traffic) and routes traffic to int tunnel 100. NAT acts again in this INSIDE OUTSIDE trip with the command that I just added, changing destination of trafffic from 192.168.2.2 to 10.0.0.2. 

5. Finally, traffic departs from int tunnel 100 towards 10.0.0.2, and source remains 6.6.6.6 It does not match any NAT ACL, so NAT does not act on this OUTSIDE-INSIDE trip, and then traffic gets sent to Host1!!

After 12 hours on my own and almost 12 hours on the Cisco Community, there is a soultion that works and makes sense. 

Thanks to Paul Driver and MHM for their interest and dedication.

Regards,

juan

[paul driver · Accepted solution]

Hello
First of all congratulations on obtaining nat translation...
However I would like to point out that your initial OP was JUST to use a single interface (ingress/egress) with primary /secondary addressing both for Lan/wan connectivity and to incorporate NAT,

An early suggestion by @MHM Cisco World  mentioned using two separate logical interfaces however you negated that stating you only wish to use a single interface (logical or physical) and really for that to be achieved I suggested NAT haipining could be a possible solution. 
However I see you have indeed ended up using additional logical "tunnel" interfaces anyway as such creating a basic static nat solution be in a rather convoluted manner that could been achieved a much simpler way.

Example:
int gig01
no ip policy route-map Nat-Loop
no interface loopback0
no route-map Nat-Loop
no ip nat pool external
no ip route 0.0.0.0 0.0.0.0 192.168.1.1
no ip access-list 10
no ip nat inside source list 10 pool external
no ip nat outside source static 10.0.0.2 192.168.2.2

Nat solution with tunnel interfaces
interface GigabitEthernet1
ip address 10.0.0.1 255.255.255.0 secondary
ip address 192.168.1.2 255.255.255.0
no ip redirects
ip nat inside

interface Tunnel100
ip address 1.1.1.1 255.255.255.252
ip nat outside
tunnel source 192.168.1.2
tunnel destination 192.168.1.1

ip route 192.168.1.1 255.255.255.255 GigabitEthernet1
ip route 0.0.0.0 0.0.0.0 Tunnel100 1.1.1.2
ip nat inside source list public interface Tunnel100 overload

[JUANNN]

Hello  @paul driver,

Your cfg is valid, but it has also some additional cfgs that you don't specify: I labbed it, but it works obvoiusly having a tunnel interface also on CMTS, otherways it does NOT work, because packets will never get de-encapsulated on CMTS. For the rest, is a valid solution. The goal was to not having to touch CMTS at all, just come up with a solution in the NAT router.

Also, you don't need ip route 192.168.1.1 255.255.255.255 GigabitEthernet1

However, I think is a much better solution than mine, since I had to rely on a static NAT rule, which makes it unscalable for more hosts. 

I labbed it and it works perfectly fine.

Thanks a lot for your answer!!

Juan

[paul driver]

Hello
thank you for the feed back, much appreciated...

 

---

@JUANNN wrote:
Also, you don't need ip route 192.168.1.1 255.255.255.255 GigabitEthernet1

---

FYI - that static is there just to make sure you do not incur any recursive routing through the tunnel do the fact the static default route is using the tunnel.

[MHM Cisco World]

sorry but I think you mis understand my previous statement

he was using in NAT Mapped IP config as secondary in interface use as IP NAT INSIDE <<<- that so so wrong 
the real IP and Mapped IP must not config under same interface, the NAT idea if traffic pass boundary and this boundary is known by interface.
that why I ask him to check subnet he use as secondary and subnet it use for NATing 

anyway I hope he get something from this lab

for summary of hairpin I will send PM to @JUANNN  for more detail 

MHM

[JUANNN]

Hello MHM,

he was using in NAT Mapped IP config as secondary in interface use as IP NAT INSIDE <<<- that so so wrong 

Yes, that is also true and it may be the reason why it wasn't working initially. Thanks a lot, I have learned a lot through this lab!

Juan

[paul driver]

Hello

---

@JUANNN wrote:
he was using in NAT Mapped IP config as secondary in interface use as IP NAT INSIDE <<<- that so so wrong 

---

FYI - @MHM Cisco World @JUANNN 
Do not forget I was suggesting hairpinng !
Also the nat domain ONLY works IF you have a nat statement that matches a specific subnet/ip OR an acl that relates to the "hidden" network you wish to have translation so having secondary addressing on a *"inside/outside" nat domain will NOT automatically assign itself to any ip address so it was NOT incorrect

[MHM Cisco World]

Also the nat domain ONLY works IF you have a nat statement that matches a specific subnet/ip OR an acl that relates to the "hidden" network you wish to have translation so having secondary addressing on a *"inside/outside" nat domain will NOT automatically assign itself to any ip address so it was NOT incorrect <<- that not correct sorry' if that correct why we need to config ip nat inside/outside under interface' the NAT don't totally relate to subnet it boundary building by interface.

Also even hairpin need boundary' that what NVI do with ip enabled' NVI work as both ip nat side interface and ip nat outside interface build boundary' i.e. make two three NAT domain.

Also that explain why we need LO and config ip nat outside for hairpin.

Anyway let please close this topic I really dont have time to answer you. So sorry.

MHM

[paul driver]

Hello

---

@MHM Cisco World wrote:
Also even hairpin need boundary' that what NVI do with ip enabled' NVI work as both ip nat side interface and ip nat outside interface build boundary' i.e. make two three NAT domain.

Also that explain why we need LO and config ip nat outside for hairpin.

---

Please review the differences between domain & domainless nat and especially when using hairpinning