topic Hi, in Network Security

Static NAT & Routing

itops — Tue, 12 Mar 2019 06:51:39 GMT

Hey all,

I need to pick your brains on this one as I have run out of ideas and still cannot figure out why I am unable to reach new destinations.

Just recently we added a new site into our WAN estate and connectivity is over SHDS to the other end. Both ends have cisco ASAs and next hop/gateway are the ASAs on each side. We have created a transit VLAN which is connecting the two sites, x.x.x.1 is the remote ASA and x.x.x.5 is my ASA. I have used a new interface E0/4 for the transite VLAN and have setup the required routing and firewall policies to allow traffic to 172.16/16 network which is reacheable via x.x.x.1

So far things are looking good as I am can get to the other end but have come across a strange thing with servers that have a static NAT in place. These hosts are unable to reach this network as traffic hits the firewall and then it goes out the WAN interface.

Traceroute from a server that has default gateway as the core switch and the core switch with default route of the firewall.

Tracing route to 172.16.101.50 over a maximum of 30 hops

1 3 ms 4 ms 2 ms colo-coresw.matches.com [10.0.0.245] - CORE SWITCH VIP
2 2 ms <1 ms <1 ms 10.0.0.254 - FIREWALL INTERFACE (INSIDE)
3 3 ms 3 ms 3 ms 172.16.101.50

Trace complete.

Traceroute from a server that has default gateway as the core switch and the core switch with default route of the firewall but with a static NAT.

Tracing route to 172.16.101.50 over a maximum of 30 hops

1 3 ms 2 ms 2 ms colo-coresw.matches.com [10.0.0.245]
2 <1 ms <1 ms <1 ms 10.0.0.254
3 <1 ms <1 ms <1 ms 154.59.137.105 - INTERNET ROUTER
4 2 ms 2 ms 2 ms port-40-199.xxxxxxxxxxxxxxxx
5 2 ms 2 ms 2 ms port-98-199.xxxxxxxxxxxxxxxx
6 2 ms 2 ms 2 ms port-83-199.xxxxxxxxxxxxxxxx

I don't understand why traffic is going out the WAN interface since there is a static route on the firewall for the 172.16/16 network

S* 0.0.0.0 0.0.0.0 [10/0] via 154.59.137.105, INTERNET-WAN

S 10.0.0.0 255.255.252.0 [1/0] via 10.0.0.245, DEFAULT

C 10.0.0.0 255.255.255.0 is directly connected, DEFAULT

L 10.0.0.254 255.255.255.255 is directly connected, DEFAULT

S 10.0.50.0 255.255.254.0 [1/0] via 10.0.0.245, DEFAULT

S 10.0.60.0 255.255.254.0 [1/0] via 10.0.0.245, DEFAULT

S 10.0.100.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT

S 10.0.101.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT

S 10.0.150.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT

S 10.0.155.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT

S 10.0.200.0 255.255.248.0 [1/0] via 10.0.0.245, DEFAULT

S 10.0.208.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT

C 10.0.254.0 255.255.255.0 is directly connected, MGMT

L 10.0.254.254 255.255.255.255 is directly connected, MGMT

C 10.0.255.16 255.255.255.248 is directly connected, FAILOVER-LAN

L 10.0.255.17 255.255.255.255 is directly connected, FAILOVER-LAN

C 10.0.255.24 255.255.255.248 is directly connected, STATEFULL-FAILOVER

L 10.0.255.25 255.255.255.255 is directly connected, STATEFULL-FAILOVER

S 10.2.0.0 255.255.0.0 [1/0] via 10.0.0.245, DEFAULT

S 10.3.0.0 255.255.0.0 [1/0] via 10.0.0.245, DEFAULT

S 10.4.0.0 255.255.0.0 [1/0] via 10.0.0.245, DEFAULT

S 10.33.52.0 255.255.252.0 [1/0] via 10.0.0.245, DEFAULT

C 10.255.255.0 255.255.255.240 is directly connected, P2P-COLO-DR

L 10.255.255.5 255.255.255.255 is directly connected, P2P-COLO-DR

C 154.59.xxx.xxx 255.255.255.248 is directly connected, INTERNET-WAN

L 154.59.xxx.xxx 255.255.255.255 is directly connected, INTERNET-WAN

S 172.16.0.0 255.255.0.0 [1/0] via 10.255.255.1, P2P-COLO-DR

C 192.168.200.0 255.255.255.240 is directly connected, P2P-COLO-DC

L 192.168.200.10 255.255.255.255 is directly connected, P2P-COLO-DC

STATIC NAT's

show nat

Manual NAT Policies (Section 1)

1 (INTERNET-WAN) to (DEFAULT) source static any any destination static repo-1.abcxremote.com-PUBLIC repo-1.abcxremote.com-PRIVATE no-proxy-arp

translate_hits = 21, untranslate_hits = 70325

2 (INTERNET-WAN) to (DEFAULT) source static any any destination static prd-inf-perc-01-PUBLIC prd-inf-perc-01-PRIVATE no-proxy-arp

translate_hits = 454368, untranslate_hits = 522017

3 (INTERNET-WAN) to (DEFAULT) source static any any destination static OWL.abcx.com-PUBLIC OWL.abcx.com-PRIVATE no-proxy-arp

translate_hits = 42324485, untranslate_hits = 44275688

4 (INTERNET-WAN) to (DEFAULT) source static any any destination static stg-inf-www-01-PUBLIC stg-inf-www-01-PRIVATE no-proxy-arp

translate_hits = 207, untranslate_hits = 70459

5 (INTERNET-WAN) to (DEFAULT) source static any any destination static prd-inf-mon-01-PUBLIC prd-inf-mon-01-PRIVATE no-proxy-arp

translate_hits = 2995324, untranslate_hits = 8470375

6 (INTERNET-WAN) to (DEFAULT) source static any any destination static test-hyb-app-01-PUBLIC test-hyb-app-01-PRIVATE no-proxy-arp

translate_hits = 8385052, untranslate_hits = 8617839

7 (INTERNET-WAN) to (DEFAULT) source static any any destination static TEST-PUBLIC 10.0.2.49-PRIVATE no-proxy-arp

translate_hits = 104, untranslate_hits = 941

8 (INTERNET-WAN) to (DEFAULT) source static any any destination static uat-inf-www-vip-PUBLIC uat-inf-www-vip-PRIVATE no-proxy-arp

translate_hits = 3082, untranslate_hits = 54812

9 (INTERNET-WAN) to (DEFAULT) source static any any destination static HO-Mail1-PUBLIC HO-Mail1-PRIVATE no-proxy-arp

translate_hits = 2382362, untranslate_hits = 2864656

10 (INTERNET-WAN) to (DEFAULT) source static any any destination static HO-Mail3-PUBLIC HO-Mail3-PRIVATE no-proxy-arp

translate_hits = 357752, untranslate_hits = 645180

11 (INTERNET-WAN) to (DEFAULT) source static any any destination static test2-hyb-app-01-PUBLIC test2-hyb-app-01-PRIVATE unidirectional no-proxy-arp

translate_hits = 0, untranslate_hits = 50089

12 (INTERNET-WAN) to (DEFAULT) source static any any destination static chef.abcxremote.com-PUBLIC chef.abcxremote.com-PRIVATE no-proxy-arp

translate_hits = 399518, untranslate_hits = 626786

13 (INTERNET-WAN) to (DEFAULT) source static any any destination static Exch-Hybrid-Public Exch-Hybrid-Private no-proxy-arp

translate_hits = 55956, untranslate_hits = 145420

14 (DEFAULT) to (INTERNET-WAN) source static DM_INLINE_NETWORK_16 DM_INLINE_NETWORK_16 destination static Store-POPUP-PRIVATE Store-POPUP-PRIVATE no-proxy-arp route-lookup

translate_hits = 0, untranslate_hits = 0

15 (DEFAULT) to (INTERNET-WAN) source static DM_INLINE_NETWORK_23 DM_INLINE_NETWORK_23 destination static 75LedburyOffice 75LedburyOffice no-proxy-arp route-lookup

translate_hits = 147515, untranslate_hits = 154175

16 (DEFAULT) to (INTERNET-WAN) source static DM_INLINE_NETWORK_25 DM_INLINE_NETWORK_25 destination static TestHOADSL-VPN TestHOADSL-VPN no-proxy-arp route-lookup

translate_hits = 1119485, untranslate_hits = 1120465

Would be grateful if anyone can identify what the issue could be ??

Waiting for your reply.

Regards,

Syed

Hi there,

Akshay Rastogi — Tue, 10 Nov 2015 09:50:53 GMT

Hi there,

Is your Destination IP 172.16.101.50 matching any of the above mentioned nat. ASA use egress interface with the help of Manual NAT if the destination keyword is used. As i couldn't find any nat statement with mapped-interface as ' P2P-COLO-DR', ASA would be choosing the Internet-WAN as the egrees interface inspite of having a route pointing towards P2P-COLO-DR interface.

Find the statement and correct it. Your traffic should not overlap with the existing manual nat or configure manual nat on line one for your concerned traffic and mapped address as P2P-COLO-DR.

Hope it helps.

Regards,

Akshay Rastogi

Hi Akshay,

itops — Tue, 10 Nov 2015 10:24:20 GMT

Hi Akshay,

No there is no manual Nat for any of the hosts in 172.0/16 range. The issue, as explained previously, only applies to hosts that have a public static NAT on the firewall. I have about 14 static nats on the firewall and all of these 14 hosts are unable to reach the new subnet (172.16.0/16) every other server, user is able to get to the destination network.

Just noticed that same issue applies to site-to-stite VPN networks.

Host (10.0.0.2) with a public static NAT is unable to get to 10.0.111.1 (Remote VPN Firewall)

Host (10.0.096) with no public NAT can get to 10.0.111.1 ??

I think I have not setup static NAT's right

Hi,

Akshay Rastogi — Tue, 10 Nov 2015 10:36:15 GMT

Hi,

Is your traffic is being initated from behind 'DEFAULT' interface. I could see that all your starting 13 NAT statements are something like:

1 (INTERNET-WAN) to (DEFAULT) source static any any destination static repo-1.abcxremote.com-PUBLIC repo-1.abcxremote.com-PRIVATE no-proxy-arp

I belive as the traffic is initiated from interface DEFAULT from host repo-1.abcxremote.com-PRIVATE(as in your nat); so for them Static any any is the destination and that covers your Sever 172.16 range servers.

Regards,

Akshay Rastogi

Hi Akshay,

itops — Tue, 10 Nov 2015 10:44:49 GMT

Hi Akshay,

Ok that makes sense and yes all traffic is originated from Default Interface. What's the solution for this ? There is an auto NAT policy at the bottom as well

1 (any) to (INTERNET-WAN) source dynamic OBJ_NAT-Any (0.0.0.0/0) interface

Hi,

Akshay Rastogi — Tue, 10 Nov 2015 10:54:11 GMT

Hi,

Use 'Route-lookup' keyward at the end of these manual statement. Use for top statement and intiate it for that private address. If that works, then perform the same on rest of the statement

or else..

Instead of creating these Manual NAT, create object nats something like :

Object net-repo-1.abcxremote.com-PRIVATE

host <private-ip>

nat (default, internet-wan) static repo-1.abcxremote.com-PUBLIC

Same thing for other public IP. In this case, it would always use route-lookup for selecting egress interface.

Hope it helps.

Regards,

Akshay Rastogi

Ok mate that has worked for

itops — Tue, 10 Nov 2015 11:06:53 GMT

Ok mate that has worked for me thank you very much. Had to setup Object Nat's to achieve this.

One last thing I am unable to get to from Natted hostst is the VPN network Ledbury75. On the NAT statements you can see Ledbury75 is a VPN subnet with Route Lookup defined.

You're Welcome.

Akshay Rastogi — Tue, 10 Nov 2015 11:23:57 GMT

You're Welcome.

Is that stopped working after these changes or was not working from starting?

I could not find any nat statement with Ledbury75? could you please mention that form 'show run nat' output. Also check if the natted ip is being added in cryptomap access-lists

Regards,

Akshay Rastogi

No actually that never worked

itops — Tue, 10 Nov 2015 11:37:24 GMT

No actually that never worked for me.

show run nat

nat (INTERNET-WAN,DEFAULT) source static any any destination static repo-1.abcxremote.com-PUBLIC repo-1.abcxremote.com-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static prd-inf-perc-01-PUBLIC prd-inf-perc-01-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static OWL.abcx.com-PUBLIC OWL.abcx.com-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static stg-inf-www-01-PUBLIC stg-inf-www-01-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static prd-inf-mon-01-PUBLIC prd-inf-mon-01-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static test-hyb-app-01-PUBLIC test-hyb-app-01-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static uat-inf-www-vip-PUBLIC uat-inf-www-vip-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static HO-mail1-PUBLIC HO-mail1-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static HO-mail3-PUBLIC HO-mail3-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static test2-hyb-app-01-PUBLIC test2-hyb-app-01-PRIVATE unidirectional no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static chef.abcxremote.com-PUBLIC chef.abcxremote.com-PRIVATE no-proxy-arp

nat (INTERNET-WAN,DEFAULT) source static any any destination static Exch-Hybrid-Public Exch-Hybrid-Private no-proxy-arp

nat (DEFAULT,INTERNET-WAN) source static DM_INLINE_NETWORK_23 DM_INLINE_NETWORK_23 destination static 75LedburyOffice 75LedburyOffice no-proxy-arp route-lookup

nat (DEFAULT,INTERNET-WAN) source static DM_INLINE_NETWORK_25 DM_INLINE_NETWORK_25 destination static TestHOADSL-VPN TestHOADSL-VPN no-proxy-arp route-lookup

object network 10.0.2.49-PRIVATE

nat (DEFAULT,INTERNET-WAN) static TEST-PUBLIC

object network OBJ_NAT-Any

nat (any,INTERNET-WAN) dynamic interface

Crypto Mat access list has the entire network VLAN 10.0.0.0/24 so that shouldn't be an issue.

Regards,

Syed

Hi Syed,

Akshay Rastogi — Tue, 10 Nov 2015 11:45:04 GMT

Hi Syed,

From the statement, i believe that the Ledburyoffice is behind Default.

It is not alone this side, your traffic should be allowed on the other side as well(check the other end cryptop-map acl?

First thing you could do is to put this statement on line 1. Edit the same NAT and 1 after : nat (DEFAULT,INTERNET-WAN) 1

Now check if it works. It migh be overlapping with above or something(can not say).

Try checking your traffic through packet-tracer utility if it is hitting a correct nat statement and access-lists to permit the traffic.

Regards,

Akshay Rastogi

Hi Akshay,

itops — Tue, 10 Nov 2015 11:52:52 GMT

Hi Akshay,

No Ledbury75 is a VPN site

object-group network DM_INLINE_NETWORK_23
network-object object DC_Object
network-object object Head_Office_VLAN-50
network-object object Head_Office_VLAN-60
network-object object Network_VLAN-1
network-object object FRA-SSRS

I can ping the natted host 10.0.0.2 (behind Default) from Ledbury's firewall (10.0.111.1) but when I try to ping from 10.0.0.2 to 10.0.111.1 i get no response.

Traceroute chucks traffic to WAN interface

Tracing route to 10.0.111.1 over a maximum of 30 hops

1 101 ms 63 ms 3 ms colo-coresw.matches.com [10.0.0.245]
2 <1 ms <1 ms <1 ms 154.59.xxx.xxx
3 2 ms 1 ms 1 ms port-40-199
4 201 ms 6 ms 2 ms port-98-199
5 2 ms 2 ms 2 ms port-83-199
6 127 ms 206 ms 2 ms port-82-199
7 * * * Request timed out.
8 * * * Request timed out.
9 * * * Request timed out.
10 * * * Request timed out.
11 * port-82-199 reports: Destination host unreachable.

Trace complete.

Ignore me please everything

itops — Tue, 10 Nov 2015 12:12:16 GMT

Ignore me please everything is working after your suggested changes.

Many thanks for all your help today 🙂

Regards,

Syed

You are welcome, Syed.

Akshay Rastogi — Tue, 10 Nov 2015 12:18:20 GMT

You are welcome, Syed.

Regards,

Akshay Rastogi

Remember to rate the helpful the helpful posts.