11-10-2015 01:25 AM - edited 03-11-2019 11:51 PM
Hey all,
I need to pick your brains on this one as I have run out of ideas and still cannot figure out why I am unable to reach new destinations.
Just recently we added a new site into our WAN estate and connectivity is over SHDS to the other end. Both ends have cisco ASAs and next hop/gateway are the ASAs on each side. We have created a transit VLAN which is connecting the two sites, x.x.x.1 is the remote ASA and x.x.x.5 is my ASA. I have used a new interface E0/4 for the transite VLAN and have setup the required routing and firewall policies to allow traffic to 172.16/16 network which is reacheable via x.x.x.1
So far things are looking good as I am can get to the other end but have come across a strange thing with servers that have a static NAT in place. These hosts are unable to reach this network as traffic hits the firewall and then it goes out the WAN interface.
Traceroute from a server that has default gateway as the core switch and the core switch with default route of the firewall.
Tracing route to 172.16.101.50 over a maximum of 30 hops
1 3 ms 4 ms 2 ms colo-coresw.matches.com [10.0.0.245] - CORE SWITCH VIP
2 2 ms <1 ms <1 ms 10.0.0.254 - FIREWALL INTERFACE (INSIDE)
3 3 ms 3 ms 3 ms 172.16.101.50
Trace complete.
Traceroute from a server that has default gateway as the core switch and the core switch with default route of the firewall but with a static NAT.
Tracing route to 172.16.101.50 over a maximum of 30 hops
1 3 ms 2 ms 2 ms colo-coresw.matches.com [10.0.0.245]
2 <1 ms <1 ms <1 ms 10.0.0.254
3 <1 ms <1 ms <1 ms 154.59.137.105 - INTERNET ROUTER
4 2 ms 2 ms 2 ms port-40-199.xxxxxxxxxxxxxxxx
5 2 ms 2 ms 2 ms port-98-199.xxxxxxxxxxxxxxxx
6 2 ms 2 ms 2 ms port-83-199.xxxxxxxxxxxxxxxx
I don't understand why traffic is going out the WAN interface since there is a static route on the firewall for the 172.16/16 network
S* 0.0.0.0 0.0.0.0 [10/0] via 154.59.137.105, INTERNET-WAN
S 10.0.0.0 255.255.252.0 [1/0] via 10.0.0.245, DEFAULT
C 10.0.0.0 255.255.255.0 is directly connected, DEFAULT
L 10.0.0.254 255.255.255.255 is directly connected, DEFAULT
S 10.0.50.0 255.255.254.0 [1/0] via 10.0.0.245, DEFAULT
S 10.0.60.0 255.255.254.0 [1/0] via 10.0.0.245, DEFAULT
S 10.0.100.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT
S 10.0.101.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT
S 10.0.150.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT
S 10.0.155.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT
S 10.0.200.0 255.255.248.0 [1/0] via 10.0.0.245, DEFAULT
S 10.0.208.0 255.255.255.0 [1/0] via 10.0.0.245, DEFAULT
C 10.0.254.0 255.255.255.0 is directly connected, MGMT
L 10.0.254.254 255.255.255.255 is directly connected, MGMT
C 10.0.255.16 255.255.255.248 is directly connected, FAILOVER-LAN
L 10.0.255.17 255.255.255.255 is directly connected, FAILOVER-LAN
C 10.0.255.24 255.255.255.248 is directly connected, STATEFULL-FAILOVER
L 10.0.255.25 255.255.255.255 is directly connected, STATEFULL-FAILOVER
S 10.2.0.0 255.255.0.0 [1/0] via 10.0.0.245, DEFAULT
S 10.3.0.0 255.255.0.0 [1/0] via 10.0.0.245, DEFAULT
S 10.4.0.0 255.255.0.0 [1/0] via 10.0.0.245, DEFAULT
S 10.33.52.0 255.255.252.0 [1/0] via 10.0.0.245, DEFAULT
C 10.255.255.0 255.255.255.240 is directly connected, P2P-COLO-DR
L 10.255.255.5 255.255.255.255 is directly connected, P2P-COLO-DR
C 154.59.xxx.xxx 255.255.255.248 is directly connected, INTERNET-WAN
L 154.59.xxx.xxx 255.255.255.255 is directly connected, INTERNET-WAN
S 172.16.0.0 255.255.0.0 [1/0] via 10.255.255.1, P2P-COLO-DR
C 192.168.200.0 255.255.255.240 is directly connected, P2P-COLO-DC
L 192.168.200.10 255.255.255.255 is directly connected, P2P-COLO-DC
STATIC NAT's
show nat
Manual NAT Policies (Section 1)
1 (INTERNET-WAN) to (DEFAULT) source static any any destination static repo-1.abcxremote.com-PUBLIC repo-1.abcxremote.com-PRIVATE no-proxy-arp
translate_hits = 21, untranslate_hits = 70325
2 (INTERNET-WAN) to (DEFAULT) source static any any destination static prd-inf-perc-01-PUBLIC prd-inf-perc-01-PRIVATE no-proxy-arp
translate_hits = 454368, untranslate_hits = 522017
3 (INTERNET-WAN) to (DEFAULT) source static any any destination static OWL.abcx.com-PUBLIC OWL.abcx.com-PRIVATE no-proxy-arp
translate_hits = 42324485, untranslate_hits = 44275688
4 (INTERNET-WAN) to (DEFAULT) source static any any destination static stg-inf-www-01-PUBLIC stg-inf-www-01-PRIVATE no-proxy-arp
translate_hits = 207, untranslate_hits = 70459
5 (INTERNET-WAN) to (DEFAULT) source static any any destination static prd-inf-mon-01-PUBLIC prd-inf-mon-01-PRIVATE no-proxy-arp
translate_hits = 2995324, untranslate_hits = 8470375
6 (INTERNET-WAN) to (DEFAULT) source static any any destination static test-hyb-app-01-PUBLIC test-hyb-app-01-PRIVATE no-proxy-arp
translate_hits = 8385052, untranslate_hits = 8617839
7 (INTERNET-WAN) to (DEFAULT) source static any any destination static TEST-PUBLIC 10.0.2.49-PRIVATE no-proxy-arp
translate_hits = 104, untranslate_hits = 941
8 (INTERNET-WAN) to (DEFAULT) source static any any destination static uat-inf-www-vip-PUBLIC uat-inf-www-vip-PRIVATE no-proxy-arp
translate_hits = 3082, untranslate_hits = 54812
9 (INTERNET-WAN) to (DEFAULT) source static any any destination static HO-Mail1-PUBLIC HO-Mail1-PRIVATE no-proxy-arp
translate_hits = 2382362, untranslate_hits = 2864656
10 (INTERNET-WAN) to (DEFAULT) source static any any destination static HO-Mail3-PUBLIC HO-Mail3-PRIVATE no-proxy-arp
translate_hits = 357752, untranslate_hits = 645180
11 (INTERNET-WAN) to (DEFAULT) source static any any destination static test2-hyb-app-01-PUBLIC test2-hyb-app-01-PRIVATE unidirectional no-proxy-arp
translate_hits = 0, untranslate_hits = 50089
12 (INTERNET-WAN) to (DEFAULT) source static any any destination static chef.abcxremote.com-PUBLIC chef.abcxremote.com-PRIVATE no-proxy-arp
translate_hits = 399518, untranslate_hits = 626786
13 (INTERNET-WAN) to (DEFAULT) source static any any destination static Exch-Hybrid-Public Exch-Hybrid-Private no-proxy-arp
translate_hits = 55956, untranslate_hits = 145420
14 (DEFAULT) to (INTERNET-WAN) source static DM_INLINE_NETWORK_16 DM_INLINE_NETWORK_16 destination static Store-POPUP-PRIVATE Store-POPUP-PRIVATE no-proxy-arp route-lookup
translate_hits = 0, untranslate_hits = 0
15 (DEFAULT) to (INTERNET-WAN) source static DM_INLINE_NETWORK_23 DM_INLINE_NETWORK_23 destination static 75LedburyOffice 75LedburyOffice no-proxy-arp route-lookup
translate_hits = 147515, untranslate_hits = 154175
16 (DEFAULT) to (INTERNET-WAN) source static DM_INLINE_NETWORK_25 DM_INLINE_NETWORK_25 destination static TestHOADSL-VPN TestHOADSL-VPN no-proxy-arp route-lookup
translate_hits = 1119485, untranslate_hits = 1120465
Would be grateful if anyone can identify what the issue could be ??
Waiting for your reply.
Regards,
Syed
Solved! Go to Solution.
11-10-2015 02:54 AM
Hi,
Use 'Route-lookup' keyward at the end of these manual statement. Use for top statement and intiate it for that private address. If that works, then perform the same on rest of the statement
or else..
Instead of creating these Manual NAT, create object nats something like :
Object net-repo-1.abcxremote.com-PRIVATE
host <private-ip>
nat (default, internet-wan) static repo-1.abcxremote.com-PUBLIC
Same thing for other public IP. In this case, it would always use route-lookup for selecting egress interface.
Hope it helps.
Regards,
Akshay Rastogi
11-10-2015 01:50 AM
Hi there,
Is your Destination IP 172.16.101.50 matching any of the above mentioned nat. ASA use egress interface with the help of Manual NAT if the destination keyword is used. As i couldn't find any nat statement with mapped-interface as ' P2P-COLO-DR', ASA would be choosing the Internet-WAN as the egrees interface inspite of having a route pointing towards P2P-COLO-DR interface.
Find the statement and correct it. Your traffic should not overlap with the existing manual nat or configure manual nat on line one for your concerned traffic and mapped address as P2P-COLO-DR.
Hope it helps.
Regards,
Akshay Rastogi
11-10-2015 02:24 AM
Hi Akshay,
No there is no manual Nat for any of the hosts in 172.0/16 range. The issue, as explained previously, only applies to hosts that have a public static NAT on the firewall. I have about 14 static nats on the firewall and all of these 14 hosts are unable to reach the new subnet (172.16.0/16) every other server, user is able to get to the destination network.
Just noticed that same issue applies to site-to-stite VPN networks.
Host (10.0.0.2) with a public static NAT is unable to get to 10.0.111.1 (Remote VPN Firewall)
Host (10.0.096) with no public NAT can get to 10.0.111.1 ??
I think I have not setup static NAT's right
11-10-2015 02:36 AM
Hi,
Is your traffic is being initated from behind 'DEFAULT' interface. I could see that all your starting 13 NAT statements are something like:
1 (INTERNET-WAN) to (DEFAULT) source static any any destination static repo-1.abcxremote.com-PUBLIC repo-1.abcxremote.com-PRIVATE no-proxy-arp
I belive as the traffic is initiated from interface DEFAULT from host repo-1.abcxremote.com-PRIVATE(as in your nat); so for them Static any any is the destination and that covers your Sever 172.16 range servers.
Regards,
Akshay Rastogi
11-10-2015 02:44 AM
Hi Akshay,
Ok that makes sense and yes all traffic is originated from Default Interface. What's the solution for this ? There is an auto NAT policy at the bottom as well
1 (any) to (INTERNET-WAN) source dynamic OBJ_NAT-Any (0.0.0.0/0) interface
11-10-2015 02:54 AM
Hi,
Use 'Route-lookup' keyward at the end of these manual statement. Use for top statement and intiate it for that private address. If that works, then perform the same on rest of the statement
or else..
Instead of creating these Manual NAT, create object nats something like :
Object net-repo-1.abcxremote.com-PRIVATE
host <private-ip>
nat (default, internet-wan) static repo-1.abcxremote.com-PUBLIC
Same thing for other public IP. In this case, it would always use route-lookup for selecting egress interface.
Hope it helps.
Regards,
Akshay Rastogi
11-10-2015 03:06 AM
Ok mate that has worked for me thank you very much. Had to setup Object Nat's to achieve this.
One last thing I am unable to get to from Natted hostst is the VPN network Ledbury75. On the NAT statements you can see Ledbury75 is a VPN subnet with Route Lookup defined.
11-10-2015 03:23 AM
You're Welcome.
Is that stopped working after these changes or was not working from starting?
I could not find any nat statement with Ledbury75? could you please mention that form 'show run nat' output. Also check if the natted ip is being added in cryptomap access-lists
Regards,
Akshay Rastogi
11-10-2015 03:37 AM
No actually that never worked for me.
show run nat
nat (INTERNET-WAN,DEFAULT) source static any any destination static repo-1.abcxremote.com-PUBLIC repo-1.abcxremote.com-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static prd-inf-perc-01-PUBLIC prd-inf-perc-01-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static OWL.abcx.com-PUBLIC OWL.abcx.com-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static stg-inf-www-01-PUBLIC stg-inf-www-01-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static prd-inf-mon-01-PUBLIC prd-inf-mon-01-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static test-hyb-app-01-PUBLIC test-hyb-app-01-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static uat-inf-www-vip-PUBLIC uat-inf-www-vip-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static HO-mail1-PUBLIC HO-mail1-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static HO-mail3-PUBLIC HO-mail3-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static test2-hyb-app-01-PUBLIC test2-hyb-app-01-PRIVATE unidirectional no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static chef.abcxremote.com-PUBLIC chef.abcxremote.com-PRIVATE no-proxy-arp
nat (INTERNET-WAN,DEFAULT) source static any any destination static Exch-Hybrid-Public Exch-Hybrid-Private no-proxy-arp
nat (DEFAULT,INTERNET-WAN) source static DM_INLINE_NETWORK_23 DM_INLINE_NETWORK_23 destination static 75LedburyOffice 75LedburyOffice no-proxy-arp route-lookup
nat (DEFAULT,INTERNET-WAN) source static DM_INLINE_NETWORK_25 DM_INLINE_NETWORK_25 destination static TestHOADSL-VPN TestHOADSL-VPN no-proxy-arp route-lookup
!
object network 10.0.2.49-PRIVATE
nat (DEFAULT,INTERNET-WAN) static TEST-PUBLIC
object network OBJ_NAT-Any
nat (any,INTERNET-WAN) dynamic interface
Crypto Mat access list has the entire network VLAN 10.0.0.0/24 so that shouldn't be an issue.
Regards,
Syed
11-10-2015 03:45 AM
Hi Syed,
From the statement, i believe that the Ledburyoffice is behind Default.
It is not alone this side, your traffic should be allowed on the other side as well(check the other end cryptop-map acl?
First thing you could do is to put this statement on line 1. Edit the same NAT and 1 after : nat (DEFAULT,INTERNET-WAN) 1
Now check if it works. It migh be overlapping with above or something(can not say).
Try checking your traffic through packet-tracer utility if it is hitting a correct nat statement and access-lists to permit the traffic.
Regards,
Akshay Rastogi
11-10-2015 03:52 AM
Hi Akshay,
No Ledbury75 is a VPN site
object-group network DM_INLINE_NETWORK_23
network-object object DC_Object
network-object object Head_Office_VLAN-50
network-object object Head_Office_VLAN-60
network-object object Network_VLAN-1
network-object object FRA-SSRS
I can ping the natted host 10.0.0.2 (behind Default) from Ledbury's firewall (10.0.111.1) but when I try to ping from 10.0.0.2 to 10.0.111.1 i get no response.
Traceroute chucks traffic to WAN interface
Tracing route to 10.0.111.1 over a maximum of 30 hops
1 101 ms 63 ms 3 ms colo-coresw.matches.com [10.0.0.245]
2 <1 ms <1 ms <1 ms 154.59.xxx.xxx
3 2 ms 1 ms 1 ms port-40-199
4 201 ms 6 ms 2 ms port-98-199
5 2 ms 2 ms 2 ms port-83-199
6 127 ms 206 ms 2 ms port-82-199
7 * * * Request timed out.
8 * * * Request timed out.
9 * * * Request timed out.
10 * * * Request timed out.
11 * port-82-199 reports: Destination host unreachable.
Trace complete.
11-10-2015 04:12 AM
Ignore me please everything is working after your suggested changes.
Many thanks for all your help today :)
Regards,
Syed
11-10-2015 04:18 AM
You are welcome, Syed.
Regards,
Akshay Rastogi
Remember to rate the helpful the helpful posts.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide