Cannot ping or access website across primary WAN but can across secondary WAN

[mbroberson1]

Have an interesting issue where clients at a remote branch site cannot ping or access a particular website across their primary circuit but can ping and access when traffic is moved to their secondary circuit.

(Diagram is attached) other Information:

• The ASR1001-X Primary WAN aggregate router can ping the website. 
• The WAN connections are typical /30's.
• There is no internal firewalling or filtering types of devices other than the edge (default gateway) firewall.
• The internal routing is "vanilla" and nothing crazy.
• The Primary WAN link is a managed circuit by a local carrier.
• The Secondary WAN link is a managed circuit by a well known national carrier.
• All the devices in the diagram and ping the website (50.x.x.x) except the ISR 4331 branch site when using the primary circuit.
• When I try to ping the website I do see the traffic being allowed on the ASA (default gateway) firewall so I know traffic is making it.
• Trace routes are near identical, only exception is of course the direction of the WAN when testing.
• When I shutdown the secondary circuit same result where pings and access fail. This is to ensure no asymmetric routing type of issue.

So far in my troubleshooting I've narrowed it to perhaps the router (4331) at the branch either having a bug perhaps or the primary carrier's circuit, but from the carrier's point of view it's just a layer 2 connection and they don't participate in any of our routing, I just do my routing across it... very straight forward. To make this more interesting this same issue is occurring at 2 other branch sites in the exact setup.

Any ideas for troubleshooting are appreciated!

[Georg Pauwen]

Hello,

where does the traceroute stop ?

Is the ASA doing the NAT ?

[mbroberson1]

I can traceroute all the way to the servers public IP across both circuits (trace across both WAN links both virtually identical when it leaves the edge ASA firewall), just can't ping it or access it's website when going across the primary circuit. The ASA is doing the NAT/PAT overload for client connectivity.

[Georg Pauwen]

Hello,

this could simply be an MTU related issue. Can you ping across both circuits and find out what the maximum packet size is as in the example below. Lower the packet size until the ping reply is successful:

C:\windows\system32>ping -l 1500 -f www.google.com

Pinging www.google.com [108.177.119.106] with 1500 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 108.177.119.106:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

ping

[mbroberson1]

I was suspecting that could be an area to consider. I tried pinging the web servers public IP from the branch router with a size of like 1200 and 1300 but failed. I'll try from a client (Windows machine) at the site.