Routing / traffic issue

[Mieczyslaw]

Hi I'm having some traffic issue, and not sure where else I can have a look as I check all routes and all looks ok to me but for some reason the traffic is not flowing as I would expect it, 

the requirement is that the shop should be able to talk to AWS 10.XX6.0.0 through MPLS network and our HO infrastructure, so if shop goes to AWS recourse the traffic should look like (shop - mpls - mpls ASA - core switch L3 - open ASA - ecomm ASA - AWS router - AWS) 

at the moment the traffic stop on the ecomm ASA 172.11.22.14 however if I do traceroute from the core switch the traffic will reach AWS router 10.XX.0.1 and this would be the last hop I would expect ot see as further all is blocked in terms of pings and traceroutes

I have attached some drawing so its easier to illustrate and changed/mask IP's for some random

any assistance would be appreciated 

[chrihussey]

A lot of information here. Very thorough, but a little tough to decipher. Anyhow I'm just going to give you my initial thoughts:

1- I see it is a VPN from the AWS router (10.xx.0.1) to AWS. Need to verify the 10.50.XX.XX space is part of that VPN's interesting traffic.

2- It looks like the routing is fine from the shop to the AWS router. It is interesting the trace route from the CORE SWITCH gets the response from 10.XX.0.1, yet the others don't. Not sure what to make of that.

3- Keep in mind that any trace route initiated by an intermediate device will probably use its outbound interface as source. So unless all the devices in the path know of that network and return route, the trace will not go as far. So the trace router from the AWS router will use 10.XX.0.1 as source, but if the MPLS ASA after the CORE SWITCH does not have that return route, the trace will end there since it cannot respond. However, perhaps if you leave the trace up it may start responding after a few hops.

Hope this is of some help.

[Mieczyslaw]

Hi chrihussey 

thank you for the responce,

1) its the BGP routing to AWS so we don't have interested traffic (not like standard site to site VPN where you specify LAN and Remote range) 

2) this is what puzzle me as well that the core switch which is before ASA's can traceroute but ASA's can't 

3) all ASA and core switch have the routes and routes for return traffic 
so each device have a route for the next hope for this range and next hop have a route for another next hope if this make sense 

so MPLS ASA for the 10.XX.6.0.0 have a route to go to Core switch, and core switch for this network have a route for OPEN ASA and open ASA will go to ECOMM and ECOMM to AWS router

and return path is 

AWS route for 10.50.X.X will go to ECOMM ASA on 10.XX.0.2 interface, from ECOMM route to Open ASA 172.11.22.13, from open asa to core switch 10.11.22.254 itd 

now open ASA and ECOMM asa are connected with cable together and interfaces are set open asa 172.11.22.13 and ECOMM asa 172.11.22.14 they are not connected via switch so not sure if this make a difference, I have configured subinterface n the ECOMM ASA as 10.11.22.240 and configured route for 10.50.XX.XX to go via 10.11.22.240 vlan and ecomm ASA can traceroute back to the shop in this situation but I still can't reach AWS from the shop.

hope this make sense

[chrihussey]

1- Can you do a trace route from AWS Router to 10.11.22.249 (MPLS ASA)?

2- Can you do a trace route from MPLS ASA to 10.XX.0.1 (AWS Router)?

3- When you do the trace route from AWS Router to the Shop, do you stop it after the first non response? If so, can you let it continue for some additional hops just to see if you start seeing a response?

[Mieczyslaw]

when i do


1) trace route from AWS Router to 10.11.22.249 (MPLS ASA)


it will go only up to the core switch 10.11.22.254 and than I have * * * *


2) when I do do a trace route from MPLS ASA to 10.XX.0.1 (AWS Router)?


it will go up to the ecomm ASA 172.XX.XX.14 and than I have * * * *


3) I let it to continue but it will go for many lines as * * * *

[chrihussey]

So fundamentally everything looks correct and it is somewhat puzzling.

However, it appears that the MPLS ASA, CORE SWITCH and OPEN ASA share a common LAN segment 10.11.22.xxx. If that is the case, why is it necessary for the MPLS ASA to go to the CORE SWITCH and then go to the OPEN ASA to get to AWS and not go directly to the OPEN ASA and vice versa and avoid the two hops in the same subnet and possible redirects?

The only reason I am asking is because from the CORE switch to the shop and to AWS the trace route succeeds. This would be the only instance where this would not be occurring. So without this redirect it appears to work.

It might not be anything, but just pointing it out.

[Mieczyslaw]

this is how the other routes were set up so I just wanted to keep it consistent, the only difference is that this is the first route going from the shop over MPLS, all works from the HO over to AWS 10.XX6.0.0 so I know the AWS part works


even if I skip the core switch I still can't traceroute from the open ASA to the shop and from MPLS ASA to AWS

[chrihussey]

It's difficulty to say what's happening. Short of the trace route anomaly, things look OK. I'd verify the Shop's network is being advertised to AWS from the AWS Router. That seems to be the only unknown.

Other than that the next step might just be to get a packet capture. Perhaps at the AWS router, just to see what's happening to the packets from the shop. Not sure what type router it is but if it has the remote packet capture feature it could provide some answers.

[Mieczyslaw]

turns out all to be ok internally , was an issue with the return patch on the MPLS network so the ISP needed to advertise the aws router 10.XX.0.1, the traceroutes were confusing though.

thanks for help.

[Mieczyslaw]

is there a way to delete the attachment now I changed the IP's so no big issues but just in case :) ?

[chrihussey]

Glad to hear you figured things out. Don't know about deleting the attachment. Not exactly sure how to do it, but you could try to reach out to one of the community forum moderators.

Regards