BGP Route issue

[CliveG]

Hi,

I have a 3 datacentre architecture. See below:

DC1: Connection to DC2 and DC3
DC2: Connection to Transit ISP 1, DC1 and X-Connect to DC3
DC3: Connection to Transit ISP 2, DC1 and X-Connect to DC2

If we look at the above, it is basically a triangle with a Transit ISP off DC2 and DC3.

Originally all of our routing externally was via the Transit ISP at DC3 (Transit ISP 2). However, the uplinks were not great. We upgraded the connectivity to Transit ISP 1 at DC2 and I have swapped BGP advertisement to go only appear from DC2.

The problem I have is that the routing table shows that the route is learned correctly at DC1 directly from DC2 but yet when I traceroute to anywhere external it still goes to DC3 first, then the X-Connect to DC2 and then to Tansit ISP 1.

I have an inherited network that was never configured fully in the first place and we must ensure that we have no "downtime" when any work is being completed.

We are utilising ISIS at Level 2 only as the IGP and there is a little MPLS.

My guess is that the routing is behaving this way because of something to do with ISIS (or maybe MPLS), but I just cannot seem to find why the following ishappening:

DC1 learns the routes from DC2 (as expected).
DC1 is sending all traffic to DC1 bound for external via DC3 instead of direct.

Anyone got any advice of where to look for the problem please?

As an add on, both upstream links from DC1 to DC2 and DC3 are 10gbps PPP.

Many thanks

[MHM Cisco World]

Only  GW can routing traffic for subnet.

Here you xconnect three DC so the traffic bridge until DC3 where there it routing.

[CliveG]

I agree with you with regards to the Gateway for the Subnets. It has to route traffic to DC3 if they exist there.

The problem is that if I ping 8.8.8.8 from DC1, that also goes via THN.

The subnets whose gateways are on DC1 are also still traversing DC3 to DC2 and then Transit.

So, something in the routing is not right.

[MHM Cisco World]

Do traceroute from host connect to DC1 and see if gw of Dc1 appear or gw of DC3.

[CliveG]

Thanks

Here is what happens:

1: Hit's the Gateway for the VLAN on DC1 VSS
2: It then traverses to another VLAN
3: This VLAN sends the packet to the firewall.
4: The firewall is a pfsense that I know nothing about. Maybe there is a NAT in place that forwards the traffic to DC3 before being directed correctly to DC2

I need to investigate this a little further.

[MHM Cisco World]

Hope you find solution.

Have a nice day 

MHM

[CliveG]

Many thanks for the help.

The pfsense has a default route that points to the second VLAN (First VLAN 3249 - Second VLAN 2500) Gateway, which is located on DC1.

I will investigate further and put the answer here (if I find one) 

[CliveG]

Looks to me like there has been, as I suspected, traffic engineered redirection with MPLS/VPLS as shown below:

pseudowire-class <Name>
encapsulation mpls
load-balance flow

Any ideas how I can change this with no disruption on a live network?