use BGP to access serial interface for failover, but keep main connection on ethernet interface?

[ChuckHaynes]

Greetings,

We have three seperate sites in close proximity that all had T1s as their sole source for their WAN connections. We were using EIGRP internally, and BGP externally. We just added a fourth site that uses a high speed cable connection. We've now connected all four locations together via fiber and route them all through the cable connection. We've removed EIGRP and just use static routes internally. We are keeping the T1s for failover (manual). I had to remove BGP to get the routing working correctly through the cable connection. Currently, we can still access the three routers via their LAN address only (as I have disabled BGP). I would like to re-enable BGP (in some sorts) so that I can access the routers via their WAN interface as well. However, I don't want the serial interfaces to be used for any routing whatsoever (at this time). I just want to use the WAN ports as a back door in case of a fiber break. Is this possible?

Thanks

[Masoud Pourshabanian]

Hello,

I am not clear about your topology. Please share your topology. If you have two links on one router and you want the routes coming fiber to be prefered, you need to configure BGP weight attribute.

As an example,

router bgp 100

 neighbor 1.1.1.1 weight 200 [ fiber]

  neighbor   2.2.2.2 weight 100  [T1]

Masoud

[ChuckHaynes]

Ok, perhaps I wasn't clear. I don't really want to use BGP for anything at all right now. I simply want to be able to access the three sites via their WAN interfaces. Before the cable connection, we could access the WAN sides via BGP. Now that BGP is gone, we no longer have access to outside interfaces on those routers. We can only access them via their LAN interfaces (through the cable connection). I don't want to create any failover routing or redundancy right now. I just want to be able to access the WAN interfaces on the routers in case the fiber breaks or the cable connection is down.

[Masoud Pourshabanian]

It would be more clear If you sketch your topology and post. By access to Wan interface, you mean SSH access?I am sorry. I am not clear yet.

Masoud

[ChuckHaynes]

Here is a quick sketch of the topology. The three locations with routers use router-on-a-stick (as opposed to a layer 3 core switch) for routing. The BGP routes kept taking prescedence over the directly connected static routes, so I disabled BGP. When I did this, it allowed me to achieve my goal and route everying internally over the fiber (instead of going out the serial interfaces and traveling between sites that way). Also, Site 4 is now the external gateway for all four locations (as it's Internet speed is a lot faster). However, when I disabled BGP, I lost access to the WAN ports on the routers from our Corporate HQ. I would like to still be able to access the WAN interfaces on the router (from the AVPN), just in case there is an internal problem and I can't get to the LAN interfaces of the routers.

[Masoud Pourshabanian]

Hello,

Thanks for the topology. It is hard giving a precise solution without having the complete configuration and the IP addresses scheme, but I just give you a turn around to solve your problem.

Suppose you have a subnet in AVPN for the managment purpose. I suppose you have a computer with an IP 192.168.2.2 for managment.

I also supposed these IPs are configured on T1 links

Site 1 T1  1.1.1.1 (on site 1)   1.1.1.2 (on AVPN)

site 2 T1    2.2.2.1                    2.2.2.2

site 3  T1   3.3.3.1                   3.3.3.2

Currently on the router with T1 connection (on sites), you have default route toward your internel network.

On each router(with T1 connection) on Site1, 2 , 3 , configure an static route toward that managmet IP on T1 link.

Some thing like this.

on site 1: ip route 192.168.2.2 255.255.255.255 1.1.1.2

on site 2: ip route 192.168.2.2 255.255.255.255 2.2.2.2

on site 3: ip route 192.168.2.2 255.255.255.255 3.3.3.2

If for example site1 internal connection is disconnected, you can ssh to 1.1.1.1 on that computer with an IP 192.168.2.2 and the reply will come back from the T1 link.

Make sure on AVPN,192.168.2.2 has route toward 1.1.1.1 and 2.2.2.1 and 3.3.3.1.

Please give me feedback after configuration,

Masoud

[ChuckHaynes]

Ok, I am reviewing what you said. Ideally, I would like to be able to access the WAN side from everything (or more than just a management PC).

[Masoud Pourshabanian]

There are two other ways if you do not want to use routing protocols.

One is using NAT on AVPN. for example, you can translate source of any packets destined to 1.1.1.1  to 1.1.1.2. In this way, packets will go with destination address of 1.1.1.1 and source address of 1.1.1.2 so the reply will come back because source and destination addresses are in the same range. In this way, any device which is in your NAT access-list can access to the T1 IPs.

Another way is using IP SLA which is very complicated and I do not suggest it.

Hope it helps,

Masoud

[ChuckHaynes]

Maybe the best option would be to just re-enable BGP on the WAN side and then make the required changes so that BGP doesn't over ride the internal routes?

[Masoud Pourshabanian]

Routing protocol is absolutely is a better option. Traffic also can travel on t1 links in case of any link failure; however, i am not sure why you are using BGP.

[ChuckHaynes]

The AVPN that I referred to is AT&T's AVPN network. They use BGP, so that's what we've been using on our WAN. We have also been running EIGRP internally. The problem with the AVPN network is that it's SLOW! Two of the three locations had a single T1 (1.5Mbps) and the third site has 3 T1s (4.5Mbps). The fourth site has a Comcast cable connection (10Mbps). We just completed a project where we have fiber (1Gbps) running between are four sites and then we upgraded the fourth site (Comcast) to 150Mbps and route all four locations through there for speed. Ideally, we would like to keep the T1s active and use them for automatic failover in case the Comcast connection id down or there is a fiber break. I have my CCENT and I'm about 75% done with my CCNA, but I wasn't sure if I was ready to configure the automatic failover scenario yet, so I was just hopefully to keep the T1s accessible so that I could manually route over them if needed.

[Masoud Pourshabanian]

If you do not feel confident enough, I do not suggest you to change your network. You may get some help here, but you are not going to get the complete solution because what we see here is might be different from what you are facing. And your network is not small to do some small changes and then done.

Try to use the second options which was NAT until you are ready to configure failover. if you POST your router configuration on AVPN, I can give you commands for NAT.

Masoud

[ChuckHaynes]

One other question...

Previsouly, wehen we were running over the AVPN network, traceroutes looked "normal". We would see approximately 5-6 hops or so and nothing seemed strange. Now that we are running over over the Comcast connection, I am seeing something strange. If I traceroute from the remote site to the corporate office, I see two hops... the internal layer 3 switch, then then layer three switch at the coorporate office (which I would expect). However, when I do the traceroute from the corporate office (or another external site), I see three hops. I see the last hop (router, server, whatever I try to traceroute) twice. The final hop is shown two times. Does that make sense?

[Masoud Pourshabanian]

Do you have any Tunnel between corporate and sites?

[ChuckHaynes]

Yes. Since the primary connection is running over the Internet now (as opposed to the more secure AVPN), we do have an IPSec tunnel that connects the remotes site to the corporate HQ. However, our other firewall site don't display the last hop twice?