BGP route shows in routing table, but will not NAT

[mark-rodgers]

I've been tasked with setting up a new Catalyst 8000 to connect to a vendor through a vpn tunnel. We will have two tunnels configured with BGP and using NAT'd IPs to connect to his network. Vendor confirms both tunnels are up and he sees our advertised route (the NAT) and I see his routes added to my side and the "best" route added to my ip route table. I'm able to ping his side of the tunnel, but I'm unable to ping the test host. Looking at the nat translations table I never see the host added as I would expect.

UUSERCNCSR03#sho ip route

Gateway of last resort is 10.28.133.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 10.28.133.1
10.0.0.0/8 is variably subnetted, 4 subnets, 3 masks
C 10.28.132.128/25 is directly connected, GigabitEthernet3
L 10.28.132.134/32 is directly connected, GigabitEthernet3
C 10.28.133.0/24 is directly connected, GigabitEthernet1
L 10.28.133.6/32 is directly connected, GigabitEthernet1
100.0.0.0/30 is subnetted, 1 subnets
S 100.80.115.32 is directly connected, Null0
167.16.0.0/25 is subnetted, 1 subnets
B 167.16.0.0 [20/0] via 169.254.9.42, 02:30:09
168.63.0.0/32 is subnetted, 1 subnets
S 168.63.129.16 [254/0] via 10.28.133.1
169.254.0.0/16 is variably subnetted, 5 subnets, 2 masks
C 169.254.9.42/31 is directly connected, Tunnel500
L 169.254.9.43/32 is directly connected, Tunnel500
C 169.254.40.234/31 is directly connected, Tunnel600
L 169.254.40.235/32 is directly connected, Tunnel600
S 169.254.169.254/32 [254/0] via 10.28.133.1

UUSERCNCSR03#sho ip bgp

Network Next Hop Metric LocPrf Weight Path
*> 100.80.115.32/30 0.0.0.0 0 32768 i
*> 167.16.0.0/25 169.254.9.42 0 27160 4200017140 4200017142 65355 4200005845 4200005850 i
*                           170.186.194.23 0 27160 4200017340 4200017342 65355 4200005845 4200005850 i

I've added what I think it the pertinent configurations below.

Vendor has two endpoints that I've setup with Tunnel500 and Tunnel600

interface Tunnel500

description Chicago

ip address 169.254.9.43 255.255.255.254

tunnel source GigabitEthernet1

tunnel mode ipsec ipv4

tunnel destination 170.186.192.23

tunnel protection ipsec profile Chicago

!

interface Tunnel600

description Dallas

ip address 169.254.40.235 255.255.255.254

tunnel source GigabitEthernet1

tunnel mode ipsec ipv4

tunnel destination 170.186.194.23

tunnel protection ipsec profile Dallas

router bgp xxxxxxxxxx

bgp log-neighbor-changes

network 100.80.115.32 mask 255.255.255.252

neighbor 169.254.9.42 remote-as xxxxx

neighbor 169.254.9.42 soft-reconfiguration inbound

neighbor 169.254.9.42 route-map Chicago-Next-Hop in

neighbor 169.254.40.234 remote-as xxxx

neighbor 169.254.40.234 soft-reconfiguration inbound

neighbor 169.254.40.234 route-map Dallas-Next-Hop in

ip route 0.0.0.0 0.0.0.0 10.28.133.1

ip route 100.80.115.32 255.255.255.252 Null0

ip nat pool POOL 100.80.115.33 100.80.115.33 netmask 255.255.255.252

ip nat inside source list CSR-NAT-List pool POOL overload

ip prefix-list Chicago seq 5 permit 167.16.0.0/25

ip prefix-list Dallas seq 5 permit 167.16.0.0/25

ip access-list extended CSR-NAT-List

10 permit icmp 10.28.0.0 0.0.255.255 any

route-map Chicago-Next-Hop permit 10

match ip address prefix-list Chicago

set ip next-hop 169.254.9.42

!

route-map Dallas-Next-Hop permit 10

match ip address prefix-list Dallas

set ip next-hop 170.186.194.23

You'll notice (for troubleshooting) I've configured the next hop for Chicago as the vendors side of the tunnel. For Dallas I've configured the next hop as the public IP of the Dallas tunnel.

With the current setup the Chicago BGP route comes back as valid and is added to the route table. Pinging the test host (167.16.0.125) I never get an entry into the ip route table.

UUSERCNCSR03#ping 167.16.0.125
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 167.16.0.125, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
UUSERCNCSR03#sho ip nat translations
Total number of translations: 0

If I disable the interface for tunnel500 (chicago) the route disappears from bgp and from the route table. The router never validates the Dallas route so it never gets added to the route table.  but now when I ping the test host, NAT gets applied.

UUSERCNCSR03#ping 167.16.0.125
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 167.16.0.125, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
UUSERCNCSR03#sho ip nat tran
UUSERCNCSR03#sho ip nat translations
Pro Inside global Inside local Outside local Outside global
icmp 100.80.115.33:73 10.28.133.6:73 167.16.0.125:73 167.16.0.125:73
Total number of translations: 1

[David Ruess]

Hello,

Have you configured a NAT inside and a NAT outside interface so the device knows where to make the translations? I didnt see it in the config?

ip nat inside/outside

-David

[mark-rodgers]

Hi David. I should have included the interfaces. Here they are.

interface VirtualPortGroup0
no ip address
ip nat inside
no mop enabled
no mop sysid
!
interface GigabitEthernet1
ip address dhcp
ip nat outside
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
ip address dhcp
no ip redirects
no ip unreachables
ip nat inside
negotiation auto
no mop enabled
no mop sysid

[Richard Burts]

There are aspects of your environment that I do not understand, and perhaps the solution is in one of those things. But it seems to me that your ip nat outside is assigned to GigabitEthernet1. That is the source address for the tunnel. But it is not the exit point for the tunnel. So it never recognizes that the traffic needs to be translated.

[mark-rodgers]

Hi Richard. Thank you for taking a look. Is there a log or configuration I can put in place to verify the traffic is or is not exiting the tunnel?

[MHM Cisco World]

you need to NAT traffic to tunnel IP but the IP I see in NAT pool is not Tunnel IP 

also you dont config tunnel as ip nat outside 

MHM

[David Ruess]

I believe you need to configure the outside interface on the tunnel as traffic is encapsulated before it exits the physical interface. Same applies with QoS as there is a command to apply interface QoS config to a tunnel interface using it as a source.

That and make sure your NAT POOL is suing a pool of addresses reachable from the other side.

-David.

[Harold Ritter]

Hi @mark-rodgers ,

In addition to all the excellent suggestions that have been made so far, I would add that you should use the default next hop address (eBGP peering address) for the BGP updates received from Dallas and Chicago as you want the traffic to traverse the tunnels. There is no need to explicitly set it like you do for Chicago, as it is the default anyway. You should remove the inbound route-map for both Chicago and Dallas.

Regards,

[mark-rodgers]

Hi Harold. I was using an example https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/118977-config-ebgp-00.html where the route map is used to change the next hop to the interface rather than the tunnel. Removing the route map from both tunnels does get me into a state where the chosen route is being written to the ip table. So I do think what you propose is the correct configuration. I’m still unable to get to the host on the other side of the tunnel. Any advice on how to troubleshoot the flow and get the darn thing NAT’d is greatly appreciated.

UUSERCNCSR03#sho ip bgp
BGP table version is 5, local router ID is 169.254.40.235

Network Next Hop Metric LocPrf Weight Path
*> 100.80.115.32/30 0.0.0.0 0 32768 i
* 167.16.0.0/25 169.254.40.234 0 27160 4200017340 4200017342 65355 4200005845 4200005850 i
*> 169.254.9.42 0 27160 4200017140 4200017142 65355 4200005845 4200005850 i
* 204.194.133.0 169.254.40.234 0 27160 4200017340 4200017342 65355 4200005845 4200005850 i
*> 169.254.9.42 0 27160 4200017140 4200017142 65355 4200005845 4200005850 i
* 204.194.134.0 169.254.40.234 0 27160 4200017340 4200017342 65355 4200006839 4200006850 i
*> 169.254.9.42 0 27160 4200017140 4200017142 65355 4200006839 4200006850 i

UUSERCNCSR03#sho ip route
Gateway of last resort is 10.28.133.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 10.28.133.1
10.0.0.0/8 is variably subnetted, 4 subnets, 3 masks
C 10.28.132.128/25 is directly connected, GigabitEthernet3
L 10.28.132.134/32 is directly connected, GigabitEthernet3
C 10.28.133.0/24 is directly connected, GigabitEthernet1
L 10.28.133.6/32 is directly connected, GigabitEthernet1
100.0.0.0/30 is subnetted, 1 subnets
S 100.80.115.32 is directly connected, Null0
167.16.0.0/25 is subnetted, 1 subnets
B 167.16.0.0 [20/0] via 169.254.9.42, 00:09:36
168.63.0.0/32 is subnetted, 1 subnets
S 168.63.129.16 [254/0] via 10.28.133.1
169.254.0.0/16 is variably subnetted, 5 subnets, 2 masks
C 169.254.9.42/31 is directly connected, Tunnel500
L 169.254.9.43/32 is directly connected, Tunnel500
C 169.254.40.234/31 is directly connected, Tunnel600
L 169.254.40.235/32 is directly connected, Tunnel600
S 169.254.169.254/32 [254/0] via 10.28.133.1
B 204.194.133.0/24 [20/0] via 169.254.9.42, 00:09:36
B 204.194.134.0/24 [20/0] via 169.254.9.42, 00:09:36


UUSERCNCSR03#sho ip route 167.16.0.125
Routing entry for 167.16.0.0/25
Known via "bgp 4200009690", distance 20, metric 0
Tag 27160, type external
Last update from 169.254.9.42 00:02:08 ago
Routing Descriptor Blocks:
* 169.254.9.42, from 169.254.9.42, 00:02:08 ago
opaque_ptr 0x744EF0F55A30
Route metric is 0, traffic share count is 1
AS Hops 6
Route tag 27160
MPLS label: none

UUSERCNCSR03#sho ip bgp 167.16.0.0
BGP routing table entry for 167.16.0.0/25, version 2
Paths: (2 available, best #2, table default)
Advertised to update-groups:
7
Refresh Epoch 1
27160 4200017340 4200017342 65355 4200005845 4200005850, (received & used)
169.254.40.234 from 169.254.40.234 (10.100.10.1)
Origin IGP, localpref 100, valid, external
Community: 4227860054 4227860055 4227860059 4227865101 4227881032 4227882642 4227919442 4227920652
Extended Community: SoO:65355:55 RT:65355:9
rx pathid: 0, tx pathid: 0
Updated on May 28 2024 16:12:58 EDT
Refresh Epoch 1
27160 4200017140 4200017142 65355 4200005845 4200005850, (received & used)
169.254.9.42 from 169.254.9.42 (10.100.10.1)
Origin IGP, localpref 100, valid, external, best
Community: 4227860054 4227860055 4227860059 4227865101 4227879632 4227881032 4227919442 4227920652
Extended Community: SoO:65355:55 RT:65355:9
rx pathid: 0, tx pathid: 0x0
Updated on May 28 2024 16:12:58 EDT

[Harold Ritter]

Hi @mark-rodgers ,

As others have suggested, you need to apply the "ip nat inside" and "ip nat outside" to the right interfaces.

The "ip nat inside" needs to be configured on the ingress L3 interface, not on VirtualPortGroup0, which has no ip address.

  The "ip nat outside" needs to be applied to the tunnel interfaces (tu500 and tu600), not the physical interfaces (gig1 and gig3). 

Regards,

[mark-rodgers]

Thank you Harold. You mention L3.  I don't have a loopback configured.  Do I need one?  And theVirtualPortGroup0 was a holdover from something else.  I removed it.

[Harold Ritter]

Hi @mark-rodgers ,

> I don't have a loopback configured.  Do I need one?

No, you don't. The ingress L3 interface I am referring to is the interface through which the IP traffic destined to the test host will arrive. This is the interface that requires the "ip nat inside".

Regards,

[MHM Cisco World]

Do I need LO? This Q is important here 

As I understand you get one public IP from SP and you must use it for all your network behind tunnel?

If that correct then 

Config LO with public IP and NAT the traffic to this LO.

You need also PBR to redirect traffic to LO to NAT the  it will go through tunnel.

If you can use this public IP as tunnel IP then that better and ypu dont need LO interface 

MHM