Problem with extending VRF instance over WAN

shanemoss · ‎02-07-2012

Hi Folks,

I have a network where we use several VRF instances to segregate the network, routing between VRFs is via a firewall. We use BGP to exchange routes between the VRFs and MPLS to provide next hop connectivty, EIGRP is used as the enterprise routing protocol. This works just fine in the LAN environment. I now wish to extend a particular VRF to a remote router across a WAN connection so that I can firewall a particular subnet attached to the remote router. I have configured the VRFs, BGP and MPLS but cannot ping between the networks - any help would be appreciated.

Thanks.

! The following is the relevent config on the remote router

ip vrf WRN

rd 65000:6760

route-target export 65000:760

route-target import 65000:760

!

mpls label protocol ldp

!

interface Loopback4

ip address 172.16.90.4 255.255.255.255

!

interface GigabitEthernet0/0

description #### WAN Link ####

ip address 172.17.91.2 255.255.255.252

mpls ip

interface GigabitEthernet0/1.93

encapsulation dot1Q 93

ip vrf forwarding WRN

ip address 172.16.93.1 255.255.255.0

!

router bgp 65000

no bgp default ipv4-unicast

no bgp default route-target filter

bgp log-neighbor-changes

neighbor 10.3.0.1 remote-as 65000

neighbor 10.3.0.1 update-source Loopback4

!

address-family ipv4

redistribute connected

no auto-summary

no synchronization

exit-address-family

!

address-family vpnv4

neighbor 10.3.0.1 activate

neighbor 10.3.0.1 send-community extended

exit-address-family

!

address-family ipv4 vrf WRN

redistribute connected

neighbor 10.3.0.1 remote-as 65000

neighbor 10.3.0.1 update-source Loopback4

neighbor 10.3.0.1 activate

neighbor 10.3.0.1 next-hop-self

no synchronization

exit-address-family

!

mpls ldp router-id Loopback4 force

Local Router

ip vrf WRN

rd 65000:3760

route-target export 65000:760

route-target import 65000:760

!

mpls label protocol ldp

tag-switching tdp router-id Loopback0 force

!

interface Loopback0

ip address 10.3.0.1 255.255.255.255

!

! This is active on a physical inetrface

interface Vlan760

description ### FW_WRN ###

ip vrf forwarding WRN

ip address 10.200.2.19 255.255.255.240

!

router bgp 65000

no bgp default ipv4-unicast

no bgp default route-target filter

bgp log-neighbor-changes

neighbor XYZ peer-group

neighbor XYZ remote-as 65000

neighbor XYZ update-source Loopback0

neighbor XYZ timers 20 60

neighbor 172.16.90.4 peer-group XYZ

!

address-family ipv4

redistribute connected

redistribute static

default-information originate

no auto-summary

no synchronization

exit-address-family

!

address-family vpnv4

neighbor XYZ activate

neighbor XYZ route-reflector-client

neighbor XYZ next-hop-self

neighbor XYZ send-community extended

neighbor 172.16.90.4 peer-group XYZ

bgp scan-time import 5

bgp scan-time 20

exit-address-family

!

address-family ipv4 vrf WRN

redistribute connected

redistribute static

default-information originate

no auto-summary

no synchronization

exit-address-family

!

ip route vrf WRN 0.0.0.0 0.0.0.0 10.200.2.30

!

On the remote router I can see the default route that has been propogated from the local router

C 172.16.93.0 is directly connected, GigabitEthernet0/1.93

10.0.0.0/28 is subnetted, 1 subnets

B 10.200.2.16 [200/0] via 10.3.0.1, 00:50:55

B* 0.0.0.0/0 [200/0] via 10.3.0.1, 00:50:55

But I cannot ping any devices in the 10.200.2.16 subnet, the following is also an issue for me; why is it active rather than established? And if it isn't established, why am I seeing routes advertised by this 10.3.0.1 router?

Router# sho ip bgp vpnv4 vrf WRN summary

BGP router identifier 172.16.90.4, local AS number 65000

BGP table version is 102, main routing table version 102

3 network entries using 411 bytes of memory

5 path entries using 340 bytes of memory

25/23 BGP path/bestpath attribute entries using 3100 bytes of memory

4 BGP rrinfo entries using 96 bytes of memory

13 BGP extended community entries using 312 bytes of memory

0 BGP route-map cache entries using 0 bytes of memory

0 BGP filter-list cache entries using 0 bytes of memory

BGP using 4259 total bytes of memory

BGP activity 103/0 prefixes, 105/0 paths, scan interval 15 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

10.3.0.1 4 65000 0 0 0 0 0 never Active

Dan-Ciprian Cicioiu · ‎02-07-2012

I think that the routes that you are seing are on the main routing table, not on the WRN table. You have one more adj. on ipv4 address family. Check :

show ip bgp all summary

First of all the adj on the address family vrf WRN is made with source/dst Loop 0 and Loop 4 , that is not on the vrf WRN.

Is there a layer 2 connection between the two routers , or how is the transport of the VRF WRN made over the WAN ?

Dan

shanemoss · ‎02-07-2012

Hi Dan,

You are correct, the routes are on the main routing table. I've removed the neighbor from the main bgp config so now it looks like the following. There is no layer 2 connection between these two routers. My understanding was that MPLS can be used instead of requiring direct L2 access between peers? My problem may be that MPLS is not set up correctly?

Thanks,

Shane.

router bgp 65000

no bgp default ipv4-unicast

no bgp default route-target filter

bgp log-neighbor-changes

!

address-family ipv4

redistribute connected

no auto-summary

no synchronization

exit-address-family

!

address-family vpnv4

neighbor 10.3.0.1 activate

neighbor 10.3.0.1 send-community extended

exit-address-family

!

address-family ipv4 vrf WRN

redistribute connected

neighbor 10.3.0.1 remote-as 65000

neighbor 10.3.0.1 update-source Loopback4

neighbor 10.3.0.1 activate

neighbor 10.3.0.1 next-hop-self

no synchronization

exit-address-family

! I've since added 10.5.0.1 as an neighbor so you'll see this listed in the output below.

Router#show ip bgp all summary

For address family: VPNv4 Unicast

BGP router identifier 172.16.90.4, local AS number 65000

BGP table version is 201, main routing table version 201

1 network entries using 137 bytes of memory

1 path entries using 68 bytes of memory

3/1 BGP path/bestpath attribute entries using 372 bytes of memory

1 BGP extended community entries using 24 bytes of memory

0 BGP route-map cache entries using 0 bytes of memory

0 BGP filter-list cache entries using 0 bytes of memory

BGP using 601 total bytes of memory

BGP activity 103/97 prefixes, 105/99 paths, scan interval 15 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

10.3.0.1 4 65000 0 0 0 0 0 never Active

10.5.0.1 4 65000 0 0 0 0 0 never Active

Router#

Dan-Ciprian Cicioiu · ‎02-07-2012

In order to use MPLS you will need layer2 conectivity between those 2 routers or an mpls cloud which will exchange mpls labels in order to have end-to-end LSP created between the routers.

Is there any consideration for the mpls configuration ? From my understading you need to divide some traffic between 2 locations , is my understanding correct ? You can use only VRF-lite ( the same technology with out mpls label exchange ).

Can you explane mode your topology , and what are you tring to achive ?

Dan

shanemoss · ‎02-08-2012

I have enabled MPLS LDP end to end. It is possible that there is something wrong in my MPLS configuration. The VRF routing table looks correct on both sides;

Router#sho ip route vrf WRN

Routing Table: WRN

Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

o - ODR, P - periodic downloaded static route

Gateway of last resort is 10.3.0.1 to network 0.0.0.0

172.16.0.0/24 is subnetted, 1 subnets

C 172.16.93.0 is directly connected, GigabitEthernet0/1.93

10.0.0.0/28 is subnetted, 1 subnets

B 10.200.2.16 [200/0] via 10.3.0.1, 02:28:20

B* 0.0.0.0/0 [200/0] via 10.3.0.1, 02:28:20

But I still cannot ping. The MPLS forwarding looks correct, the command;

Router#sho ip bgp vpnv4 vrf WRN labels

Network Next Hop In label/Out label

Route Distinguisher: 65000:6760 (WRN)

0.0.0.0 10.5.0.1 nolabel/223

10.3.0.1 nolabel/203

10.200.2.16/28 10.5.0.1 nolabel/221

10.3.0.1 nolabel/202

172.16.93.0/24 0.0.0.0 16/aggregate(WRN)

shows label 202 or 221 to reach network 10.200.2.16/28, these appear to be correct end to end - has anyone any suggestions why I cannot pass traffic across the network between these two VRF interfaces?

Thanks in advance.

wilson-danny · ‎02-08-2012

What about your Vlans?

shanemoss · ‎02-08-2012

What do you mean about the VLANs?

wilson-danny · ‎02-08-2012

Shane I'm whipping up a basic config in GNS.. I will paste it shortly.

Dan-Ciprian Cicioiu · ‎02-08-2012

Hi,

MPLS it's not black magic, the packets need to be trasfered over a layer 2 and layer 1 transport . The thing with the mpls is that there is no IP DST check as in normal IP routing. There are some label distribution protocols ( LDP or TDP ) that are generating those labels based on the routing table. In order to exchange labels you need to have LDP or TDP adjancency between the routers that are running MPLS. I do belive that your locations are connected via a provider that maybe is providing you this connection via MPLS. Can you confirm ?

You can check "your" LDP adj with : sh mpls ldp neighbor

Dan

shanemoss · ‎02-09-2012

Thanks Dan,

I do have end to end layer 2 connectivity. Our core network uses MPLS, our WAN link to the remote site is a point to point link with an ethernet presentation - no MPLS from the provider.

Router#sho mpls ldp neighbor

Peer LDP Ident: 172.16.90.5:0; Local LDP Ident 172.16.90.4:0

TCP connection: 172.16.90.5.58102 - 172.16.90.4.646

State: Oper; Msgs sent/rcvd: 1380/1687; Downstream

Up time: 20:01:16

LDP discovery sources:

GigabitEthernet0/0, Src IP addr: 172.17.91.1

Addresses bound to peer LDP Ident:

172.17.91.6 172.16.90.1 172.16.90.3 172.16.90.5

172.17.91.1

Router#

172.17.91.1 is the router on the local side of the WAN link. My understanding is that vrf labels are not assigned by LDP but rather by BGP so;

Router#show ip bgp vpnv4 vrf WRN labels

Network Next Hop In label/Out label

Route Distinguisher: 65000:6760 (WRN)

0.0.0.0 10.5.0.1 nolabel/223

10.3.0.1 nolabel/203

10.200.2.16/28 10.5.0.1 nolabel/221

10.3.0.1 nolabel/202

172.16.93.0/24 0.0.0.0 16/aggregate(WRN)

Router#

I'm at a loss to explain why I cannot ping 10.200.2.16/28 addresses.

Thanks again,

Shane.

Dan-Ciprian Cicioiu · ‎02-09-2012

Can you paste from both ends :

show mpls interface

show mpls neigh

show mpls for

Dan

wilson-danny · ‎02-08-2012

Hey Shane,

I didn't get too much time to spend on it, but it works..

VLAN ISL Id: 100

Name: vrf

Media Type: Ethernet

VLAN 802.10 Id: 100100

State: Operational

MTU: 1500

ip cef

no ip domain lookup

!

ip vrf test

rd 500:500

route-target export 500:500

route-target import 500:500

!

interface Loopback172

ip vrf forwarding test

ip address 172.16.0.2 255.255.255.255

!

interface FastEthernet0/0

ip address 10.0.0.2 255.255.255.0

duplex auto

speed auto

!

interface FastEthernet0/0.100

encapsulation dot1Q 100

ip vrf forwarding test

ip address 192.168.1.2 255.255.255.0

ip router isis

mpls label protocol ldp

mpls ip

!

router isis

net 59.0001.0000.0000.0004.00

is-type level-1

!

router bgp 500

no synchronization

bgp router-id 172.16.0.2

bgp log-neighbor-changes

no auto-summary

!

address-family ipv4 vrf test

redistribute static

redistribute eigrp

neighbor 192.168.1.1 remote-as 500

neighbor 192.168.1.1 ebgp-multihop 255

neighbor 192.168.1.1 update-source FastEthernet0/0.100

neighbor 192.168.1.1 activate

neighbor 192.168.1.1 send-community both

neighbor 192.168.1.1 soft-reconfiguration inbound

no synchronization

exit-address-family

!

no ip http server

no ip http secure-server

ip route vrf test 172.16.0.1 255.255.255.255 192.168.1.1

shanemoss · ‎02-13-2012

Hi folks,

I now believe that the problem is related specifically to MPLS and either the way it is configured or some issue with the WAN link. My setup is as follows;

Core Network -> Local WAN Router -> Remote Site Router

Issueing the command "show mpls forwarding-table" on any core router or the Local WAN router shows a lot of tags but the same command on the remote router shows only the directly attached network (The one configured as a vrf interface)

Router#sho mpls forwarding-table

Local Outgoing Prefix Bytes tag Outgoing Next Hop

tag tag or VC or Tunnel Id switched interface

17 Aggregate 172.16.93.0/24[V] 520

Router#

I would imagine that the bgp peer 10.3.0.1 should be listed here also but its not. I've tested my configuration by putting a loopback interface on the local wan router into the WRN vrf and have peered with the 10.3.0.1 neighbor and it all works fine, so it looks like there may be something wrong with MPLS not working across my WAN link. Any ideas what I could be missing? Thanks in advance.

shanemoss · ‎02-15-2012

Figured it out, the issue was due to the next hop not being in the MPLS forwarding-table of the remote router. This was because of a default summary route being advertised by EIGRP from the local router. Removal of the summary route meant that the host route to the next hop was then in the global routing table as opposed to a default route allowing a tag to be generated and populated into the MPLS forwarding-table. (A static route to the next hop also worked)

Thanks for your help.