Can't get 6VPE working

magnus.maatta · ‎12-19-2013

Hi,

I've been trying to debug this problem for a few days now but I can't figure out what is wrong.

I configured BGP according to documentation on 3 of my routers which will have IPv6 for now

but I will focus on two of them as they are directly connected without any IPv4 MPLS cloud between

them.

So the setup is basicly:

I couldn't get the connectivity going between the peering router and the primary/secondary router, so I moved on to testing

between primary and secondary router instead.

BGP is setup with

address-family vpnv6

neighbor 83.150.xxx.xxx activate

neighbor 83.150.xxx.xxx send-community both

neighbor 83.150.zzz.zzz activate

neighbor 83.150.zzz.zzz send-community both

exit-address-family

address-family ipv6 vrf vrf230

redistribute connected

redistribute static

no synchronization

exit-address-family

Of course with modifications on each router so that it points to the other ones (and IPv4 is working fine).

ipv6 unicast routing have been enabled, ipv6 cef also.

So I setup one interface with one address on the primary router and another interface on the secondary router

and tried to ping from the vrf with IPv6 enabled (first making sure all tables look correct etc)

Secondary router preparations before test

secondary#show ipv6 cef vrf vrf230 2A00:1208:1000:100::FFFE/64 det

2A00:1208:1000:100::/64, epoch 1, flags attached, connected

attached to GigabitEthernet1/3.117

secondary#show mpls forwarding-table labels 45

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

45 Pop Label IPv6 VRF[V] 8664 aggregate/vrf230

secondary#show mpls forwarding-table vrf vrf230 2A00:1208:0:10E6:1000:10B0:1:101/126

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

None 53 2A00:1208:0:10E6:1000:10B0:1:100/12[V] \

0 Po1 83.150.188.129

secondary#show ipv6 cef vrf vrf230 2A00:1208:0:10E6:1000:10B0:1:101/126 det

2A00:1208:0:10E6:1000:10B0:1:100/126, epoch 1, flags rib defined all labels

recursive via 83.150.188.1 label 53

nexthop 83.150.188.129 Port-channel1 label explicit-null

secondary(config)#access-list 2700 permit any 45 any any

secondary#debug mpls packet 2700

Packet debugging is on with ACL 2700

secondary#debug ipv6 packet

3d08h: IPV6: source FE80::20B:45FF:FEB5:2F40 (local)

3d08h: dest FF02::1 (Port-channel1)

3d08h: traffic class 224, flow 0x0, len 72+0, prot 58, hops 255, originating

3d08h: IPv6: Sending on Port-channel1

Except for the above output and some other unrelated link-local activity there was no output at all.

Primary router preparation and test

This was done on the primary router (ping after enabling all debug on the secondary one).

primary#show mpls forwarding-table vrf vrf230 2A00:1208:1000:100::FFFE/64

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

None 45 2A00:1208:1000:100::/64[V] \

0 Po1 83.150.188.130

primary#debug ipv6 packet

primary#ping vrf vrf230 ipv6 2A00:1208:1000:100::FFFE rep 1 source 2A00:1208:0:10E6:1000:10B0:1:101

Type escape sequence to abort.

Sending 1, 100-byte ICMP Echos to 2A00:1208:1000:100::FFFE, timeout is 2 seconds:

Packet sent with a source address of 2A00:1208:0:10E6:1000:10B0:1:101%vrf230

3d07h: IPv6: Looking up 2A00:1208:1000:100::FFFE [Source 2A00:1208:0:10E6:1000:10B0:1:101] in FIB

3d07h: FIBfwd-proc: vrf230:2A00:1208:1000:100::/64 proces level forwarding

3d07h: FIBfwd-proc: depth 0 first_idx 0 paths 1 long 0(0)

3d07h: FIBfwd-proc: try path 0 (of 1) v6-rnh-83.150.188.2[v4:Default] first short ext 67769AA8(0)

3d07h: FIBfwd-proc: v6-rnh-83.150.188.2[v4:Default] valid short

3d07h: FIBfwd-proc: label[0] 45 connid 0 link ILLEGAL

3d07h: FIBfwd-proc: ip_pak_table 503316481 ip_nh_table 0 if none nh 83.150.188.2 deag 0 via fib 527D5A98 path type recursive nexthop

3d07h: FIBfwd-proc: depth 1 first_idx 0 paths 1 long 0(0)

3d07h: FIBfwd-proc: try path 0 (of 1) v4-anh-83.150.188.130-Po1 first short ext 51244DC8(0)

3d07h: FIBfwd-proc: v4-anh-83.150.188.130-Po1 valid short

3d07h: FIBfwd-proc: label[1] 0 connid 0 link TAG

3d07h: FIBfwd-proc: ip_pak_table 503316481 ip_nh_table 65535 if Port-channel1 nh 83.150.188.130 deag 0 via fib 0 path type attached nexthop

3d07h: FIBfwd-proc: packet routed to Port-channel1 83.150.188.130(503316481) with labels 45 explicit-null

3d07h: IPv6: FIB lookup for 2A00:1208:1000:100::FFFE succeeded. if=Port-channel1, nexthop 2A00:1208:1000:100::FFFE

3d07h: IPV6: source 2A00:1208:0:10E6:1000:10B0:1:101 (local)

3d07h: dest 2A00:1208:1000:100::FFFE (Port-channel1)

3d07h: traffic class 0, flow 0x0, len 100+0, prot 58, hops 64, originating

3d07h: FIBfwd-proc: ip_pak_table 503316481 ip_nh_table 65535 if Port-channel1 nh 83.150.188.130 uhp 0 deag 0 ttlexp 0

3d07h: FIBfwd-proc: encapsulating link TAG ip_pak_table 503316481 ip_nh_table 65535 if Port-channel1 nh 83.150.188.130 uhp 0 deag 0 ttlexp 0

3d07h: IPv6: Sending on Port-channel1.

Success rate is 0 percent (0/1)

primary#show bgp vpnv6 unicast vrf vrf230

BGP table version is 55523, local router ID is 83.150.188.1

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

r RIB-failure, S Stale

Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

Route Distinguisher: 31331:230 (default for vrf vrf230)

*> 2A00:1208::/32 :: 0 32768 ?

* i2A00:1208:0:10E6:1000:1000:1:200/126

::FFFF:83.150.188.2

0 100 0 ?

*> :: 0 32768 ?

*> 2A00:1208:0:10E6:1000:10B0:1:100/126

:: 0 32768 ?

* i2A00:1208:1000::/64

::FFFF:83.150.188.2

0 100 0 ?

*> :: 0 32768 ?

*> 2A00:1208:1000:10::/64

:: 0 32768 ?

*>i2A00:1208:1000:100::/64

::FFFF:83.150.188.2

0 100 0 ?

*>i2A02:C58:F:B::/64

::FFFF:83.150.188.3

0 100 0 ?

primary#show ipv6 route vrf vrf230 2A00:1208:1000:100::FFFE/64

Routing entry for 2A00:1208:1000:100::/64

Known via "bgp 31331", distance 200, metric 0, type internal

Route count is 1/1, share count 0

Routing paths:

83.150.188.2%Default-IP-Routing-Table indirectly connected

MPLS label: 45

Last updated 1d22h ago

Ping works between routers on the linknet (if I add a linknet between them on a subinterface on the port-channel)

and also on one shared L2 subinterface where they both have an IP-address - but not via MPLS.

And as I said, the same goes for the internet router, can't ping that one either but to rule out any weirdness in the

MPLS cloud I tried debugging between the routers. I also get the same silence from the debugs if I turn the test the

other way around.

Anyone got any idea what's wrong ?

Version: 12.2(33)SXJ5

CPU: WS-SUP720-3BXL

Cisco 7606

Nagendra Kumar Nainar · ‎01-19-2014

Hi,

Do you have "mls ipv6 vrf" enabled?.

Thanks,

Nagendra

magnus.maatta · ‎01-19-2014

Hi Nagendra,

Yes. I've address-family ipv6 in the vrf also.

I should have included a trace also.

traceroute to 2a00:1208:1000::2 (2a00:1208:1000::2), 30 hops max, 40 byte packets
 1  2a02:348:82::1 (2a02:348:82::1)  1.808 ms  1.703 ms  1.642 ms
 2  rt-eu02-v2.xl-is.net (2a02:348:2::4)  0.536 ms  0.509 ms  0.488 ms
 3  xe-8-2-1.edge6.Amsterdam1.Level3.net (2001:1900:5:2:2::2775)  1.028 ms  1.145 ms  1.066 ms
 4  vl-52.edge4.Amsterdam1.Level3.net (2001:1900:102:2::9)  1.071 ms vl-51.edge4.Amsterdam1.Level3.net (2001:1900:102:1::9)  1.068 ms  1.036 ms
 5  2001:1900:5:3::216 (2001:1900:5:3::216)  1.368 ms  1.341 ms 2001:1900:5:3::212 (2001:1900:5:3::212)  1.367 ms
 6  kbn-b3-v6.telia.net (2001:2000:3018:2e::1)  38.779 ms  38.718 ms  38.596 ms
 7  mb-peer3-link.ipv6.telia.net (2001:2000:3080:30e::2)  16.272 ms  15.976 ms  16.245 ms
 8  2001:2000:4020:4b::2 (2001:2000:4020:4b::2)  27.796 ms  30.427 ms  30.301 ms
 9  2001:2000:4020:4b::2 (2001:2000:4020:4b::2)  31.971 ms  31.581 ms  30.778 ms
10  * * *
11  * * *
12  * * *
13  * * *
14  2a00:1208:1000::1 (2a00:1208:1000::1)  47.421 ms  47.133 ms  45.898 ms
15  * * *
16  * * *

Hop 14 is the router before the host ::2. It's also the router from which the output above have been collected.

Ping doesn't go through to ::1 from internet and not ::2 either. Trace does not work to the router, in that case it stops

on the hop before ::1.

eduardopozo56 · ‎01-22-2014

From where are you doing the traceroute? Does vrf230 (the one you´re debugging) know how to reach the internet?

In the "show bgp vpnv6 unicast vrf vrf230" i dont see any default route to the internet

Check your ipv6 routing tables for the routes to peering router/internet

Regards,

magnus.maatta · ‎01-22-2014

Due to some unrelated issues in the IPv6-transit the BGP peering was down in my original post, but came up again later that night.

I could paste all IPv6 internet routes in vrf230 but seems redundant so I'll paste the first hop (from the trace) instead:

#show ip bgp vpnv6 unicast vrf vrf230 2a02:348::/32

BGP routing table entry for [31331:230]2A02:348::/32, version 186828

Paths: (1 available, best #1, table vrf230)

Not advertised to any peer

29217 29217 3301 1299 3356 35470

::FFFF:83.150.188.3 (metric 80) from 83.150.188.3 (83.150.188.3)

Origin IGP, metric 0, localpref 100, valid, internal, best

Extended Community: RT:31331:230,

mpls labels in/out nolabel/113

ashirkar · ‎02-20-2014

Hi Magnus,

Did you able to resolve this issue ? Are you using any XR device in your setup.For 6VPE ,you form neighborship on IPv4 neighbor only.

Regards,

Ashish Shirkar

magnus.maatta · ‎02-20-2014

Hi

No, it still isn't working. All routers have peering setup with the IPv4 addresses as seen in the original post.

The primary, secondary and peering routers are all 7600's with same firmware.

I know the IPv6 transit provider is using ASR9001 on their side though

Regards,

Magnus Määttä

eduardopozo56 · ‎02-20-2014

Hi,

I think you went too far on the debugging and we are missing a more simple point/configuration

Can you repost the setup with the IPv4/v6 addresses used on the interfaces?

Show bgp config on all routers

Show ipv6 route vrf XXX on all routers

show mpls forwarding-table vrf XXX

show mpls forwarding-table

show bgp vpnv6 unicast summ

If you have an issue on basic MPLS labels, vrf Label wont work

Regards,

paul driver · ‎02-20-2014

Hello

Can you provide a topology diagram and preferably some running configs

Res
Paul

Sent from Cisco Technical Support iPad App

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

magnus.maatta · ‎02-25-2014

Hi,

Here's the topology. Will post configuration and show commands in separate post.

ashirkar · ‎02-25-2014

Hi Magnus,

Thanks for updating thread ,Let us know to which router you are forming neighborships ,Make sure you dont post any public IP address in your config.Did you form neighboship with 7606 dennis to 7604 alf and between them you are running MPLS ? bcoz in previous diagram it says 7606 is connected to MPLS cloud .

Regards,

Ashish Shirkar

magnus.maatta · ‎02-25-2014

Dennis, Daniel and Alf are IPv6 enabled and are BGP neighbours for address-family vpnv6. I just added all the other routers in the MPLS cloud in the second image, so alf would be the peering router in the first image.

magnus.maatta · ‎02-25-2014

I removed many thousand lines of output from various parts and tried to include what I thought was most important from the configurations, like ignoring the IPv4-only routers in the BGP config and many-many IPv4 access lists for peering and all interface configurations and other IPv4 VRF's configured.

Just to make it clear, I'm posting this via IPv4 from VRF vrf230, interface GigabitEthernet 1/1.8 found down a bit in the config. Same interface I've IPv6 enabled machines on which I'm trying to get working.

<<< removed configuration >>>

ashirkar · ‎02-25-2014

Hi Magnus ,

Does "2A00:1208:0:10E6:1000:10B0:1:101/126" network connected to 7604 ALF router ?

As per my understanding you are forming BGP VPNv6 neighborship between denial,denis and ALF routers and you are correctly getting routes from ALF,can u confirm you can ping local ipv6 interface assigned in VRF between this two routers

For Example:

On ALF if you have 2001:1 r locally connceted network inside VRF and you advertise this through VPNv6 to denis then can you ping this subnet from denis.

Regards,

Ashish

magnus.maatta · ‎02-25-2014

Hi,

Some javascript hung my browser so I have to rewrite my post.

No that network is a future linknet to a customer whenever 6VPE start working

The routers have no IPv6 linknets between them (the whole point of 6VPE).

As I wrote in the original post, ping works fine on directly connected interfaces or

interfaces sharing L2 network and the servers can ping dennis and daniel if they

are configured with an IP on that L2 network.

But, ping doesn't work between two different interfaces between the routers even

though the interfaces belong to the same VRF and the routes are correct.

Also as I wrote in another post, traceroute via IPv6 works to the second last hop

for example if I try to trace a server behind an interface on dennis, dennis will show

in the trace, but not the server. If I try to trace the IP on the interface on dennis, I

won't reach that one:

14  2a00:1208:1000::1 (2a00:1208:1000::1)  47.421 ms  47.133 ms  45.898 ms
15  * * *

The server had 2a00:1208:1000::2 and could ping the router (dennis) with

IP 2a00:1208:1000::1 without any problem.

Interfaces configure in VRF vrf230 on routers:

Dennis: 2A00:1208:1000::1/64

Daniel: 2a00:1208:1000:9::fffd/64

dennis#show ipv6 route vrf vrf230 2a00:1208:1000:9::/64

Routing entry for 2A00:1208:1000:9::/64

Known via "bgp 31331", distance 200, metric 0, type internal

Route count is 1/1, share count 0

Routing paths:

83.150.188.2%Default-IP-Routing-Table indirectly connected

MPLS label: 45

Last updated 00:05:52 ago

dennis#show ipv6 cef vrf vrf230 2a00:1208:1000:9::/64 detail

2A00:1208:1000:9::/64, epoch 1, flags rib defined all labels

recursive via 83.150.188.2 label 45

nexthop 83.150.188.130 Port-channel1 label explicit-null

dennis#show mpls forwarding-table labels 53 detail

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

53 Pop Label IPv6 VRF[V] 3362867 aggregate/vrf230

MAC/Encaps=0/0, MRU=0, Label Stack{}

VPN route: vrf230

No output feature configured

daniel#show ipv6 route vrf vrf230 2A00:1208:1000::/64

Routing entry for 2A00:1208:1000::/64

Known via "bgp 31331", distance 200, metric 0, type internal

Route count is 1/1, share count 0

Routing paths:

83.150.188.1%Default-IP-Routing-Table indirectly connected

MPLS label: 53

Last updated 03:01:13 ago

daniel#show ipv6 cef vrf vrf230 2A00:1208:1000::/64 detail

2A00:1208:1000::/64, epoch 1, flags rib defined all labels

recursive via 83.150.188.1 label 53

nexthop 83.150.188.129 Port-channel1 label explicit-null

daniel#show mpls forwarding-table labels 45 detail

Local Outgoing Prefix Bytes Label Outgoing Next Hop

Label Label or Tunnel Id Switched interface

45 Pop Label IPv6 VRF[V] 11944 aggregate/vrf230

MAC/Encaps=0/0, MRU=0, Label Stack{}

VPN route: vrf230

No output feature configured

With debug ipv6 packet

access-list 2700 any 45 any any

access-list 2700 any 53 any any

debug mpls packet 2700

on both routers, this is the only output (nothing on daniel)

dennis#ping vrf vrf230 ipv6 2a00:1208:1000:9::fffd source 2A00:1208:1000::1 repeat 1

Type escape sequence to abort.

Sending 1, 100-byte ICMP Echos to 2A00:1208:1000:9::FFFD, timeout is 2 seconds:

Packet sent with a source address of 2A00:1208:1000::1%vrf230

10w1d: IPv6: Looking up 2A00:1208:1000:9::FFFD [Source 2A00:1208:1000::1] in FIB

10w1d: FIBfwd-proc: vrf230:2A00:1208:1000:9::/64 proces level forwarding

10w1d: FIBfwd-proc: depth 0 first_idx 0 paths 1 long 0(0)

10w1d: FIBfwd-proc: try path 0 (of 1) v6-rnh-83.150.188.2[v4:Default] first short ext 645691F8(0)

10w1d: FIBfwd-proc: v6-rnh-83.150.188.2[v4:Default] valid short

10w1d: FIBfwd-proc: label[0] 45 connid 0 link ILLEGAL

10w1d: FIBfwd-proc: ip_pak_table 503316481 ip_nh_table 0 if none nh 83.150.188.2 deag 0 via fib 527D5A98 path type recursive nexthop

10w1d: FIBfwd-proc: depth 1 first_idx 0 paths 1 long 0(0)

10w1d: FIBfwd-proc: try path 0 (of 1) v4-anh-83.150.188.130-Po1 first short ext 51244DC8(0)

10w1d: FIBfwd-proc: v4-anh-83.150.188.130-Po1 valid short

10w1d: FIBfwd-proc: label[1] 0 connid 0 link TAG

10w1d: FIBfwd-proc: ip_pak_table 503316481 ip_nh_table 65535 if Port-channel1 nh 83.150.188.130 deag 0 via fib 0 path type attached nexthop

10w1d: FIBfwd-proc: packet routed to Port-channel1 83.150.188.130(503316481) with labels 45 explicit-null

10w1d: IPv6: FIB lookup for 2A00:1208:1000:9::FFFD succeeded. if=Port-channel1, nexthop 2A00:1208:1000:9::FFFD

10w1d: IPV6: source 2A00:1208:1000::1 (local)

10w1d: dest 2A00:1208:1000:9::FFFD (Port-channel1)

10w1d: traffic class 0, flow 0x0, len 100+0, prot 58, hops 64, originating

10w1d: FIBfwd-proc: ip_pak_table 503316481 ip_nh_table 65535 if Port-channel1 nh 83.150.188.130 uhp 0 deag 0 ttlexp 0

10w1d: FIBfwd-proc: encapsulating link TAG ip_pak_table 503316481 ip_nh_table 65535 if Port-channel1 nh 83.150.188.130 uhp 0 deag 0 ttlexp 0

10w1d: IPv6: Sending on Port-channel1.

Success rate is 0 percent (0/1)