Hi all,
I have 2 PoP (2 PES each), with no P in the middle, iBGP between them and just one VRF mgmt at the moment, I will call the first one "POP1" and the second one "POP2":
Scenario:
The "POP1" has an old network (192.168.102.0/24) directly connected which is in vrf forwarding 'mgmt', and has also connected a new network (192.168.111.0/24) which is in the same vrf too.
The "POP2" has only the new network (192.168.110.0/24) which is in vrf forwarding 'mgmt'.
Configuration POP1 (only for one of the 2 PEs):
vrf definition mgmt
rd 3.3.3.3:110
route-target export 110:110
route-target import 110:110
interface Loopback0
ip address 3.3.3.3 255.255.255.255
ip ospf 100 area 0
interface Port-channel1
description switch_port-channel
mtu 1560
no ip address
negotiation auto
!
interface Port-channel1.111
encapsulation dot1Q 111
vrf forwarding mgmt
ip address 192.168.111.2 255.255.255.0
standby 111 ip 192.168.111.1
standby 111 priority 110
standby 111 preempt
!
interface Port-channel1.120
description interface_to_reach_192.168.109.0
encapsulation dot1Q 120
vrf forwarding mgmt
ip address 192.168.102.200 255.255.255.0
!
interface Port-channel1.2
description connection_between_sites
encapsulation dot1Q 2
ip address 172.30.1.3 255.255.255.248
ip ospf 100 area 0
mpls ip
mpls mtu 1512
router bgp 12345
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 12345
neighbor 1.1.1.1 ebgp-multihop 2
neighbor 1.1.1.1 update-source Loopback0
neighbor 1.1.1.1 description to_pe1_site2
neighbor 2.2.2.2 remote-as 12345
neighbor 2.2.2.2 ebgp-multihop 2
neighbor 2.2.2.2 update-source Loopback0
neighbor 2.2.2.2 description to_pe2_site2
neighbor 4.4.4.4 remote-as 12345
neighbor 4.4.4.4 ebgp-multihop 2
neighbor 4.4.4.4 update-source Loopback0
neighbor 4.4.4.4 description to_pe2_site1
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community both
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community both
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community both
exit-address-family
!
address-family ipv4 vrf mgmt
redistribute connected
maximum-paths ibgp 2
exit-address-family
!
ip route vrf mgmt 0.0.0.0 0.0.0.0 192.168.102.1
Configuration POP2 (only for one of the 2 PEs):
vrf definition mgmt
rd 1.1.1.1:110
route-target export 110:110
route-target import 110:110
interface Loopback0
ip address 1.1.1.1 255.255.255.255
ip ospf 100 area 0
interface Port-channel1
description switch_port-channel
mtu 1560
no ip address
negotiation auto
interface Port-channel1.110
description description Mgmt_Vlan_L3
encapsulation dot1Q 110
vrf forwarding mgmt
ip address 192.168.110.2 255.255.255.0
standby 110 ip 192.168.110.1
standby 110 priority 110
standby 110 preempt
interface Port-channel1.2
description connection_between_sites
encapsulation dot1Q 2
ip address 172.30.1.1 255.255.255.248
ip ospf 100 area 0
mpls ip
mpls mtu 1512
router bgp 12345
bgp log-neighbor-changes
redistribute connected
neighbor 2.2.2.2 remote-as 12345
neighbor 2.2.2.2 description to_pe2_site2
neighbor 2.2.2.2 ebgp-multihop 2
neighbor 2.2.2.2 update-source Loopback0
neighbor 3.3.3.3 remote-as 12345
neighbor 3.3.3.3 description to_pe1_site1
neighbor 3.3.3.3 ebgp-multihop 2
neighbor 3.3.3.3 update-source Loopback0
neighbor 4.4.4.4 remote-as 12345
neighbor 4.4.4.4 description to_pe2_site1
neighbor 4.4.4.4 ebgp-multihop 2
neighbor 4.4.4.4 update-source Loopback0
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community both
neighbor 3.3.3.3 activate
neighbor 3.3.3.3 send-community both
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community both
exit-address-family
!
address-family ipv4 vrf mgmt
redistribute connected
maximum-paths ibgp 2
exit-address-family
ip route vrf mgmt 0.0.0.0 0.0.0.0 192.168.102.200 default_via_to_pe1_site1
ip route vrf mgmt 0.0.0.0 0.0.0.0 192.168.102.201 default_via_to_pe2_site1
Goal:
I wanted to reach the new mgmt networks (POP1 and POP2) from the world behind the old network (using the 192.168.102.0/24 interface or next-hop as default-gateway).
How's gone:
Everything is worked fine, to do that I've attached a router simulating a host on the old network as next hop (192.168.102.1) towards the world behind it that I call "old world" (192.168.109.0/24), so I was reaching the old network from the new subnets and viceversa....
192.168.109.0/24 <---> 192.168.110.0/24
192.168.109.0/24 <---> 192.168.111.0/24
How I did that:
I don't know why, but I was trying to do something different as a lab test, and I've tried to configure a static default route (instead of share a default via BGP from the POP1) on the "POP2" (2 PEs) towards all the unknown networks in the "mgmt" vrf with the PE1 and PE2 on the POP1 as next-hops (ip addresses on the old subnet):
ip route 0.0.0.0 0.0.0.0 192.168.102.200
ip route 0.0.0.0 0.0.0.0 192.168.102.201
So the PEs in the POP2 were knowing the subnet 192.168.102.0/24 (because of the redistribute connected on the POP1 iBGP) and answering correctly to the old network.
Problem:
The weird problem was actually that I have 1 host on the new subnet on POP2 (which is a switch providing layer 2 service between PE and access network) with the IP 192.168.110.4 and I was trying to telnet it from 192.168.109.1, the strange thing was that I was able to ping it and access it randomly, sometime I was unable to access it from another terminal while connected on a first one, it was very frustrating, because i didn't understand yet why.
Solution:
I've deleted the static default routes (2 each) from the PEs on the POP2 and I've added the default route redistribution under the BGP configs on the POP1.
Everything is working perfectly now, with no timeout in accessing the host 192.168.110.3 and no ping problems neither!
On POP2:
no ip route vrf mgmt 0.0.0.0 0.0.0.0 192.168.102.200
no ip route vrf mgmt 0.0.0.0 0.0.0.0 192.168.102.201
on POP1:
router bgp 12345
address-family ipv4 vrf mgmt
network 0.0.0.0
Conclusion:
Anybody has idea about the problem I've experienced with the connectivity that was randomly giving problem in accessing and pinging the switch on the mgmt subnet (192.18.110.4)?
I tried to figure out the reason why, and I was wondering something about the double default static routes, and then I though instead something about the next-hop that wasn't directly connected and then a recursive lookup was needed but that should be fair enough...
I don't really know, even if I know the better and correct config should be to share the default via iBGP, I wanted to clarify the reason why I was experiencing too many weird problems in this scenario.
Many thanks,
Ciao
L.
