cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
565
Views
10
Helpful
4
Replies
Highlighted
Beginner

VXLAN with selective underlay links VNI manipulation

Hello!!!

 

We are trying to implement an scenario to provide L2 transport services from POP A to POP B, connected with three different carriers transport links. In the future, we may want to implement POP C and have the ability to provide L2 services between any of the POPs.

The goals we are trying to achieve are:

  1. Scalability. More POPs will be added
  2. Sub second convergence
  3. Traffic engineering capabilities in order to establish conditional business logic based on the state of the transport links.

We got 4x9216 Nexus Boxes and We have tried two VXLAN approachs

  1. based on BGP underlay using route-maps to establish local preference and to control what carrier link will transport each VNI, by manipulating the reachability of the peer VTEPS, using a loopback interface for each VLAN/VNI
  2. based on OSPF underlay and BPG L2VPN EVPN and vPC

The problem with the first approach configuration comes when defining the source interface for the nve interface, as it has to be only one even if we define different peer destinations per VNI. If we define loopback1 as source, only vlan 101 is carried. If we define loopback 2 as source, only vlan 102 is carried, and so on. Disregarding the fact that every peer VTEP is reachable when sourcing traffic from any of the loopbacks (1 to 3).

 

The problem with the second approach is every VLAN works, but we cant figure a way to manipulate the VNI so they are selectively transported over the different transport links.

 

Below configuration is for approach 1, POP A, red switch, but you can figure the other 3. Eht1/6 and Eth1/7 are 2 of the transport links. Blue switches have Eth1/5 for the third link. Eth1/53 is for POP switches interlink.

Is there any chance this configuration may work? Does anyone knows another approach to achieve above goals?

 

Approach 2 is a basic bgp l2vpn evpn approach. Is there a way to manipulate the VNIs so they are prioritizes/allowed/blocked in underlying physical links? I think it may be difficult (if not impossible) because the nve just uses a single source lookpback.

 

Any clue?

 

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature bfd
feature nv overlay

!
vlan 1,101-103
vlan 101
  vn-segment 10101

vlan 102
  vn-segment 10102

vlan 103
  vn-segment 10103
!
spanning-tree port type edge bpduguard default
spanning-tree vlan 101-103 priority 0
!
ip prefix-list REMOTE_TEP_101 seq 5 permit 31.31.31.31/32 
ip prefix-list REMOTE_TEP_101 seq 10 permit 41.41.41.31/32 
ip prefix-list REMOTE_TEP_102 seq 5 permit 32.32.32.32/32 
ip prefix-list REMOTE_TEP_102 seq 10 permit 42.42.42.42/32 
ip prefix-list REMOTE_TEP_103 seq 5 permit 33.33.33.33/32 
ip prefix-list REMOTE_TEP_103 seq 10 permit 43.43.43.43/32 

route-map FROM_LINK1 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 300

route-map FROM_LINK1 permit 20
  match ip address prefix-list REMOTE_TEP_102 
  set local-preference 200

route-map FROM_LINK1 permit 30
  match ip address prefix-list REMOTE_TEP_103 
  set local-preference 100

route-map FROM_LINK1 permit 100

route-map FROM_LINK2 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 100

route-map FROM_LINK2 permit 20
  match ip address prefix-list REMOTE_TEP_102 
  set local-preference 300

route-map FROM_LINK2 permit 30
  match ip address prefix-list REMOTE_TEP_103 
  set local-preference 200

route-map FROM_LINK2 permit 100

route-map FROM_LINK3 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 200

route-map FROM_LINK3 permit 30
  match ip address prefix-list REMOTE_TEP_102 REMOTE_TEP_103 
  set local-preference 300

route-map FROM_LINK3 permit 100
!

vrf context management
!
interface Vlan1
  no shutdown
!
interface Vlan101
  no shutdown
  ip address 91.91.91.1/24
  mtu 9216
!
interface Vlan102
  no shutdown
  ip address 92.92.92.1/24
  mtu 9216
!
interface Vlan103
  no shutdown
  ip address 93.93.93.1/24
  mtu 9216
!
interface nve1
  no shutdown
  source-interface loopback1
  member vni 10101
    ingress-replication protocol static
      peer-ip 31.31.31.31
  member vni 10102
    ingress-replication protocol static
      peer-ip 32.32.32.32
  member vni 10103
    ingress-replication protocol static
      peer-ip 33.33.33.33
!
interface Ethernet1/53
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 201.201.201.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/6
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 102.102.102.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/7
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 101.101.101.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/48
  switchport
  switchport access vlan 101
  spanning-tree port type edge
  mtu 9216
  no shutdown
!
interface loopback0
  description ### VXLAN - ROUTING PURPOSES ###
  ip address 1.1.1.1/32
!
interface loopback1
  description ### VXLAN - TEP-NVE - VLAN 101 ###
  ip address 11.11.11.11/32
  ip address 10.101.0.1/32 secondary
!
interface loopback2
  description ### VXLAN - TEP-NVE - VLAN 102 ###
  ip address 12.12.12.12/32
  ip address 10.102.0.1/32 secondary
!
interface loopback3
  description ### VXLAN - TEP-NVE - VLAN 103 ###
  ip address 13.13.13.13/32
  ip address 10.103.0.1/32 secondary
!
router bgp 100
  router-id 1.1.1.1
  timers bgp 5 15
  address-family ipv4 unicast
    network 11.11.11.11/32
    network 12.12.12.12/32
    network 13.13.13.13/32
    network 10.101.0.1/32
    network 10.102.0.1/32
    network 10.103.0.1/32
  neighbor 101.101.101.2
    remote-as 200
        bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      route-map FROM_LINK1 in
  neighbor 102.102.102.2
    remote-as 200
    bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      route-map FROM_LINK2 in
  neighbor 201.201.201.2
    remote-as 100
    bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      next-hop-self
!
end
!

Thanks in advance,

Miguel

4 REPLIES 4
Highlighted
Participant

Hi,

Looks like you ran into the main problem I have with VXLAN technology –that is the lack of TE capabilities. I have no idea why the whole Data-Centre industry got crazy about VXLAN –like MPLS L2VPNs don’t exist. Imagine you’d have MPLS within both DCs and in the transport network in between –you could then define end-to-end TE path even on per VLAN granularity if you’d like to, selecting appropriate exit and entry points from DC etc… and there would be no need for interworking on the DC-to-Core boundary.

 

Not sure what are the capabilities on nx9216 but  what you can do is to terminate each VNI into a separate L2VPN –and then configure MPLS-TE so that each L2VPN uses its own TE Tunnel, since you’d be in control of the TE tunnel path you’d be essentially in control of the path the VXLAN traffic takes across the underlay –however such solution assumes MPLS capable underlay.  

   

 

adam

 

netconsultings.com

::carrier-class solutions for the telecommunications industry::

adam
Highlighted

Hi Adam,


Thanks for your analysis. It is important for us to know the limits were already there.


Yes, you are right, having a MPLS TE enable underlay would solve the problem. Unfortunately, 9216 doesnt have those capabilities and our options are to include ASR like devices in the POP.


Kind regards,


Miguel
Highlighted

Check if you platform supports Segment Routing Traffic Engineering (SRTE) and test with SRTE in order to engineer traffic through the core.

Elvin
Highlighted
Beginner


@mberniz wrote:

Hello!!!

 

We are trying to implement an scenario to provide L2 transport services from POP A to POP B, connected with three different carriers transport links. In the future, we may want to implement POP C and have the ability to provide L2 services between any of the POPs.

The goals we are trying to achieve are:

  1. Scalability. More POPs will be added
  2. Sub second convergence
  3. Traffic engineering capabilities in order to establish conditional business logic based on the state of the transport links. iTunes Mobdro TutuApp

We got 4x9216 Nexus Boxes and We have tried two VXLAN approachs

  1. based on BGP underlay using route-maps to establish local preference and to control what carrier link will transport each VNI, by manipulating the reachability of the peer VTEPS, using a loopback interface for each VLAN/VNI
  2. based on OSPF underlay and BPG L2VPN EVPN and vPC

The problem with the first approach configuration comes when defining the source interface for the nve interface, as it has to be only one even if we define different peer destinations per VNI. If we define loopback1 as source, only vlan 101 is carried. If we define loopback 2 as source, only vlan 102 is carried, and so on. Disregarding the fact that every peer VTEP is reachable when sourcing traffic from any of the loopbacks (1 to 3).

 

The problem with the second approach is every VLAN works, but we cant figure a way to manipulate the VNI so they are selectively transported over the different transport links.

 

Below configuration is for approach 1, POP A, red switch, but you can figure the other 3. Eht1/6 and Eth1/7 are 2 of the transport links. Blue switches have Eth1/5 for the third link. Eth1/53 is for POP switches interlink.

Is there any chance this configuration may work? Does anyone knows another approach to achieve above goals?

 

Approach 2 is a basic bgp l2vpn evpn approach. Is there a way to manipulate the VNIs so they are prioritizes/allowed/blocked in underlying physical links? I think it may be difficult (if not impossible) because the nve just uses a single source lookpback.

 

Any clue?

 

nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature bfd
feature nv overlay

!
vlan 1,101-103
vlan 101
  vn-segment 10101

vlan 102
  vn-segment 10102

vlan 103
  vn-segment 10103
!
spanning-tree port type edge bpduguard default
spanning-tree vlan 101-103 priority 0
!
ip prefix-list REMOTE_TEP_101 seq 5 permit 31.31.31.31/32 
ip prefix-list REMOTE_TEP_101 seq 10 permit 41.41.41.31/32 
ip prefix-list REMOTE_TEP_102 seq 5 permit 32.32.32.32/32 
ip prefix-list REMOTE_TEP_102 seq 10 permit 42.42.42.42/32 
ip prefix-list REMOTE_TEP_103 seq 5 permit 33.33.33.33/32 
ip prefix-list REMOTE_TEP_103 seq 10 permit 43.43.43.43/32 

route-map FROM_LINK1 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 300

route-map FROM_LINK1 permit 20
  match ip address prefix-list REMOTE_TEP_102 
  set local-preference 200

route-map FROM_LINK1 permit 30
  match ip address prefix-list REMOTE_TEP_103 
  set local-preference 100

route-map FROM_LINK1 permit 100

route-map FROM_LINK2 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 100

route-map FROM_LINK2 permit 20
  match ip address prefix-list REMOTE_TEP_102 
  set local-preference 300

route-map FROM_LINK2 permit 30
  match ip address prefix-list REMOTE_TEP_103 
  set local-preference 200

route-map FROM_LINK2 permit 100

route-map FROM_LINK3 permit 10
  match ip address prefix-list REMOTE_TEP_101 
  set local-preference 200

route-map FROM_LINK3 permit 30
  match ip address prefix-list REMOTE_TEP_102 REMOTE_TEP_103 
  set local-preference 300

route-map FROM_LINK3 permit 100
!

vrf context management
!
interface Vlan1
  no shutdown
!
interface Vlan101
  no shutdown
  ip address 91.91.91.1/24
  mtu 9216
!
interface Vlan102
  no shutdown
  ip address 92.92.92.1/24
  mtu 9216
!
interface Vlan103
  no shutdown
  ip address 93.93.93.1/24
  mtu 9216
!
interface nve1
  no shutdown
  source-interface loopback1
  member vni 10101
    ingress-replication protocol static
      peer-ip 31.31.31.31
  member vni 10102
    ingress-replication protocol static
      peer-ip 32.32.32.32
  member vni 10103
    ingress-replication protocol static
      peer-ip 33.33.33.33
!
interface Ethernet1/53
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 201.201.201.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/6
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 102.102.102.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/7
  no switchport
  mtu 9216
  bfd interval 50 min_rx 50 multiplier 3
  bfd ipv4 interval 50 min_rx 50 multiplier 3
  no ip redirects
  ip address 101.101.101.1/24
  no ipv6 redirects
  no shutdown
!
interface Ethernet1/48
  switchport
  switchport access vlan 101
  spanning-tree port type edge
  mtu 9216
  no shutdown
!
interface loopback0
  description ### VXLAN - ROUTING PURPOSES ###
  ip address 1.1.1.1/32
!
interface loopback1
  description ### VXLAN - TEP-NVE - VLAN 101 ###
  ip address 11.11.11.11/32
  ip address 10.101.0.1/32 secondary
!
interface loopback2
  description ### VXLAN - TEP-NVE - VLAN 102 ###
  ip address 12.12.12.12/32
  ip address 10.102.0.1/32 secondary
!
interface loopback3
  description ### VXLAN - TEP-NVE - VLAN 103 ###
  ip address 13.13.13.13/32
  ip address 10.103.0.1/32 secondary
!
router bgp 100
  router-id 1.1.1.1
  timers bgp 5 15
  address-family ipv4 unicast
    network 11.11.11.11/32
    network 12.12.12.12/32
    network 13.13.13.13/32
    network 10.101.0.1/32
    network 10.102.0.1/32
    network 10.103.0.1/32
  neighbor 101.101.101.2
    remote-as 200
        bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      route-map FROM_LINK1 in
  neighbor 102.102.102.2
    remote-as 200
    bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      route-map FROM_LINK2 in
  neighbor 201.201.201.2
    remote-as 100
    bfd
    address-family ipv4 unicast
      send-community
      send-community extended
      next-hop-self
!
end
!

Thanks in advance,

Miguel


Not sure what are the capabilities on nx9216 but  what you can do is to terminate each VNI into a separate L2VPN –and then configure MPLS-TE so that each L2VPN uses its own TE Tunnel, since you’d be in control of the TE tunnel path you’d be essentially in control of the path the VXLAN traffic takes across the underlay –however such solution assumes MPLS capable underlay.