Re: Nexus BGP an multiple VRF, sessions establishment issues

Jean-Daniel · ‎09-15-2021

I have a Nexus 3K with NXOS 9.3(7) and having trouble configuring BGP on multiple VRFs.

I have a config like this on 2 switches:

vlan 1
vlan 10
  name nx<-external->nx
vlan 20
  name nx<-internal->nx

vrf context external
vrf context internal
vrf context management

interface Vlan1

interface Vlan10
  description nx<->nx (external)
  no shutdown
  no autostate
  vrf member external
  no ip redirects
  ip address 10.0.0.0/31

interface Vlan20
  description nx<->nx (internal)
  no shutdown
  no autostate
  vrf member internal
  no ip redirects
  ip address 10.0.0.2/31

interface Ethernet1/1
  description nx<->nx
  switchport access vlan 10
  no shutdown

interface Ethernet1/2
  description nx<->nx
  switchport access vlan 20
  no shutdown

router bgp 65100
  template peer DataCenter
    timers 3 9
    address-family ipv4 unicast
      advertisement-interval 1
      next-hop-self
      soft-reconfiguration inbound
  vrf external
    router-id 81.201.183.161
    timers bgp 3 9
    bestpath as-path multipath-relax
    log-neighbor-changes
    address-family ipv4 unicast
      maximum-paths 16
    neighbor 10.0.0.1 remote-as 65100
      inherit peer DataCenter
  vrf internal
    router-id 172.22.0.6
    timers bgp 3 9
    bestpath as-path multipath-relax
    log-neighbor-changes
    address-family ipv4 unicast
      maximum-paths 16
    neighbor 10.0.0.3 remote-as 65100
      inherit peer DataCenter

The session properly comes UP on VRF external, but it can't be established on VRF internal.

Both sides can ping the other side on both addresses.

sh ip bgp neighbors vrf internal 
BGP neighbor is 10.0.0.3, remote AS 65100, ibgp link, Peer index 3
  Inherits peer configuration from peer-template DataCenter
  BGP version 4, remote router ID 0.0.0.0
  Neighbor previous state = Idle
  BGP state = Idle, down for 00:32:38
  Neighbor vrf: internal, retry in 00:00:37
  Peer is directly attached, interface Vlan20
  Connections established 0, dropped 0
  Connection attempts 27

There is many attempts but they always fails.

When running ethanalyzer on both switches, the only traffic between 10.0.0.2 to 10.0.0.3 I see is repeated ARP requests and answers (batch of 5 requests, pause, batch of 5 requests, …). No BGP connection attempts.

Is there some known limitations when using BGP on multiple VRF ?

Giuseppe Larosa · ‎09-15-2021

Hello @Jean-Daniel ,

as a first attempt can you change the BGP ASN on peer 10.0.0.3 to a different value ?

Hope to help

Giuseppe

Peter Paluch · ‎09-21-2021

Hello @Jean-Daniel ,

I suspect that this problem is related to a basic IP reachability issue between 10.0.0.2 and 10.0.0.3. Can you even ping 10.0.0.3 from 10.0.0.2 in the VRF internal? You wrote yourself that in the ethanalyzer, you only see batches of ARP requests but you did not mention any replies. So it would seem that 10.0.0.2 cannot even resolve the MAC address of 10.0.0.3 using ARP. Of course, then the TCP connection for BGP cannot be established.

Please let us know.

Best regards,
Peter