Solved: Re: ASR 9000 VXLAN support

vladdar1349 · ‎01-27-2020

Hello Cisco community,

I am trying to lab VXLAN on 3 routers - 2x ASR9910 and 1x ASR9006 with IOS-XR 5.3.3

All of them should support VXLAN encasulation and BUM replication over multicast. Unfortunately it works only on newer ASR9910. Those 2 routers does not see ASR9006 in nve peers and it looks like ASR9006 is not able to encapsulate and send traffic over VXLAN.

Is it possible that I am missing the licensing? I found out that there is a RTU: Part #: S-A9K-VXLN-RTU-10 ASR 9000 VxLAN 10Gbps Right to Use License

But it works on ASR9910 without any license..

Thank You.

Bridge group: vxlan-test, bridge-domain: vlan88, id: 2, state: up, ShgId: 0, MSTi: 0
  Coupled state: disabled
  MAC learning: enabled
  MAC withdraw: enabled
    MAC withdraw for Access PW: enabled
    MAC withdraw sent on: bridge port up
    MAC withdraw relaying (access to access): disabled
  Flooding:
    Broadcast & Multicast: enabled
    Unknown unicast: enabled
  MAC aging time: 300 s, Type: inactivity
  MAC limit: 4000, Action: none, Notification: syslog
  MAC limit reached: no
  MAC port down flush: enabled
  MAC Secure: disabled, Logging: disabled
  Split Horizon Group: none
  Dynamic ARP Inspection: disabled, Logging: disabled
  IP Source Guard: disabled, Logging: disabled
  DHCPv4 snooping: disabled
  IGMP Snooping: enabled
  IGMP Snooping profile: none
  MLD Snooping profile: none
  Storm Control: disabled
  Bridge MTU: 1500
  MIB cvplsConfigIndex: 3
  Filter MAC addresses:
  P2MP PW: disabled
  Create time: 24/01/2020 15:24:05 (2d19h ago)
  No status change since creation
  ACs: 1 (1 up), VFIs: 0, PWs: 0 (0 up), PBBs: 0 (0 up)
  List of ACs:
    AC: GigabitEthernet0/1/0/39.88, state is up
      Type VLAN; Num Ranges: 1
      VLAN ranges: [88, 88]
      MTU 9188; XC ID 0x1880001; interworking none
      MAC learning: enabled
      Flooding:
        Broadcast & Multicast: enabled
        Unknown unicast: enabled
      MAC aging time: 300 s, Type: inactivity
      MAC limit: 4000, Action: none, Notification: syslog
      MAC limit reached: no
      MAC port down flush: enabled
      MAC Secure: disabled, Logging: disabled
      Split Horizon Group: none
      Dynamic ARP Inspection: disabled, Logging: disabled
      IP Source Guard: disabled, Logging: disabled
      DHCPv4 snooping: disabled
      IGMP Snooping: enabled
      IGMP Snooping profile: none
      MLD Snooping profile: none
      Storm Control: bridge-domain policer
      Static MAC addresses:
      Statistics:
        packets: received 8156 (multicast 4, broadcast 8148, unknown unicast 0, unicast 0), sent 432
        bytes: received 1259728 (multicast 412, broadcast 1258866, unknown unicast 0, unicast 0), sent 57700
        MAC move: 0
      Storm control drop counters: 
        packets: broadcast 0, multicast 0, unknown unicast 0 
        bytes: broadcast 0, multicast 0, unknown unicast 0 
      Dynamic ARP inspection drop counters: 
        packets: 0, bytes: 0
      IP source guard drop counters: 
        packets: 0, bytes: 0
  List of VNIs:
    VNI 10088, state is up
      XC ID 0x80000003
      Encap type VXLAN,      Data Plane L2 Service
      Overlay nve1, Source X.X.X.X, Multicast Group 239.239.8.8, UDP Port 4789
      Anycast VTEP 0.0.0.0, Anycast Multicast Group 0.0.0.0
      MAC learning: enabled
      Flooding:
        Broadcast & Multicast: enabled
        Unknown unicast: enabled
      MAC aging time: 300 s, Type: inactivity
      MAC limit: 4000, Action: none, Notification: syslog
      MAC limit reached: no
      MAC port down flush: enabled
      MAC Secure: disabled, Logging: disabled
      Split Horizon Group: none
      Dynamic ARP Inspection: disabled, Logging: disabled
      IP Source Guard: disabled, Logging: disabled
      DHCPv4 snooping: disabled
      IGMP Snooping: enabled
      IGMP Snooping profile: none
      MLD Snooping profile: none
      Storm Control: bridge-domain policer
      Statistics:
        packets: received 432 (unicast 0), sent 0
        bytes: received 57700 (unicast 0), sent 0
        MAC move: 0
  List of Access PWs:
  List of VFIs:

smilstea · ‎01-29-2020

Can you be more specific when you say VXLAN EVPN is not working?

VXLAN Support
Hardware:
- Support on typhoon (-TR and -SE) for all features introduced in 5.2.0 to 6.2.1with few exceptions:

** Not supported on Typhoon:

* EVPN-VXLAN to VPLS L2 Gateway
* EVPN ESI Multi pathing for VXLAN - RFC compliance
* L2VPN ISSU for ASR9K
- No support on typhoon for features to be introduced in 6.3.1 and later.
- Supported on Tomahawk all line cards (-TR and -SE) (XR 5.3.1 or higher)

1/ Could you please confirm that VXLAN BGP EVPN (L2GW) is supported on typhoon
line cards?
Yes

2/ Could you please detail which VXLAN feature wont be supported on typhoon
cards?
ESI Multipath per-flow load balancing is not supported on Typhoon LC,
Typhoon LC only can do per-EVI ESI Multipath load balancing.

Sam

View solution in original post

smilstea · ‎01-27-2020

What is the show platform from all 3 routers?

Thanks,

Sam

vladdar1349 · ‎01-28-2020

Hi,

#sh platform
Tue Jan 28 10:30:02.409 utc
Node Type State Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0 A99-RSP-TR(Active) IOS XR RUN PWR,NSHUT,MON
0/RSP1/CPU0 A99-RSP-TR(Standby) IOS XR RUN PWR,NSHUT,MON
0/0/CPU0 A9K-MOD160-TR IOS XR RUN PWR,NSHUT,MON
0/0/0 A9K-MPA-8X10GE OK PWR,NSHUT,MON
0/0/1 A9K-MPA-2X40GE OK PWR,NSHUT,MON
0/3/CPU0 A9K-4T16GE-TR IOS XR RUN PWR,NSHUT,MON
0/7/CPU0 A9K-MOD160-TR IOS XR RUN PWR,NSHUT,MON
0/7/0 A9K-MPA-8X10GE OK PWR,NSHUT,MON
0/7/1 A9K-MPA-2X40GE OK PWR,NSHUT,MON

#sh platform
Tue Jan 28 10:30:43.377 utc
Node Type State Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0 A99-RSP-TR(Active) IOS XR RUN PWR,NSHUT,MON
0/RSP1/CPU0 A99-RSP-TR(Standby) IOS XR RUN PWR,NSHUT,MON
0/0/CPU0 A9K-MOD160-TR IOS XR RUN PWR,NSHUT,MON
0/0/0 A9K-MPA-8X10GE OK PWR,NSHUT,MON
0/0/1 A9K-MPA-2X40GE OK PWR,NSHUT,MON
0/1/CPU0 A9K-4T16GE-TR IOS XR RUN PWR,NSHUT,MON
0/3/CPU0 A9K-40GE-TR IOS XR RUN PWR,NSHUT,MON
0/4/CPU0 A9K-40GE-TR IOS XR RUN PWR,NSHUT,MON
0/5/CPU0 A9K-36x10GE-TR IOS XR RUN PWR,NSHUT,MON
0/6/CPU0 A9K-4T16GE-TR IOS XR RUN PWR,NSHUT,MON
0/7/CPU0 A9K-MOD160-TR IOS XR RUN PWR,NSHUT,MON
0/7/0 A9K-MPA-8X10GE OK PWR,NSHUT,MON
0/7/1 A9K-MPA-2X40GE OK PWR,NSHUT,MON

ASR9006

#sh platform
Tue Jan 28 10:30:59.411 utc
Node Type State Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0 A9K-RSP440-SE(Active) IOS XR RUN PWR,NSHUT,MON
0/RSP1/CPU0 A9K-RSP440-SE(Standby) IOS XR RUN PWR,NSHUT,MON
0/0/CPU0 A9K-40GE-B IOS XR RUN PWR,NSHUT,MON
0/1/CPU0 A9K-40GE-B IOS XR RUN PWR,NSHUT,MON
0/2/CPU0 A9K-16T/8-B IOS XR RUN PWR,NSHUT,MON
0/3/CPU0 A9K-24x10GE-TR IOS XR RUN PWR,NSHUT,MON

smilstea · ‎01-28-2020

On the 9006 can you make sure that you are using LC3 for VxLAN?

The other LCs are the older trident cards (-B) and don't support VxLAN due to limited uCode space.

In fact even the typhoon based LCs have limited VxLAN support, only the 4x100, 8x100, MOD200, and MOD400 (tomahawk LCs) have full VxLAN support.

Sam

vladdar1349 · ‎01-29-2020

Thank You, I will try to test it on the typhoon card.

What do you mean by limited support? We only need L2VPNs to run over vxlan using multicast (PIM-SM) for BUM replication.

Looks like VXLAN EVPN or ingress replication is not supported even on newer ASR9910 running 6.x code

smilstea · ‎01-29-2020

Can you be more specific when you say VXLAN EVPN is not working?

VXLAN Support
Hardware:
- Support on typhoon (-TR and -SE) for all features introduced in 5.2.0 to 6.2.1with few exceptions:

** Not supported on Typhoon:

* EVPN-VXLAN to VPLS L2 Gateway
* EVPN ESI Multi pathing for VXLAN - RFC compliance
* L2VPN ISSU for ASR9K
- No support on typhoon for features to be introduced in 6.3.1 and later.
- Supported on Tomahawk all line cards (-TR and -SE) (XR 5.3.1 or higher)

1/ Could you please confirm that VXLAN BGP EVPN (L2GW) is supported on typhoon
line cards?
Yes

2/ Could you please detail which VXLAN feature wont be supported on typhoon
cards?
ESI Multipath per-flow load balancing is not supported on Typhoon LC,
Typhoon LC only can do per-EVI ESI Multipath load balancing.

Sam

vladdar1349 · ‎01-29-2020

Sorry Sam, you are right VXLAN EVPN is supported.

RP/0/RSP0/CPU0:R1(config-if)#member vni 10008 host-reachabilty protocol bgp

so only ingress replication is not supported.

Thank You.

Elvin Arias · ‎01-28-2020

If you are planning to run VXLAN, I would say go to a 6.x+ version instead.

Elvin

smilstea · ‎01-28-2020

Of course a 6.x version is recommended, but the hardware on the router with the problem doesn't support 6.x. If they wish to continue using the old hardware and support VxLAN on typhoon then the code they are on is fine, they just won't get every feature of VxLAN, it all depends on what they need.

Sam