L2VPN over MC-LAG using ASR9k as POA and ASR903 as DHD

miqbal · ‎07-24-2016

I have two ASR9k, two ASR901 and one ASR903 routers configured as below with MC-LAG. The attached ppt shows the connectivity. The purpose is to carry VLAN 91 from CE1 to CE2 utilizing the MC-LAG. I am unable to ping across. The relevant configs are shown below. The MC-LAG appears to be working. I need some help to make it work.

Thank you very much

Regards

Javed

=================================================================================================================

ASR9k-1 (POA-1)
----------------
interface GigabitEthernet0/1/0/10
description Connected to POA-2 ASR9k-4
cdp
mtu 9100
ipv4 address 10.5.1.1/24
proxy-arp
!
interface GigabitEthernet0/1/0/18
description Connected to DHD-1 ASR903-1
bundle id 1 mode active
cdp
lacp period short
!
interface Loopback99
ipv4 address 10.5.99.1 255.255.255.255
!
interface Bundle-Ether1
mtu 9100
ipv4 address 1.1.1.1/24
lacp switchover suppress-flaps 300
mlacp iccp-group 99
mac-address 0.deaf.0
bundle wait-while 100
!
router ospf 1
area 0
interface Loopback0
!
interface Loopback99
!
interface GigabitEthernet0/1/0/10
!
!
!
mpls ldp
log
neighbor
!
router-id 10.5.99.1
address-family ipv4
discovery targeted-hello accept
!
interface GigabitEthernet0/1/0/10
!
!
interface Bundle-Ether1.91
encapsulation dot1q 91
!
interface GigabitEthernet0/1/0/16
description Connected to ASR901-1
cdp
speed 1000
!
l2vpn
nsr
pw-class MPLS
encapsulation mpls
!
!
bridge group iBasis
bridge-domain iBasis
interface Bundle-Ether1.91
!
interface GigabitEthernet0/1/0/16.91
!
vfi iBasis
neighbor 10.5.99.2 pw-id 91
pw-class MPLS
!
!
!
!
!
==========================================================================================
ASR9k-4 (POA-2)
------------------

interface Loopback99
ipv4 address 10.5.99.2 255.255.255.255
!
interface GigabitEthernet0/2/0/6
description Connected to POA-1 ASR9k-1
cdp
mtu 9100
ipv4 address 10.5.1.2 255.255.255.0
proxy-arp
!
interface GigabitEthernet0/2/0/29
description Connected to DHD-1 ASR903-1
bundle id 1 mode active
cdp
lacp period short
!

interface Bundle-Ether1
description Connected to DHD-1 ASR903-1
mtu 9100
lacp switchover suppress-flaps 300
mlacp iccp-group 99
mac-address 0.deaf.0
bundle wait-while 100
!
router ospf 1
log adjacency changes
area 0
interface Loopback0
passive enable
!
interface Loopback99
!
interface GigabitEthernet0/2/0/6
!
!
!
mpls ldp
router-id 10.5.99.2
discovery targeted-hello accept
log
neighbor
adjacency
!
interface GigabitEthernet0/2/0/6
!
!

interface Bundle-Ether1.91 l2transport
encapsulation dot1q 91
!
l2vpn
nsr
bridge group iBasis
bridge-domain iBasis
interface Bundle-Ether1.91
!
interface GigabitEthernet0/2/0/30.91
!
vfi iBasis
neighbor 10.5.99.1 pw-id 91
pw-class MPLS
!
!
!
!
================================================================================

ASR903 (DHD)
-------------

interface GigabitEthernet0/1/4
description Connected to POA-1 ASR9k-1
mtu 9100
no ip address
negotiation auto
cdp enable
channel-group 1 mode active
!
interface GigabitEthernet0/1/3
description to POA-1 ASR9k-4
mtu 9100
no ip address
negotiation auto
cdp enable
channel-group 1 mode active
!
interface Port-channel1
description To ASR9k-1 and ASR9k-4, Gig0/1/4 and G0/1/3
mtu 9100
no ip address
no negotiation auto
lacp fast-switchover
lacp max-bundle 1
service instance 91 ethernet
encapsulation dot1q 91
bridge-domain 91
!
!
interface GigabitEthernet0/0/6
description Connected to ASR901-2 GigabitEthernet0/6
no ip address
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
bridge-domain 91
!
!
==================================================================================

ASR901-01 (CE1)
---------------

interface GigabitEthernet0/3
description to ASR9k-01 GigabitEthernet0/1/0/16
no ip address
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
rewrite ingress tag pop 1 symmetric
bridge-domain 91
!
!
interface GigabitEthernet0/4
description Connected to ASR9k-4 Gig0/2/0/30
no ip address
media-type auto-select
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
rewrite ingress tag pop 1 symmetric
bridge-domain 91
!
!
interface Vlan91
ip address 10.91.91.1 255.255.255.252
!
=====================================================================================

ASR901-02 (CE2)
---------------

interface GigabitEthernet0/6
description Connected to ASR903-1 GigabitEthernet0/0/6
no ip address
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
rewrite ingress tag pop 1 symmetric
bridge-domain 91
!
!
interface Vlan91
ip address 10.91.91.2 255.255.255.252
!

Aleksandar Vidakovic · ‎07-26-2016

hi Javed,

your config is incomplete, you didn't include the config of the l2transport GigE subinterface.

If you only have two attachment circuits in a bridge domain, you would be better off using an xconnect instead.

For any further troubleshooting use the output of this command:

sh l2vpn xconnect group <group> xc-name <name> detail

This command will help you understand what's going on.

If you prefer to stick to bridge-domain (in case you plan on adding more ACs), the equivalent command is:

sh l2vpn bridge-domain group <group> bd-name <name> detail

hope this helps,

/Aleksandar

miqbal · ‎07-26-2016

Hi Aleksander

Thank you very much for your reply. For now the solution is in POC stage as an alternate to clusters. Once I get this working in Cisco AS lab, customer will also test in their lab and deploy with many other bridge-domains as needed. The sub-interface was configured but missing in the configs uploaded (good catch). Updated config is at the below after the requested show command from the two ASR9k's:

ASR9k-1:

RP/0/RSP0/CPU0:ASR9006-01#sh l2vpn bridge group iBasis bd-name iBasis detail
Tue Jul 26 08:38:36.741 PDT
Legend: pp = Partially Programmed.
Bridge group: iBasis, bridge-domain: iBasis, id: 0, state: up, ShgId: 0, MSTi: 0
Coupled state: disabled
MAC learning: enabled
MAC withdraw: enabled
    MAC withdraw for Access PW: enabled
    MAC withdraw sent on: bridge port up
    MAC withdraw relaying (access to access): disabled
Flooding:
    Broadcast & Multicast: enabled
    Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: disabled, Logging: disabled
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 snooping: disabled
IGMP Snooping: enabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: disabled
Bridge MTU: 1500
MIB cvplsConfigIndex: 1
Filter MAC addresses:
P2MP PW: disabled
Create time: 19/07/2016 14:42:05 (6d17h ago)
No status change since creation
ACs: 2 (1 up), VFIs: 1, PWs: 1 (1 up), PBBs: 0 (0 up)
List of ACs:
    AC: Bundle-Ether1.91, state is unresolved
      MAC learning: enabled
      Flooding:
        Broadcast & Multicast: enabled
        Unknown unicast: enabled
      MAC aging time: 300 s, Type: inactivity
      MAC limit: 4000, Action: none, Notification: syslog
      MAC limit reached: no
      MAC port down flush: enabled
      MAC Secure: disabled, Logging: disabled
      Split Horizon Group: none
      Dynamic ARP Inspection: disabled, Logging: disabled
      IP Source Guard: disabled, Logging: disabled
      DHCPv4 snooping: disabled
      IGMP Snooping: enabled
      IGMP Snooping profile: none
      MLD Snooping profile: none
      Storm Control: bridge-domain policer
      Static MAC addresses:
    AC: GigabitEthernet0/1/0/16.91, state is up
      Type VLAN; Num Ranges: 1
      VLAN ranges: [91, 91]
      MTU 1504; XC ID 0x1880002; interworking none
      MAC learning: enabled
      Flooding:
        Broadcast & Multicast: enabled
        Unknown unicast: enabled
      MAC aging time: 300 s, Type: inactivity
      MAC limit: 4000, Action: none, Notification: syslog
      MAC limit reached: no
      MAC port down flush: enabled
      MAC Secure: disabled, Logging: disabled
      Split Horizon Group: none
      Dynamic ARP Inspection: disabled, Logging: disabled
      IP Source Guard: disabled, Logging: disabled
      DHCPv4 snooping: disabled
      IGMP Snooping: enabled
      IGMP Snooping profile: none
      MLD Snooping profile: none
      Storm Control: bridge-domain policer
      Static MAC addresses:
      Statistics:
        packets: received 42540 (multicast 0, broadcast 42540, unknown unicast 0, unicast 0), sent 18875
        bytes: received 2722560 (multicast 0, broadcast 2722560, unknown unicast 0, unicast 0), sent 1208000
        MAC move: 18672
      Storm control drop counters:
        packets: broadcast 0, multicast 0, unknown unicast 0
        bytes: broadcast 0, multicast 0, unknown unicast 0
      Dynamic ARP inspection drop counters:
        packets: 0, bytes: 0
      IP source guard drop counters:
        packets: 0, bytes: 0
List of Access PWs:
List of VFIs:
    VFI iBasis (up)
      PW: neighbor 10.5.99.2, PW ID 91, state is up ( established )
        PW class MPLS, XC ID 0xc0000004
        Encapsulation MPLS, protocol LDP
        Source address 10.5.99.1
        PW type Ethernet, control word disabled, interworking none
        Sequencing not set

        PW Status TLV in use
          MPLS         Local                          Remote
          ------------ ------------------------------ -------------------------
          Label        32770                          16028
          Group ID     0x0                            0x21
          Interface    iBasis                         iBasis
          MTU          1500                           1500
          Control word disabled                       disabled
          PW type      Ethernet                       Ethernet
          VCCV CV type 0x2                            0x2
                       (LSP ping verification)        (LSP ping verification)
          VCCV CC type 0x6                            0x6
                       (router alert label)           (router alert label)
                       (TTL expiry)                   (TTL expiry)
          ------------ ------------------------------ -------------------------
        Incoming Status (PW Status TLV):
          Status code: 0x0 (Up) in Notification message
        MIB cpwVcIndex: 3221225476
        Create time: 19/07/2016 14:56:40 (6d17h ago)
        Last time status changed: 19/07/2016 15:10:38 (6d17h ago)
        MAC withdraw messages: sent 8, received 10
        Static MAC addresses:
        Statistics:
          packets: received 18875 (unicast 0), sent 42368
          bytes: received 1208000 (unicast 0), sent 2710904
          MAC move: 18677
        Storm control drop counters:
          packets: broadcast 0, multicast 0, unknown unicast 0
          bytes: broadcast 0, multicast 0, unknown unicast 0
      DHCPv4 snooping: disabled
      IGMP Snooping profile: none
      MLD Snooping profile: none
      VFI Statistics:
        drops: illegal VLAN 0, illegal length 0
RP/0/RSP0/CPU0:ASR9006-01#

ASR9k-4:

RP/0/RSP1/CPU0:USSP-ASR9006-04#sh l2vpn bridge group iBasis bd-name iBasis detail
Tue Jul 26 15:44:43.205 UTC
Legend: pp = Partially Programmed.
Bridge group: iBasis, bridge-domain: iBasis, id: 33, state: up, ShgId: 0, MSTi: 0
Coupled state: disabled
MAC learning: enabled
MAC withdraw: enabled
MAC withdraw for Access PW: enabled
MAC withdraw sent on: bridge port up
MAC withdraw relaying (access to access): disabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: disabled, Logging: disabled
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: disabled
Bridge MTU: 1500
MIB cvplsConfigIndex: 34
Filter MAC addresses:
Create time: 19/07/2016 21:53:25 (6d17h ago)
No status change since creation
ACs: 2 (2 up), VFIs: 1, PWs: 1 (1 up), PBBs: 0 (0 up)
List of ACs:
AC: Bundle-Ether1.91, state is up, active in RG-ID 99
Type VLAN; Num Ranges: 1
VLAN ranges: [91, 91]
MTU 9090; XC ID 0xa0000001; interworking none
MAC learning: enabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: disabled, Logging: disabled
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: disabled
Static MAC addresses:
Statistics:
packets: received 0, sent 60961
bytes: received 0, sent 3901504
Storm control drop counters:
packets: broadcast 0, multicast 0, unknown unicast 0
bytes: broadcast 0, multicast 0, unknown unicast 0
Dynamic ARP inspection drop counters:
packets: 0, bytes: 0
IP source guard drop counters:
packets: 0, bytes: 0
AC: GigabitEthernet0/2/0/30.91, state is up
Type VLAN; Num Ranges: 1
VLAN ranges: [91, 91]
MTU 1504; XC ID 0x2080001; interworking none
MAC learning: enabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: disabled, Logging: disabled
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: disabled
Static MAC addresses:
Statistics:
packets: received 18875, sent 18677
bytes: received 1208000, sent 1195328
Storm control drop counters:
packets: broadcast 0, multicast 0, unknown unicast 0
bytes: broadcast 0, multicast 0, unknown unicast 0
Dynamic ARP inspection drop counters:
packets: 0, bytes: 0
IP source guard drop counters:
packets: 0, bytes: 0
List of Access PWs:
List of VFIs:
VFI iBasis (up)
PW: neighbor 10.5.99.1, PW ID 91, state is up ( established )
PW class MPLS, XC ID 0xc0000021
Encapsulation MPLS, protocol LDP
Source address 10.5.99.2
PW type Ethernet, control word disabled, interworking none
Sequencing not set

PW Status TLV in use
MPLS Local Remote
------------ ------------------------------ -------------------------
Label 16028 32770
Group ID 0x21 0x0
Interface iBasis iBasis
MTU 1500 1500
Control word disabled disabled
PW type Ethernet Ethernet
VCCV CV type 0x2 0x2
(LSP ping verification) (LSP ping verification)
VCCV CC type 0x6 0x6
(router alert label) (router alert label)
(TTL expiry) (TTL expiry)
------------ ------------------------------ -------------------------
Incoming Status (PW Status TLV):
Status code: 0x0 (Up) in Notification message
MIB cpwVcIndex: 3221225505
Create time: 19/07/2016 22:03:45 (6d17h ago)
Last time status changed: 19/07/2016 22:13:12 (6d17h ago)
MAC withdraw messages: sent 10, received 8
Static MAC addresses:
Statistics:
packets: received 42368, sent 18875
bytes: received 2710904, sent 1208000
DHCPv4 snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
VFI Statistics:
drops: illegal VLAN 0, illegal length 0
RP/0/RSP1/CPU0:USSP-ASR9006-04#

RELEVANT CONFIGURATION

ASR9k-1 (POA-1)
-----------------------
interface GigabitEthernet0/1/0/10
description Connected to POA-2 ASR9k-4
cdp
mtu 9100
ipv4 address 10.5.1.1/24
proxy-arp
!
interface GigabitEthernet0/1/0/18
description Connected to DHD-1 ASR903-1
bundle id 1 mode active
cdp
lacp period short
!
interface Loopback99
ipv4 address 10.5.99.1 255.255.255.255
!
interface Bundle-Ether1
mtu 9100
ipv4 address 1.1.1.1/24
lacp switchover suppress-flaps 300
mlacp iccp-group 99
mac-address 0.deaf.0
bundle wait-while 100
!
router ospf 1
area 0
interface Loopback0
!
interface Loopback99
!
interface GigabitEthernet0/1/0/10
!
!
!
mpls ldp
log
neighbor
!
router-id 10.5.99.1
address-family ipv4
discovery targeted-hello accept
!
interface GigabitEthernet0/1/0/10
!
!
interface Bundle-Ether1.91
encapsulation dot1q 91
!
interface GigabitEthernet0/1/0/16
description Connected to ASR901-1
cdp
speed 1000
!
interface GigabitEthernet0/1/0/16.91 l2transport
encapsulation dot1q 91
!
l2vpn
nsr
pw-class MPLS
encapsulation mpls
!
!
bridge group iBasis
bridge-domain iBasis
   interface Bundle-Ether1.91
   !
   interface GigabitEthernet0/1/0/16.91
   !
   vfi iBasis
    neighbor 10.5.99.2 pw-id 91
     pw-class MPLS
    !
   !
!
!
!
==========================================================================================
ASR9k-4   (POA-2)
------------------------

interface Loopback99
ipv4 address 10.5.99.2 255.255.255.255
!
interface GigabitEthernet0/2/0/6
description Connected to POA-1 ASR9k-1
cdp
mtu 9100
ipv4 address 10.5.1.2 255.255.255.0
proxy-arp
!
interface GigabitEthernet0/2/0/29
description Connected to DHD-1 ASR903-1
bundle id 1 mode active
cdp
lacp period short
!

interface Bundle-Ether1
description Connected to DHD-1 ASR903-1
mtu 9100
lacp switchover suppress-flaps 300
mlacp iccp-group 99
mac-address 0.deaf.0
bundle wait-while 100
!
router ospf 1
log adjacency changes
area 0
interface Loopback0
   passive enable
!
interface Loopback99
!
interface GigabitEthernet0/2/0/6
!
!
!
mpls ldp
router-id 10.5.99.2
discovery targeted-hello accept
log
neighbor
adjacency
!
interface GigabitEthernet0/2/0/6
!
!
interface Bundle-Ether1.91 l2transport
encapsulation dot1q 91
!
interface GigabitEthernet0/2/0/30
cdp
!
interface GigabitEthernet0/2/0/30.91 l2transport
encapsulation dot1q 91
!
l2vpn
nsr
bridge group iBasis
bridge-domain iBasis
   interface Bundle-Ether1.91
   !
   interface GigabitEthernet0/2/0/30.91
   !
   vfi iBasis
    neighbor 10.5.99.1 pw-id 91
     pw-class MPLS
    !
   !
!
!
================================================================================

ASR903 (DHD)
--------------------

interface GigabitEthernet0/1/4
description Connected to POA-1 ASR9k-1
mtu 9100
no ip address
negotiation auto
cdp enable
channel-group 1 mode active
!
interface GigabitEthernet0/1/3
description to POA-1 ASR9k-4
mtu 9100
no ip address
negotiation auto
cdp enable
channel-group 1 mode active
!
interface Port-channel1
description To ASR9k-1 and ASR9k-4, Gig0/1/4 and G0/1/3
mtu 9100
no ip address
no negotiation auto
lacp fast-switchover
lacp max-bundle 1
service instance 91 ethernet
encapsulation dot1q 91
bridge-domain 91
!
!
interface GigabitEthernet0/0/6
description Connected to ASR901-2 GigabitEthernet0/6
no ip address
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
bridge-domain 91
!
!
==================================================================================

ASR901-01 (CE1)
-----------------------

interface GigabitEthernet0/3
description to ASR9k-01 GigabitEthernet0/1/0/16
no ip address
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
rewrite ingress tag pop 1 symmetric
bridge-domain 91
!
!
interface GigabitEthernet0/4
description Connected to ASR9k-4 Gig0/2/0/30
no ip address
media-type auto-select
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
rewrite ingress tag pop 1 symmetric
bridge-domain 91
!
!
interface Vlan91
ip address 10.91.91.1 255.255.255.252
!
=====================================================================================

ASR901-02 (CE2)
-----------------------

interface GigabitEthernet0/6
description Connected to ASR903-1 GigabitEthernet0/0/6
no ip address
negotiation auto
cdp enable
service instance 91 ethernet
encapsulation dot1q 91
rewrite ingress tag pop 1 symmetric
bridge-domain 91
!
!
interface Vlan91
ip address 10.91.91.2 255.255.255.252
!

Aleksandar Vidakovic · ‎07-29-2016

hi Javed,

the bundle AC is unresolved on ASR9k-1:

RP/0/RSP0/CPU0:ASR9006-01#sh l2vpn bridge group iBasis bd-name iBasis detail
<...>
  List of ACs:
    AC: Bundle-Ether1.91, state is unresolved

You have to configure all ACs as l2transport (sub)interfaces:

interface Bundle-Ether1.91 l2transport
 encapsulation dot1q 91

Try this out and let me know.

/Aleksandar

miqbal · ‎07-29-2016

Thank you sir, fixed the typo on ASR9006-01. But I still cannot ping across.

RP/0/RSP1/CPU0:ASR9006-01#sh run int bundle-ether 1.91
Fri Jul 29 09:46:33.496 PDT
interface Bundle-Ether1.91 l2transport
encapsulation dot1q 91
!

RP/0/RSP1/CPU0:ASR9006-01#sh l2vpn bridge group iBasis bd-name iBasis detail
Fri Jul 29 09:38:31.564 PDT
Legend: pp = Partially Programmed.
Bridge group: iBasis, bridge-domain: iBasis, id: 0, state: up, ShgId: 0, MSTi: 0
Coupled state: disabled
MAC learning: enabled
MAC withdraw: enabled
MAC withdraw for Access PW: enabled
MAC withdraw sent on: bridge port up
MAC withdraw relaying (access to access): disabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: disabled, Logging: disabled
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 snooping: disabled
IGMP Snooping: enabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: disabled
Bridge MTU: 1500
MIB cvplsConfigIndex: 1
Filter MAC addresses:
P2MP PW: disabled
Create time: 27/07/2016 16:08:58 (1d17h ago)
No status change since creation
ACs: 2 (2 up), VFIs: 1, PWs: 1 (1 up), PBBs: 0 (0 up)
List of ACs:
AC: Bundle-Ether1.91, state is up, standby in RG-ID 99
Type VLAN; Num Ranges: 1
VLAN ranges: [91, 91]
MTU 9090; XC ID 0xa0000005; interworking none
MAC learning: enabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: disabled, Logging: disabled
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 snooping: disabled
IGMP Snooping: enabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: bridge-domain policer
Static MAC addresses:
Statistics:
packets: received 0 (multicast 0, broadcast 0, unknown unicast 0, unicast 0), sent 0
bytes: received 0 (multicast 0, broadcast 0, unknown unicast 0, unicast 0), sent 0
MAC move: 0
Storm control drop counters:
packets: broadcast 0, multicast 0, unknown unicast 0
bytes: broadcast 0, multicast 0, unknown unicast 0
Dynamic ARP inspection drop counters:
packets: 0, bytes: 0
IP source guard drop counters:
packets: 0, bytes: 0
AC: GigabitEthernet0/1/0/16.91, state is up
Type VLAN; Num Ranges: 1
VLAN ranges: [91, 91]
MTU 1504; XC ID 0x1880002; interworking none
MAC learning: enabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: disabled, Logging: disabled
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 snooping: disabled
IGMP Snooping: enabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: bridge-domain policer
Static MAC addresses:
Statistics:
packets: received 68717 (multicast 0, broadcast 68717, unknown unicast 0, unicast 0), sent 18944
bytes: received 4397888 (multicast 0, broadcast 4397888, unknown unicast 0, unicast 0), sent 1212416
MAC move: 18740
Storm control drop counters:
packets: broadcast 0, multicast 0, unknown unicast 0
bytes: broadcast 0, multicast 0, unknown unicast 0
Dynamic ARP inspection drop counters:
packets: 0, bytes: 0
IP source guard drop counters:
packets: 0, bytes: 0
List of Access PWs:
List of VFIs:
VFI iBasis (up)
PW: neighbor 10.5.99.2, PW ID 91, state is up ( established )
PW class MPLS, XC ID 0xc0000004
Encapsulation MPLS, protocol LDP
Source address 10.5.99.1
PW type Ethernet, control word disabled, interworking none
Sequencing not set

PW Status TLV in use
MPLS Local Remote
------------ ------------------------------ -------------------------
Label 32770 16028
Group ID 0x0 0x21
Interface iBasis iBasis
MTU 1500 1500
Control word disabled disabled
PW type Ethernet Ethernet
VCCV CV type 0x2 0x2
(LSP ping verification) (LSP ping verification)
VCCV CC type 0x6 0x6
(router alert label) (router alert label)
(TTL expiry) (TTL expiry)
------------ ------------------------------ -------------------------
Incoming Status (PW Status TLV):
Status code: 0x0 (Up) in Notification message
MIB cpwVcIndex: 3221225476
Create time: 27/07/2016 16:08:58 (1d17h ago)
Last time status changed: 28/07/2016 06:31:24 (1d03h ago)
Last time PW went down: 28/07/2016 06:31:16 (1d03h ago)
MAC withdraw messages: sent 0, received 0
Static MAC addresses:
Statistics:
packets: received 18944 (unicast 0), sent 68544
bytes: received 1212416 (unicast 0), sent 4386168
MAC move: 18745
Storm control drop counters:
packets: broadcast 0, multicast 0, unknown unicast 0
bytes: broadcast 0, multicast 0, unknown unicast 0
DHCPv4 snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
VFI Statistics:
drops: illegal VLAN 0, illegal length 0
RP/0/RSP1/CPU0:ASR9006-01#

USSP-ASR901-01#sh int vlan91
Vlan91 is up, line protocol is up
Hardware is Ethernet SVI, address is 4403.a70c.8ee4 (bia 4403.a70c.8ee4)
Internet address is 10.91.91.1/30
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:06:11, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
0 packets input, 0 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
69178 packets output, 4151220 bytes, 0 underruns
0 output errors, 1 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
USSP-ASR901-01#ping 10.91.91.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.91.91.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
USSP-ASR901-01#ping 10.91.91.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.91.91.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
USSP-ASR901-01#

Aleksandar Vidakovic · ‎07-29-2016

The next thing you have to fix is the L2VPN configuration. You don't need at all the section highlighted in red:

l2vpn
 !
 bridge group iBasis
  bridge-domain iBasis
   interface Bundle-Ether1.91
   !
   interface GigabitEthernet0/1/0/16.91
   !
   vfi iBasis
    neighbor 10.5.99.2 pw-id 91
     pw-class MPLS
    !
   !

10.5.99.2 belongs to the Loopback on the other asr9k. Between the two asr9k you only need the ICCP.

If you only have two attachment circuits in the bridge domain iBasis, you can switch to xconnect.

Once you fix the L2VPN configuration, if pings still don't flow, check the MAC address table on the active asr9k.

miqbal · ‎07-29-2016

Removed the experimental vfi that was placed while trying something else. Still no change.

Aleksandar Vidakovic · ‎08-02-2016

you should also configure "rewrite ingress tag pop 1 symmetric" on both l2transport sub-interfaces.

If pings still don't flow, check again the "sh l2vpn bridge-domain group <group> bd-name <name> detail" and the MAC address table on the active asr9k.

miqbal · ‎08-02-2016

Sir,

I added rewrite ingress tag pop 1 symmetric in the following order and checked after each change. No change. Ping still don't flow.

1- ASR9k-01 under GigabitEthernet0/1/0/16.91 l2transport and Bundle-Ether1.91 l2transport

Check, no change

2- Also on ASR9k-4, add under GigabitEthernet0/2/0/30.91 l2transport and Bundle-Ether1.91 l2transport

Check, no change

3- Also on ASR903-01 add under Service Instance 91 Ethernet under Port Channel1 and GigabitEthernet0/0/6

Check, no change

4- ASR901-01 and ASR901-02 already have the rewrite commands under the service instance.

Checked bridge mac tables on asr9k, the mac for the bundle-Ether nor the Bundle-Ether1.91 does not appear. Rest are fine.

Regards

Javed

Aleksandar Vidakovic · ‎08-02-2016

hi Javed,

the MAC addresses that I was asking for are the MAC addresses of the CE1 and CE2. Only these should appear in the MAC address table of your bridge domain.

Can you open a TAC SR for this? I'm sure this can be resolved in one webex session. We have to confirm the status of bridge domains on each asr90x and asr9k and check which of these devices has leared which MAC address. I don't expect this to be an issue with MAC learning because ping should work even if all unicast frames are flooded, but the MAC table will confirm which point a frames from each CE reaching.

miqbal · ‎08-04-2016

Hi Aleksander,

I opened TAC SR 639217209. Let's see where it goes.

Regards

Javed