How to make ASR9000 bridge domain forward traffic between sub interfaces of same physical interface?

[colin.franco]

Hi,

I regularly use bridge domains to connect sub interfaces on different vlans using this sort of configuration:

interface GigabitEthernet0/0/0/5.21 l2transport

description CUSTOMER A WAN

encapsulation dot1q 21

rewrite ingress tag pop 1 symmetric

interface GigabitEthernet0/0/0/10.3122 l2transport

description CUSTOMER A CORE

encapsulation dot1q 3122

rewrite ingress tag pop 1 symmetric

l2vpn

bridge group WANLINKS

  bridge-domain CUSTOMERA

   interface GigabitEthernet0/0/0/5.21

   interface GigabitEthernet0/0/0/10.3122

When I try to use the same method to bridge two sub interfaces on the same physical interface so as to create a L2 VPN no data flows:

interface GigabitEthernet0/0/0/5.21 l2transport

description CUSTOMER A WAN

encapsulation dot1q 21

rewrite ingress tag pop 1 symmetric

interface GigabitEthernet0/0/0/5.22 l2transport

description CUSTOMER A WAN2

encapsulation dot1q 22

rewrite ingress tag pop 1 symmetric

l2vpn

bridge group WANLINKS

  bridge-domain CUSTOMERA

   interface GigabitEthernet0/0/0/5.21

   interface GigabitEthernet0/0/0/5.22

If I add a BVI interface to the bridge domain then the CE devices at the remote end of the WAN interface can both ping the BVI IP but they remain unable to ping each other.

Is this because tag rewrites are not happening since packets don't leave the physical interface?

How can I work around this and establish a L2 connection between the two subinterfaces?

Thank you

[Christer Lind]

Hello,

You might be correct in that statement. I just tested it out with just single-tagged VLANs and it worked without a problem.

The way it's connected now is

End-nodes -> Cat 2970G -> Cat 3560 -> Cat 3750 -> ASR920* -> 4500X -> ASR9k

* = Double-tagging happens here;
service instance 4 ethernet
encapsulation dot1q 50,64
bridge-domain 400

Any ideas?

[Christer Lind]

Hello again, Alexander.

After further investigations it would appear that the root to the problem is that, the same mac-address "appears" in the same VLAN (Or well, in this case, outer VLAN) in multiplie places at once (First where it's actually connected but then also in the 9k where it's bridged).

Now, I don't want to be causing any loops in our network, so i'm coming here for some guidance how to overcome this dilemma, how do others handle this?

[xthuijs]

hi christer, the a9k in a bridged mode would maintain a bridge/mac table per bridge domain. If the mac would change from one EFP to another, it would constitute a mac move which is perfectly fine.

What I think is happening is that because of the way you are doing vlan imposition on the attached switch, that switch only has a mac table per outer vlan and therefore thinks there is a mac flip going on eventhough there are different inner vlans. It is like a 1 vs 2 dimensional table so to speak.

Since you only want to connect 2 circuits together few possibilities/suggestions are:

- use p2p on the a9k, there will be no MAC learning whatsoever, packet in will be packet out, simple switching like that.

- on the switch, see if you can create a service intsance that binds the 2 circuits directly there without the need for a backhaul to the a9k and associated p2p connection.

- use single stack, but unique vlan's for the 2 circuits, and haul the single tags to EFP's on the a9k to link them together with a p2p xcon.

cheers

xander

[Christer Lind]

Hello Alexander, thank you for your reply and assistance in the matter.

What you said is what I also concluded, the asr9k has nothing to do with it, it was obvious when checking the mac address table for the 4500X really, that's what you get for focusing your efforts on the wrong thing

In the described scenario I only tried to attach 2 circuits, later on in there's actually quite a few more which rules out any p2p solutions, unfortunately. Most of our use cases will also need a bvi attached.

Any suggestions for that or are we just left with using single-tagged vlans for those cases?

Thanks

[xthuijs]

ah you know christer in that case it is always a good learning experience right :)

if you have the "vlan room" on your c4k5, I would just leave it single tagged, haul them over to the a9k in their designated EFP's (l2transport subinterfaces) and then pull them in a p2p xcon (if only 2 intfs) or BD (when 3 or more are sharing the same L3 subnet)

in teh BD you can pull in your BVI no problem.

Only stipulation is that with BVI, you have to pop the tags (so it is plain untagged ether inside the BD)

since you have different vlan tags for the different circuits, you'd would wanna pop (and rewrite) them any case so cool on that.

cheers!

xander

[adrian.porreca]

what is the difference in functionality between "MAC withdraw sent on: bridge port up" (default) and "MAC withdraw sent on: bridge port down" (configuring mac withdraw optimize)
regards
Thank you

RP/0/RSP0/CPU0#show l2vpn bridge-domain detail
Wed Feb 21 10:58:29.062 ARG
Legend: pp = Partially Programmed.
Bridge group: HSI, bridge-domain: HSI_B4G, id: 0, state: up, ShgId: 0, MSTi: 0
Coupled state: disabled
VINE state: Default
MAC learning: enabled
MAC withdraw: enabled
MAC withdraw for Access PW: enabled
MAC withdraw sent on: bridge port up ---------------------------Default
MAC withdraw relaying (access to access): disabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: enabled, Logging: enabled, Action: restrict
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 Snooping: disabled
DHCPv4 Snooping profile: none
IGMP Snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: disabled
Bridge MTU: 1500
MIB cvplsConfigIndex: 1
Filter MAC addresses:
Load Balance Hashing: src-dst-ip
P2MP PW: disabled
Create time: 19/01/2018 00:55:21 (4w5d ago)
No status change since creation
ACs: 1 (1 up), VFIs: 1, PWs: 1 (1 up), PBBs: 0 (0 up), VNIs: 0 (0 up)
List of ACs:
AC: TenGigE0/0/0/4.346, state is up
Type VLAN; Num Ranges: 1
Outer Tag: 346
VLAN ranges: [4096, 4096]
MTU 9206; XC ID 0x801ad; interworking none
MAC learning: enabled
Flooding:
Broadcast & Multicast: enabled
Unknown unicast: enabled
MAC aging time: 300 s, Type: inactivity
MAC limit: 4000, Action: none, Notification: syslog
MAC limit reached: no
MAC port down flush: enabled
MAC Secure: enabled, Logging: enabled, Action: restrict
Split Horizon Group: none
Dynamic ARP Inspection: disabled, Logging: disabled
IP Source Guard: disabled, Logging: disabled
DHCPv4 Snooping: disabled
DHCPv4 Snooping profile: none
IGMP Snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
Storm Control: bridge-domain policer
Static MAC addresses:
Statistics:
packets: received 0 (multicast 0, broadcast 0, unknown unicast 0, unicast 0), sent 0
bytes: received 0 (multicast 0, broadcast 0, unknown unicast 0, unicast 0), sent 0
MAC move: 0
Storm control drop counters:
packets: broadcast 0, multicast 0, unknown unicast 0
bytes: broadcast 0, multicast 0, unknown unicast 0
Dynamic ARP inspection drop counters:
packets: 0, bytes: 0
IP source guard drop counters:
packets: 0, bytes: 0
List of Access PWs:
List of VFIs:
VFI HSI-B4G (up)
PW: neighbor 190.225.250.51, PW ID 114346, state is up ( established )
PW class not set, XC ID 0xc0000308
Encapsulation MPLS, protocol LDP
Source address 190.225.250.50
PW type Ethernet, control word disabled, interworking none
Sequencing not set
Load Balance Hashing: src-dst-ip

PW Status TLV in use
MPLS Local Remote
------------ ------------------------------ -------------------------
Label 24716 31267
Group ID 0x0 0x0
Interface HSI-B4G HSI-B4G
MTU 1500 1500
Control word disabled disabled
PW type Ethernet Ethernet
VCCV CV type 0x2 0x2
(LSP ping verification) (LSP ping verification)
VCCV CC type 0x6 0x6
(router alert label) (router alert label)
(TTL expiry) (TTL expiry)
------------ ------------------------------ -------------------------
Incoming Status (PW Status TLV):
Status code: 0x0 (Up) in Notification message
MIB cpwVcIndex: 3221226248
Create time: 19/01/2018 00:55:22 (4w5d ago)
Last time status changed: 19/01/2018 00:59:22 (4w5d ago)
MAC withdraw messages: sent 9, received 8
Forward-class: 0
Static MAC addresses:
Statistics:
packets: received 0 (unicast 0), sent 0
bytes: received 0 (unicast 0), sent 0
MAC move: 0
Storm control drop counters:
packets: broadcast 0, multicast 0, unknown unicast 0
bytes: broadcast 0, multicast 0, unknown unicast 0
DHCPv4 snooping: disabled
IGMP Snooping profile: none
MLD Snooping profile: none
VFI Statistics:
drops: illegal VLAN 0, illegal length 0

[jamahato]

In my opinion,
In case of pure l2 case, this is split horizon case, because,   example, hu0/0/0/1.10 and hu0/0/0/1.20   are part of same bridge-domain bd1.
and now on interface hu0/0/0/1 we received a pkt with vlan tag 10, so as per flooding , this traffic to be flooded to hu0/0/01.20 also.
But this pkt  with vlan tag 10 has now go out of same physical interface hu0/0/0/1 and that is where split horizon filter will be applied and pkt will be dropped.

For L3 case, from CE side we need to send a ARP pkt which will be flooded and go out of hu0/0/01 interface with vlan tag 10, but when the reply comes back which is ARP reply, that arp reply must be handled as that will be received by hu0/0/0/1.20 now and this ARP reply should be correctly sent out of interface hu0/0/0/1.10 .
If ARP request and reply is traced and confirmed correctly exchanged then we are good else traffic will have issue while forwarding.

Hope this helps.

[Christer Lind]

Hello,

Yes, i've checked IP configuration, it's just a dummy 10.0.0.0/24 net with an IP each in the net, once in a blue moon a ping comes through, very odd behaviour...

And they show up in eachother ARP tables... sometimes, again, very odd behaviour.

I get matches on the suggested ACL..

ethernet-services access-list mupp
10 permit any any 0806 (95 hw matches)
20 permit any any (15 hw matches)

[jamahato]

In my opinion,
In case of pure l2 case, this is split horizon case, because,   example, hu0/0/0/1.10 and hu0/0/0/1.20   are part of same bridge-domain bd1.
and now on interface hu0/0/0/1 we received a pkt with vlan tag 10, so as per flooding , this traffic to be flooded to hu0/0/01.20 also.
But this pkt  with vlan tag 10 has now go out of same physical interface hu0/0/0/1 and that is where split horizon filter will be applied and pkt will be dropped.

For L3 case, from CE side we need to send a ARP pkt which will be flooded and go out of hu0/0/01 interface with vlan tag 10, but when the reply comes back which is ARP reply, that arp reply must be handled as that will be received by hu0/0/0/1.20 now and this ARP reply should be correctly sent out of interface hu0/0/0/1.10 .
If ARP request and reply is traced and confirmed correctly exchanged then we are good else traffic will have issue while forwarding.

Hope this helps.