Introduction
According to the RFC 7432:
Extended Community enables split-horizon procedures for multihomed sites.
The ESI Label field represents an ES by the advertising PE, and it is used in split-horizon
filtering by other PEs that are connected to the same multihomed Ethernet segment.
ESI Label in extended community is encoded in 24 bits. It further needs to be decoded into 20-bit MPLS label. RFC does not explicitly mention which bits should be omitted during decode procedure.
RFC 8317 gives explicit example on how it should be implemented:
When extended community is advertised along with the MAC/IP Advertisement route
(for known unicast traffic), the Leaf-Indication flag MUST be set to one and the Leaf label
SHOULD be set to zero.
When this extended community is advertised along with the EAD-ES route (with an ESI of zero)
for BUM traffic to enable egress filtering on disposition PEs, the Leaf label MUST be set
to a valid MPLS label (i.e., a non-reserved, assigned MPLS label [RFC3032])
and the Leaf-Indication flag SHOULD be set to zero.
The value of the 20-bit MPLS label is encoded in the high-order 20 bits of the Leaf label field.
The receiving PE MUST ignore the Leaf-Indication flag. A non-valid MPLS label, when sent
along with the EAD-ES route, should be ignored and logged as an error.
To adhere to RFC 8317 recommendations Cisco is changing the way MPLS label being encoded/decoded from Extended Community:
Previous Behavior
In earlier releases Cisco was using the lower 20 bits of this extended community to encode the SHG label.
Current Behavior
Cisco is changing the SHG label encoding to be done from higher 20 bits of extended community.
Introduced by
CSCvm89608 Correction of EVPN ESI Label extcomm
Integrated-releases: 6.5.2, 6.6.1, 7.0.1, 7.1.1
Problem
According to this change routers in same ethernet-segment running with old and new behavior will decode extended community differently, which may lead to inconsistent SHG labels on peering EVPN PE routers. In most of the cases both will just drop BUM packets with incorrect SHG label (in their perspective) but in certain conditions (e.g. when label incorrectly read as NULL) it may cause remote PE to accept such packets and forward to CE potentially causing a loop.
Today Multi-Vendor behavior:
- Cisco IOS-XR and IOS-XE both do bit-shift into higher-order bits of extended community to derive a Label
- Nokia and JunOS is also doing it
Example of output
PE1 (new IOS-XR):
RP/0/RSP0/CPU0:PE1#show evpn ethernet-segment interface bundle-Ether 8 det
***SNIP***
Local SHG label : 24012
Remote SHG labels : 1
2048 : nexthop 1.1.1.1
BGP extended community on PE1 (upper 20 bits):
0000 0101 1101 1100 1100 0000 --- 24012
PE2 (OLD IOS-XR):
BGP extended community on PE1 (lower 20 bits):
0000 0101 1101 1100 1100 0000 --- 384192
RP/0/RSP0/CPU0:PE2#show evpn ethernet-segment interface bundle-Ether 8 det
***SNIP***
Local SHG label : 32779
Remote SHG labels : 1
384192 : nexthop 2.2.2.2
Recommendation to prevent problem
As such it is recommended to:
- Minimize the time both PEs are running different versions (old vs new (see Integrated-releases above))
- Isolate upgraded node from EVPN before the activity:
- "Cost-out" node
- Shutdown corresponding Attachment Circuit Bundle
- Once both PE are upgraded to same release we can bring both in-service
Similar recommendations are applicable to peering with different vendor with SHG label assignment not matching RFC 8317.
