North-South traffic in SD-Access (border node MAC address)

[axe1501]

My question is on a typical north-south packet walk scenario where a host in an SD-Access fabric needs to communicate with an external node. Below is a simple representation:

Src_Host------[Edge_Node]-------[Underlay]--------[Border_Node]-------[Fusion_Rtr]--------Dst_Host

When the Edge_Node looks up the IP address of Dst_Host from the Control_Plane/Map_Server/Resolver node (not shown above), the Control_Plane node returns a negative map reply (i.e. if the Border_Node did not explicitly register the address of Dst_Host). The Edge_Node will then send the packet to its configured Proxy_Etr (Which is the Border_Node).

My question is...given the packet will be encapsulated in VXLAN before sending to Underlay, what source and destination MAC address does Edge_Node use for the Original frame, prior to encapsulation?

Please let me know if you need any clarifying details. Appreciate the help on this.

Andrew

[inderdeeps]

I think the below link will help you to understand more on L2 EID and L3 EID where MAC address using LISP Instance Id.

https://www.ciscolive.com/c/dam/r/ciscolive/emea/docs/2020/pdf/R6BGArNQ/TECCRS-3810.pdf

Regards

Inderdeep Singh

www.thenetworkdna.com 

[axe1501]

Thanks for the reference document. I will take a look

[kristoff1]

Hi Andrew,

Why is this important? Because the L2 part isn't important between the border and Edge.

If traffic is going to the PETR the traffic is routed so at that moment the mac address doesn't matter anymore in my opinion.

Can you please clarify what you want to achieve with this question?

Kind regards,

kristoff

[axe1501]

Sure...and thanks for the response. I'm trying to understand the behavior of VXLAN when forwarding via L3 VNI.

In the L2 VNI case, its straightforward...the MAC addresses in the VXLAN packet represent the src and dst hosts (which is what the Fabric_Edge_Node queried the Map Server to retrieve).

However in the L3 VNI forwarding case, after the Fabric_Edge_Node performs a LISP lookup and determines the RLOC to be the PETR/Border_Node (i.e. in my original example), because a VXLAN packet needs to be sent to the Border_Node, there needs to be a src and dst MAC address embedded in the frame (VXLAN requires this vs. LISP which does not). I understand this wouldn't be needed (i.e. given this is a L3 forwarding scenario) and in fact isn't needed if this were a pure LISP-encapsulated packet. However in the VXLAN case there needs to be a src and dst MAC address and i'm trying to understand the Fabric_Edge_Node's logic in determining those addresses.

If there's something fundamental i'm missing please let me know.

Thanks,

Andrew

[willwetherman]

Hi @axe1501 

I looked into this recently as well as I couldn't find any supporting documentation that explains this logic.

I took a packet capture when a host within the fabric (10.120.0.10) pings a host outside the fabric (10.1.10.11). From what I can see, the fabric edge node inserts a dummy source/destination MAC address into the inner header of the L3VNI VXLAN encapsulated packet when it is sent to the PETR/Border node. See attached that I put together. 

Source MAC: 00:00:0c:9f:00:00

Destination MAC: ba:25:cd:f4:ad:38

I took a packet capture on different fabric edge node to border node uplinks, on different fabric edge nodes as well as for different source/destination IP addresses, and the encapsulated source/destination MAC addresses were the same every time so I'm assuming that these are programmed in software just for the purposes for L3VNI VXLAN encapsulation

Hopefully someone else can confirm this behaviour as I'm interested to know myself.

[axe1501 · Accepted solution]

@willwetherman 

Thanks so much for that response! This answers my question. That makes sense if Cisco uses dummy values since MAC addresses aren't relevant for L3VNI comms.

I appreciate the pcaps...it really helped!

[willwetherman]

That's not a problem. Let me know if you find any other resources that supports this.

Also I just noticed from my packet capture that my fabric edge to border node source MAC address isn't actually 00:00:0c:9f:ad:f1. The last 2 bytes of ad:f1 are correct, however the first 4 bytes have been replaced by the same 4 bytes used by the source MAC address within the inner header. ARP resolution between the fabric edge and border node is correct so I'm not sure why this is unless it is something odd that Wireshark has changed. 

[axe1501]

@willwetherman 

Interesting. There prob. a formula Cisco uses to vary the value.

Will let you know if I find any other theories.

Thanks!