Re: Nexus 93180 Pseudowire config

bepanterprises · ‎08-04-2020

I have a nexus 93180 that I need to run a traffic test on. Connecting a Tberd traffic tester to an SFP in port 1. In port 3 and 4 are two identical optics connected to each other. Ports 5&6 have another set of two identical optics connected together. 7&8 same thing. Port 10 will be 1 last optic on a loop. All of the optics interfaces show up and functional. What I need to be able to do is pass traffic through all of the ports to port 10, then back. The recommendation to accomplish this was to create a pseudowire between the optical pairs and pass traffic to the last port. I attached a graphic given to me as an example. Can anyone provide a sample config of how to set this up?

Christopher Hart · ‎08-04-2020

Hello!

I don't think a Pseudowire will work for what you're trying to accomplish. Traditionally, a Pseudowire is an L2VPN over MPLS. What you need is a configuration that will allow a single packet to iterate through each port on the switch.

The simplest way to do this would be to configure each port such that it is an access switchport in its own unique VLAN. For example, Eth1/1 would be in VLAN 1, Eth1/2 would be in VLAN 2, Eth1/3 would be in VLAN 3, etc. You will also likely need to configure each interface as a Spanning Tree Edge port and/or enable BPDUFilter - otherwise, the device will receive Spanning Tree BPDUs that it created, which could cause some interfaces to stop passing traffic.

A sample configuration for a single interface is below:

interface Ethernet1/1
  no shutdown
  switchport
  switchport mode access
  switchport access vlan 1
  spanning-tree port type edge
  spanning-tree bpdufilter enable

I hope this helps! Thank you!

-Christopher

bepanterprises · ‎08-04-2020

How would the traffic pass through each ports transceiver and back to the test set? The idea is that the switch and the optics are going to be environmentally stress tested, and they want to see if they get any sort of packet loss or error at the test set. In this setup, would I even need to connect two of the ports together, or just loop them all if they are all on their own Vlan?

Sergiu.Daniluk · ‎08-05-2020

Hi @bepanterprises

Although the config suggested by @Christopher Hart is perfectly fine and it will work (you only need to configure a L3 port or SVI on the last interface/vlan in the chain.

Alternatively, you can consider all of the switches as L3 ports, grouped together in pairs of two in different VRFs to allow the routing to flow correctly.

Example:

Eth1/1
 no switchport
 ip address 1.1.1.2/30
 vrf member vrf12

Eth1/2 
 no switchport
 ip address 1.1.1.5/30
 vrf member vrf12

vrf context vrf12
 ip route 1.1.1.94/32 1.1.1.6

Eth1/3
 no switchport
 ip address 1.1.1.6/30
 vrf member vrf34

Eth1/4 
 no switchport
 ip address 1.1.1.9/30
 vrf member vrf34

vrf context vrf34
 ip route 1.1.1.94/32 1.1.1.10
 ip route 1.1.1.1/32 1.1.1.5

(....)

Eth1/45
 no switchport
 ip address 1.1.1.93/30
 vrf member vrf44-45

vrf context vrf44-45
 ip route 1.1.1.1/32 1.1.1.5


Eth1/46   ! this will not work if the interface is down but this is an example
 no switchport
 ip address 1.1.1.94/30
 vrf member vrf46

vrf context vrf34
 ip route 1.1.1.1/32 1.1.1.93

The advantage of this approach is that you can verify hop by hop the traffic/rate/drops etc. Again, this might be not the best alternative, but for sure is fun ^_^.

Stay safe,

Sergiu

Christopher Hart · ‎08-05-2020

Hello!

The answer to your question depends a bit on what type of traffic profile you would like to test with. Multidestination (broadcast, multicast, unknown unicast) traffic would pass through each port's transceiver with this Layer 2 setup and configuration. As such, you would need to configure the T-BERD to send traffic to a MAC address that does not exist or to the broadcast MAC address.

You would still need ports to be connected to each other (Eth1/1 to the traffic generator, Eth1/2 to Eth1/3, Eth1/4 to Eth1/5, etc.) for this test to work. I'd also like to correct something I stated in my previous post - each interface would not be in its own unique VLAN, but rather you would put a pair of ports into its own VLAN. For example:

Eth1/1 and Eth1/2 - VLAN 1
Eth1/3 and Eth1/4 - VLAN 2
Eth1/5 and Eth1/6 - VLAN 3
Eth1/7 and Eth/18 - VLAN 4
etc.

You could also go with @Sergiu.Daniluk's suggestion of configuring Layer 3 ports within VRFs, which will also work. However, one thing to be aware of is that you will need to manually set unique MAC addresses for each L3 interface (this is due to an ASIC limitation - the Cloud Scale ASIC that the Nexus 93180 series uses will drop a packet that is sourced from its own MAC address and destined to the exact same MAC address). I also believe there's a limitation on how many unique MAC addresses you can configure on interfaces of the device (I think it's 16, but this value might be old or incorrect) which would limit the maximum number of interfaces you can test in an L3 setup before packets begin to drop.

I hope this helps!

-Christopher

bepanterprises · ‎09-09-2020

Christopher

Sorry for the delay, this got back burnered, but back on it. Does this look correct? am i missing anything?

vlan 1-12

interface Ethernet1/1
Description to TestSet
switchport
switchport access vlan 1
spanning-tree port type edge
spanning-tree bpdufilter enable
speed 1000
no shutdown

interface Ethernet1/5
description 1G LX - Patched to 1/6
switchport
switchport access vlan 1
spanning-tree port type edge
spanning-tree bpdufilter enable
speed 1000
no shutdown

interface Ethernet1/6
switchport
switchport access vlan 2
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/9
description 1G EX - Patched to 1/13
switchport
switchport access vlan 2
spanning-tree port type edge
spanning-tree bpdufilter enable
speed 1000
no shutdown

interface Ethernet1/13
description 1G ZX
switchport
switchport access vlan 3
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/15
description 1G T - Patched to 1/17
switchport
switchport access vlan 3
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/17
description 10G T
switchport
switchport access vlan 4
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/19
description 10G SR - Patched to 1/20
switchport
switchport access vlan 4
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/20
switchport
switchport access vlan 5
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/23
description 10G LR - Patched to 1/24
switchport
switchport access vlan 5
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/24
switchport
switchport access vlan 6
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/27
description 10G ER - Patched to 1/31
switchport
switchport access vlan 6
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/31
description 10G ZR
switchport
switchport access vlan 7
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/37
description 25G SR - Patched to 1/38
switchport
switchport access vlan 7
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/38
switchport
switchport access vlan 8
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/41
description 25G LR - Patched to 1/42
switchport
switchport access vlan 8
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/42
switchport
switchport access vlan 9
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/49
description 40G LR - Patched to 1/50
switchport
switchport access vlan 9
spanning-tree port type edge
spanning-tree bpdufilter enable
speed 40000
no negotiate auto
no shutdown

interface Ethernet1/50
switchport
switchport access vlan 10
spanning-tree port type edge
spanning-tree bpdufilter enable
speed 40000
no negotiate auto
no shutdown

interface Ethernet1/51
description 100G SR - Patched to 1/52
switchport
switchport access vlan 10
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/52
switchport
switchport access vlan 11
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/53
description 100G LR Patched to 1/54
switchport
switchport access vlan 11
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

interface Ethernet1/54
switchport
switchport access vlan 12
spanning-tree port type edge
spanning-tree bpdufilter enable
no shutdown

Christopher Hart · ‎09-09-2020

Hello!

That looks correct to me! Looking forward to knowing the results of this.

Thank you!

-Christopher

bepanterprises · ‎09-09-2020

Will do. Given this config, would a L3 broadcast traffic not hit them all as well given the VLAN config? Just ping 255.255.255.255 instead of sending to the MAC broadcast?

Christopher Hart · ‎10-24-2020

Hello! Apologies for the delay in response to this.

I would anticipate that an L3 broadcast should traverse across all interfaces the same way an L2 broadcast would. Technically speaking, an L3 broadcast should have an identical Ethernet header as an L2 broadcast (namely, a destination MAC address of ffff.ffff.ffff).

Thank you!

-Christopher