Hi all, looking for advice on an issue that we facing for traffic visibility by connecting spine and leaf via 40G BiDi taps.
This is the brief description of the issue.
We are connecting ACI spine and leaf via NetScout tap as in attached picture.
Spine and leaf connection up when the cables are connected to the network slots of the taps.
Once the monitor slots are cabled to Nexus data broker, we expect traffic to be sent to the data broker.
Once the link is enabled manually in the data broker ports (which are currently shut), we see it’s not coming up and also the links at spine and leaf goes down.
If I shut the data broker ports, the spine and leaf ports comes up. I am not sure about this behavior.
As a testing, we removed NetScout tap and plug a leaf link directly to the data broker ports, they are up.
Its just via the NetScout tap the links are not coming up at the data broker. We already installed NDB inside the data broker chassis and through GUI, I see everything seems to be fine.
I am not sure what is the issue from switching side via the Data Broker, hence need some guidance. We can't find a working sample for this anywhere, however a few points to state as below.
1. Spine and leaf has not been discovered or registered yet via the APIC contollers, we just testing if all cabling is good at this point.
2. Spine and leaf are on ACI OS, where spine is 9504 model while leaf are 93180FX.
3. Data broker is using NXOS with embedded NDB installed. All the basic configurations on data broker is done (spanning-tree bpdufilter enable on ports, no spanning tree on all vlans, spanning tree as mst and ports blocking multicast and unicast traffic with operational mode tap-aggregation)
To me, it sounds something related to autonegotiation. have you tried configuring speed manually to 40G on NDB N9K?
Apart of the problem, why would you want to add the tap inline between spine-leaf? Note that the traffic inside the fabric is iVXLAN encapsulated. Also if you have for example Intersite L3out there are options to enable CloudSec between VTEPs, so your monitoring will not work.
I would recommend Access SPAN instead of Taps. Reason is this gives best visibility and allows you to capture traffic that is switched within a leaf as well as that goes across the fabric. What exactly are you trying to accomplish with the TAP that can't be done with SPAN?