I am trying to attach two identical baremetal linux servers with Mullanox 40Gig cards to ACI fabric. One to Leaf103 and another to Leaf104. Both Leaves are C9336C-FX2. I am getting no light on the switch interfaces or on the NICs. LED on the leaves change from amber to off, never go green. ACI throws a fault F0532: "Port is down, reason being notconnect, used by EPG on node 103." Same for node 104. I can take the same QSFP DAC cable and connect it across the two servers and Mullanox cards come up fine. Anything special I need to do for 40G Leaf Access port policy group? Tried it on ACI using 40G and with Auto.
Switch CLI:
#show int status
Eth1/1 LAB_PS2 notconnect trunk full 40G QSFP-H40G-CU5M
# show int e1/1
Ethernet1/1 is down (notconnect)
admin state is up, Dedicated Interface
Port description is LAB_PS2
Hardware: 10000/100000/25000/40000/auto Ethernet, address: 700f.6a15.d013 (bia 700f.6a15.d013)
MTU 9000 bytes, BW 40000000 Kbit, DLY 1 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is trunk
full-duplex, 40 Gb/s, media type is 40G
FEC (forward-error-correction) : disable-fec
Beacon is turned off
Auto-Negotiation is turned on
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
EEE (efficient-ethernet) : n/a
Last link flapped 01:36:14
Last clearing of "show interface" counters never
1 interface resets
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 0 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
RX
0 unicast packets 21 multicast packets 0 broadcast packets
21 input packets 3884 bytes
0 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 input buffer drop 0 input total drop
0 Rx pause
TX
0 unicast packets 11 multicast packets 0 broadcast packets
11 output packets 3663 bytes
0 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 output buffer drops 0 output total drops
0 Tx pause
