FEX with ACI - host interfaces stay up during upgrade

[Alexander Pickar]

Hi Community,

We have faced an issue with FEX connected devices during our recent ACI upgrade.

- We did an ACI Fabric software upgrade upgrading first Odd and the Even switches, having FEXes attached to Leaf switches (Leaf 209 connecting to FEX101-A, Leaf 210 connecting to FEX101-B, no cross-connection between Leaf 209 and FEX101-B or Leaf210 and FEX101-A).

- While waiting for Leaf switches to be upgraded, FEX was put to "offline" state,

- Interfaces towards the connected hosts remained in the "Up" state,

- Because dual attached servers did not have a link down situation, they did not fall over to the other interface / fex / leaf.

Any hint, how to change this behaviour?

Is there anything else we can do except for shutting down the FEX HIF ports before upgrading the Leaf switches?

Here's an example of FEX status:

LEAF210# show fex
FEX FEX FEX FEX
Number Description State Model Serial
--------------------------------------------------------------------------
101 FEX0101 Offline N2K-C2248TP-1GE SSI15430NPA
102 FEX0102 Offline N2K-C2248TP-1GE SSI161501XQ
103 FEX0103 Offline N2K-C2248TP-1GE SSI161507W6
LEAF210#

LEAF210#show interface status

...

Eth101/1/1 mbcznt0b51lis1 connected trunk full 100M 10/100/1000BaseT
Eth101/1/2 mbcznt0b51lis2 connected trunk full 100M 10/100/1000BaseT
Eth101/1/3 mbcznt0b51lis3 connected trunk full 100M 10/100/1000BaseT
Eth101/1/4 mbcznt0b51lmp01:03 connected trunk full 100M 10/100/1000BaseT

....

Eth102/1/1 apcbbclant connected trunk full 100M 10/100/1000BaseT
Eth102/1/2 MYCZSL0BL0ACPS connected trunk full 1G 10/100/1000BaseT

....

Thanks,

Alexander

[maamann]

Hi

If i remember me correctly , we had the same problem in our POC.

The statement from Cisco was , it works as designed and it is a hardware issue that can not be solved via Software.

regards Markus

[jvenzhof]

Hi Markus,

could you please share reference to your SR?

It seems it might be "just" a SW problem.

Thanks,

Josef

[maamann]

HI

what i can share is the Bug ID CSCvf06777 and it is fixed in NX-OS

This BUG is related to classical Networking with N5K and Fexes.

But that BUG is the reason why the decision has fallen against Fexes in general.

Regards Markus

[jvenzhof]

Hi Markus,
in ACI it seems to be a different one (CSCvp71221) which is fixed in the upcoming 4.2 release.
Hopefully we could get the fix into current release trains as well.

Regards,
Josef

[maamann]

Hi Josef

i hope you mean with "Hopefully we could get the fix into current release trains as well."

The long term release  3.2.x ?

Regards Markus