Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
2501
Views
5
Helpful
3
Replies

Nexus 5696Q / 2348TQ: FEX don´t download new image

Martin Zett
Level 1
Level 1

‎01-15-2016 12:30 PM - edited ‎03-01-2019 08:09 AM

Hi there,

I have build up two Nexus 5696Q as vPC peer and six single-homed FEX 2348TQ connected to N5696-M12Q modules with 2x40G QSFP+ Bidi transceivers. The components were delivered with installed 7.0(4)N1(1) image. When I finished the configuration everything went fine and the FEX were connected and prepared for hosts. At least I upgraded to the suggested version 7.0(7)N1(1) without any problems.

But after this the fabric extenders on both parent switches hang up in an image download loop. Searching for the cause I found out that this image version is not supported for this FEX models. This is strange because with the primary installed (and also unsupported) image version 7.0(4)N1(1) they were running...?!

So I've upgraded to the supported image version 7.1(3)N1(1) but it doesn't work. The fabric extenders are still offline and don't download the new image from their parent switches:

After upgrade rebooting:

N5696Q1# sh fex det
FEX: 121 Description: FEX121   state: Connected
  FEX version: Unknown [Switch version: 7.1(3)N1(1)]
  FEX Interim version: Unknown
  Switch Interim version: 7.1(3)N1(1)
  Module Sw Gen: 0  [Switch Sw Gen: 21]
  post level: complete
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/2
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po121 - Interface Up. State: Active
    Eth1/2 - Interface Up. State: Active
    Eth1/8 - Interface Up. State: Active
Logs:

FEX: 161 Description: FEX161   state: Connected
  FEX version: Unknown [Switch version: 7.1(3)N1(1)]
  FEX Interim version: Unknown
  Switch Interim version: 7.1(3)N1(1)
  Module Sw Gen: 0  [Switch Sw Gen: 21]
  post level: complete
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/3
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po161 - Interface Up. State: Active
    Eth1/3 - Interface Up. State: Active
    Eth1/9 - Interface Up. State: Active
Logs:

FEX: 171 Description: FEX171   state: Connected
  FEX version: Unknown [Switch version: 7.1(3)N1(1)]
  FEX Interim version: Unknown
  Switch Interim version: 7.1(3)N1(1)
  Module Sw Gen: 0  [Switch Sw Gen: 21]
  post level: complete
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/4
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po171 - Interface Up. State: Active
    Eth1/4 - Interface Up. State: Active
    Eth1/10 - Interface Up. State: Active
Logs:

----------------------------------------------------------------

A few minutes later:

N5696Q1# sh fex det
FEX: 121 Description: FEX121   state: Offline
  FEX version: Unknown [Switch version: 7.1(3)N1(1)]
  FEX Interim version: Unknown
  Switch Interim version: 7.1(3)N1(1)
  Module Sw Gen: 0  [Switch Sw Gen: 21]
  post level: complete
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/2
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po121 - Interface Up. State: Active
    Eth1/2 - Interface Up. State: Active
    Eth1/8 - Interface Up. State: Active
Logs:
03/01/2001 16:03:39.352027: Module timed out
03/01/2001 16:09:25.650685: Deleting route to FEX
03/01/2001 16:09:25.660237: Module disconnected
03/01/2001 16:09:25.661398: Offlining Module
03/01/2001 16:09:25.742278: Deleting route to FEX
03/01/2001 16:09:25.749976: Module disconnected
03/01/2001 16:09:25.751719: Offlining Module
03/01/2001 16:09:25.830050: Deleting route to FEX
03/01/2001 16:09:25.840535: Module disconnected
03/01/2001 16:09:25.842388: Offlining Module

FEX: 161 Description: FEX161   state: Offline
  FEX version: Unknown [Switch version: 7.1(3)N1(1)]
  FEX Interim version: Unknown
  Switch Interim version: 7.1(3)N1(1)
  Module Sw Gen: 0  [Switch Sw Gen: 21]
  post level: complete
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/3
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po161 - Interface Up. State: Active
    Eth1/3 - Interface Up. State: Active
    Eth1/9 - Interface Up. State: Active
Logs:
03/01/2001 16:03:39.472020: Module timed out
03/01/2001 16:09:25.678542: Deleting route to FEX
03/01/2001 16:09:25.690552: Module disconnected
03/01/2001 16:09:25.691694: Offlining Module
03/01/2001 16:09:25.774290: Deleting route to FEX
03/01/2001 16:09:25.784713: Module disconnected
03/01/2001 16:09:25.786728: Offlining Module
03/01/2001 16:09:25.851404: Deleting route to FEX
03/01/2001 16:09:25.859644: Module disconnected
03/01/2001 16:09:25.861410: Offlining Module

FEX: 171 Description: FEX171   state: Offline
  FEX version: Unknown [Switch version: 7.1(3)N1(1)]
  FEX Interim version: Unknown
  Switch Interim version: 7.1(3)N1(1)
  Module Sw Gen: 0  [Switch Sw Gen: 21]
  post level: complete
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/4
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po171 - Interface Up. State: Active
    Eth1/4 - Interface Up. State: Active
    Eth1/10 - Interface Up. State: Active
Logs:
03/01/2001 16:03:39.552016: Module timed out
03/01/2001 16:09:25.706495: Deleting route to FEX
03/01/2001 16:09:25.718185: Module disconnected
03/01/2001 16:09:25.719318: Offlining Module
03/01/2001 16:09:25.798109: Deleting route to FEX
03/01/2001 16:09:25.807360: Module disconnected
03/01/2001 16:09:25.810007: Offlining Module
03/01/2001 16:09:25.882301: Deleting route to FEX
03/01/2001 16:09:25.890307: Module disconnected
03/01/2001 16:09:25.892799: Offlining Module

Is it possible to force an upload from the parent switches to the fabric extenders? Or do you have an idea how I can resolve this problem?

Thanks in advance!

1 person had this problem
Labels:
0 Helpful
3 Replies 3

Martin Zett
Level 1
Level 1

‎01-18-2016 04:41 AM

Upgrade to the 7.2(1)N1(1) image version with the same result:

N5696Q1# sh fex det

FEX: 121 Description: FEX121   state: Connected

  FEX version: Unknown [Switch version: 7.2(1)N1(1)]

  FEX Interim version: Unknown

  Switch Interim version: 7.2(1)N1(1)

  Module Sw Gen: 0  [Switch Sw Gen: 21]

  post level: complete

pinning-mode: static    Max-links: 1

  Fabric port for control traffic: Eth1/2

  FCoE Admin: false

  FCoE Oper: true

  FCoE FEX AA Configured: false

  Fabric interface state:

    Po121 - Interface Up. State: Active

    Eth1/2 - Interface Up. State: Active

    Eth1/8 - Interface Up. State: Active

Logs:

 

FEX: 161 Description: FEX161   state: Connected

  FEX version: Unknown [Switch version: 7.2(1)N1(1)]

  FEX Interim version: Unknown

  Switch Interim version: 7.2(1)N1(1)

  Module Sw Gen: 0  [Switch Sw Gen: 21]

  post level: complete

pinning-mode: static    Max-links: 1

  Fabric port for control traffic: Eth1/3

  FCoE Admin: false

  FCoE Oper: true

  FCoE FEX AA Configured: false

  Fabric interface state:

    Po161 - Interface Up. State: Active

    Eth1/3 - Interface Up. State: Active

    Eth1/9 - Interface Up. State: Active

Logs:

 

FEX: 171 Description: FEX171   state: Connected

  FEX version: Unknown [Switch version: 7.2(1)N1(1)]

  FEX Interim version: Unknown

  Switch Interim version: 7.2(1)N1(1)

  Module Sw Gen: 0  [Switch Sw Gen: 21]

  post level: complete

pinning-mode: static    Max-links: 1

  Fabric port for control traffic: Eth1/4

  FCoE Admin: false

  FCoE Oper: true

  FCoE FEX AA Configured: false

  Fabric interface state:

    Po171 - Interface Up. State: Active

    Eth1/4 - Interface Up. State: Active

    Eth1/10 - Interface Up. State: Active

Logs:

 

 

------------------------------------------------------------------

 

A few minutes later:

 

 

N5696Q1# sh fex det

FEX: 121 Description: FEX121   state: Offline

  FEX version: Unknown [Switch version: 7.2(1)N1(1)]

  FEX Interim version: Unknown

  Switch Interim version: 7.2(1)N1(1)

  Module Sw Gen: 0  [Switch Sw Gen: 21]

  post level: complete

pinning-mode: static    Max-links: 1

  Fabric port for control traffic: Eth1/2

  FCoE Admin: false

  FCoE Oper: true

  FCoE FEX AA Configured: false

  Fabric interface state:

    Po121 - Interface Up. State: Active

    Eth1/2 - Interface Up. State: Active

    Eth1/8 - Interface Up. State: Active

Logs:

03/04/2001 14:44:48.250780: Module timed out

 

FEX: 161 Description: FEX161   state: Offline

  FEX version: Unknown [Switch version: 7.2(1)N1(1)]

  FEX Interim version: Unknown

  Switch Interim version: 7.2(1)N1(1)

  Module Sw Gen: 0  [Switch Sw Gen: 21]

  post level: complete

pinning-mode: static    Max-links: 1

  Fabric port for control traffic: Eth1/3

  FCoE Admin: false

  FCoE Oper: true

  FCoE FEX AA Configured: false

  Fabric interface state:

    Po161 - Interface Up. State: Active

    Eth1/3 - Interface Up. State: Active

    Eth1/9 - Interface Up. State: Active

Logs:

03/04/2001 14:44:48.440764: Module timed out

 

FEX: 171 Description: FEX171   state: Offline

  FEX version: Unknown [Switch version: 7.2(1)N1(1)]

  FEX Interim version: Unknown

  Switch Interim version: 7.2(1)N1(1)

  Module Sw Gen: 0  [Switch Sw Gen: 21]

  post level: complete

pinning-mode: static    Max-links: 1

  Fabric port for control traffic: Eth1/4

  FCoE Admin: false

  FCoE Oper: true

  FCoE FEX AA Configured: false

  Fabric interface state:

    Po171 - Interface Up. State: Active

    Eth1/4 - Interface Up. State: Active

    Eth1/10 - Interface Up. State: Active

Logs:

03/04/2001 14:44:48.550758: Module timed out

0 Helpful

Rajeshkumar Gatti
Cisco Employee
Cisco Employee
In response to Martin Zett

‎01-19-2016 07:42 PM

Can you print the output of "show interface e1/x-y brief"

x-y are the fex fabric interface

If the fex fabric interface shows "SDP timeout/SFP Mismatch" then there is a likelihood that you may be hitting CSCur89241. The fix for the bug is already in the code that you have in the parent switch but it may require the fex to be power cycled a few times to see if it connects with the parent switch.

If it doesn't work then you may have to call TAC and get the steps to manually flash the FEX.

-Raj

Martin Zett
Level 1
Level 1
In response to Rajeshkumar Gatti

‎01-21-2016 12:52 PM

Thank you for answering!

With the TAC we weren´t be able to bring back the fabric extenders in running state. Obviously, the BIOS of the FEX has been damaged. Like you the TAC asked us for power cycle the FEX 4-5 times, unfortunately without any affect. Not sure if the bug you described is responsible. For experiments to flash the FEX manually we don´t have enough time so the TAC opened a RMA and we have replaced the fabric extenders. Now they´re running and finally I can begin to build up our new mirror datacenter next week, thanks SmartNet...

0 Helpful
Customers Also Viewed These Support Documents