Nexus Fex Offline When Pre-Provisioning

[Anthony Jentsch]

I am setting up a new environment with 2 5548's and some 2248TP-1GE Fex's and Im running into an issue. I have the peer link and peer-keepalive link that appear to be good. When I configure the fex and vpc for the fex  manually on each switch without pre-provisioning the slot the fex comes online and everything appears to be good. I can see all the ineterfaces when doing a sh int br and the sh fex detail shows all good. When I do the exact same thing but pre-provision the slot the fex stays in an offline state. Ive tried diabling the port(s) connected to the fex while configuring everything then enabling them but same thing. below is the state of the fex when pre-provisionng.

NEXUS-5K-2(config-if)# sh fex 100 detail

FEX: 100 Description: FEX0100   state: Offline

  FEX version: 5.2(1)N1(1b) [Switch version: 5.2(1)N1(1b)]

  FEX Interim version: 5.2(1)N1(1b)

  Switch Interim version: 5.2(1)N1(1b)

  Extender Serial: SSI17040FCG

  Extender Model: N2K-C2248TP-1GE,  Part No: 73-13232-01

  Card Id: 99, Mac Addr: f0:29:29:b1:94:02, Num Macs: 64

  Module Sw Gen: 21  [Switch Sw Gen: 21]

  post level: complete

  Pinning-mode: static    Max-links: 1

  Fabric port for control traffic: Eth1/32

  FCoE Admin: false

  FCoE Oper: true

  FCoE FEX AA Configured: false

  Fabric interface state:

    Po100 - Interface Up. State: Active

    Eth1/32 - Interface Up. State: Active

  Fex Port        State  Fabric Port

Logs:

05/10/2013 08:23:52.2900: Module register received

05/10/2013 08:23:52.4031: Registration response sent

05/10/2013 08:23:52.327015: Module Online Sequence

05/10/2013 08:23:52.330597: LC insert failed at sequence 1 :Provision SAP

05/10/2013 08:23:52.330657: Module insert failed

05/10/2013 08:23:52.331040: Module Offline Sequence

05/10/2013 08:23:52.348114: Module Offline

05/10/2013 08:28:52.467775: Deleting route to FEX

05/10/2013 08:28:52.476677: Module disconnected

05/10/2013 08:28:52.477227: Offlining Module

05/10/2013 08:28:52.477598: Deleting route to FEX

05/10/2013 08:28:52.484063: Module disconnected

05/10/2013 08:28:52.485071: Offlining Module

05/10/2013 08:28:52.495437: Deleting route to FEX

05/10/2013 08:28:52.502016: Module disconnected

05/10/2013 08:28:52.502968: Offlining Module

05/10/2013 08:29:27.387077: Module register received

05/10/2013 08:29:27.388227: Registration response sent

05/10/2013 08:29:28.533499: Module Online Sequence

05/10/2013 08:29:28.536795: LC insert failed at sequence 1 :Provision SAP

05/10/2013 08:29:28.536853: Module insert failed

05/10/2013 08:29:28.537235: Module Offline Sequence

05/10/2013 08:29:28.556657: Module Offline

05/10/2013 08:34:28.735906: Deleting route to FEX

05/10/2013 08:34:28.742662: Module disconnected

[Reza Sharifi]

What do you mean by "pre-provision the slot"?

Are you referring to slot/port allocation for VPC?

Can you clarify?

[Anthony Jentsch]

For example for fex 100 and vpc 100 on both 5k's

config t

slot 100

provision model  N2K-C2248TP-1GE

[Reza Sharifi]

You don't need to specify slot number for 5ks.  By default the slot number is 1

for example:

e1/10 is slot 1 port 10.  If you add an additional 16 port module to your 5ks then that is slot 2, but if you don't have any additional port module, then everything is in part of slot 1.

HTH

[fuhrersk8]

Hi Reza and thanks for your reply;

What are the commands for pre-provisioning the FEX's?

[mikej0001]

Hi Resa, when configuring a fex you can use any slot number from 100-199 which also becomes the fex number.

Thus a typical slot pre-provisioning would be:

slot 101
  provision model N2K-C2232P

fex 101
  pinning max-links 1
  description "fex-101"

[Anthony Jentsch]

Reza,

All the documentation Ive been reading recomends pre-provisioning the fex using a slot #. Ive tried not using a slot # in the command and it wont take. Here is an excerpt from the 5k 2k design guide.

Under certain conditions, you may want to apply the interface configurations before the fabric extender has been fully discovered or before the fabric extender has been attached to the Cisco Nexus 5000 Series Switch. To do so, you can use the preprovisioning feature:

slot 198

provision model N2K-C2232P

With the preceding configuration in place, you can configure the ports on the fabric extender even before the fabric extender is connected to the Cisco Nexus 5000 Series Switch.

[Anthony Jentsch]

I figured out what the issue was. I still dont understand it, but instead of using the actual model# (N2K-C2248TP-E-1GE)

in the provision command I used N2K-C2248T and everything worked.

[wesley.dawson]

We experienced this exact same issue last night, and we were initially confused why it would take a less specific model number.  In your 'show fex 100 detail', the FEX information is given (Extender Model: N2K-C2248TP-1GE).  However we made the same exact mistake by using the below config:

slot 100

provision N2K-C2248TP-E-1GE

It may look like a minor difference, but the "E" models of FEXs support bursty applications and I believe some other features.  Your model of FEX is not an "E" model, so when you specified N2K-C2248T you actually specified the actual model of your FEX.

Information for the FEX 2248TP-E:  http://www.cisco.com/en/US/products/ps12136/index.html