Nexus 5010 - not allowing more than 1 fex interface into a port-channel

[dave.chadwick]

Hi - I am trying to create a port-channel that contains 2 fex interfaces to provide SLB to a UNiX server, each time I try to add more that 1 interface into the Channel-group I get the following message:-"port channel cannot have more than 1 members".  I have configured the fex interfaces up as trunk interfaces and have set the lacp to active mode.

interface port-channel62
  switchport mode trunk

interface Ethernet107/1/37
  description NAS DATAMOVER L cge2
  switchport mode trunk
  channel-group 62 mode active

NEXUS-2-5010-B(config-if)# int e107/1/38
NEXUS-2-5010-B(config-if)# channel-group 62 mode act
port channel cannot have more than 1 members

Please note that I am trying to configure multiple interfaces on the FEX not on the 5010...

[DAVIDE RENDINIELLO]

Where I can find cisco's official documentation on this problem?

Is possible that NX-OS 5 has fixed it?

BR,

Dave

[tiwang]

I had the same problem - the only fix is to buy 2248 instead of 2148 - is some broken design in the asic's as far as I know

best regards /ti

[vdsudame]

There is nothing broken or any kind of design flaw here. This is a hardware limitation with the Nexus 2148 platform. This was our first FEX and at the time there was no support in n2k hardware to support more than one link in a portchannel from the fex to the host. Since this is a hardware limitation there wont be any software fix for it.

The current documentation is here:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps10110/data_sheet_c78-507093.html

Table 9 - Layer 2 Features

PortChannel on server ports on Cisco Nexus 2200  platforms

Note: The platform says Nexus 2200. It does not account Nexus 2148.

[Manas Dutta]

Hello Lucein,

I am facing the same issue but while trying to connect my 2248 ports to NetApp FAS3200 Dual Controller.

The 10Gig connection works fine with vpc 13 and vpc 14 as follows (member ports are 5548 ports):

interface port-channel13

  description 10G connection to NetApp_Controller1

  switchport mode trunk

  switchport trunk allowed vlan 161

  spanning-tree port type edge trunk

  speed 10000

  duplex full

  vpc 13

interface port-channel14

  description 10G connection to NetApp_Controller2

  switchport mode trunk

  switchport trunk allowed vlan 161

  spanning-tree port type edge trunk

  speed 10000

  vpc 14

But when I use for 2248 ports which are connected a dual-homed (EvPC) to both 5Ks, I get the error '

operation failed vpc already exists' when I try to use vpc 17 for e102/1/17 & e103/1/17 and vpc 18 for e102/1/18 & e103/1/18.

--------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

      Channel

--------------------------------------------------------------------------------

17    Po17(SU)    Eth      LACP      Eth102/1/17(P)  Eth103/1/17(P)

18    Po18(SU)    Eth      LACP      Eth102/1/18(P)  Eth103/1/18(P)

However, they are up in PO right now but I can't add them to vpc. No matter what vpc number I use. It gives the same error.

Please help.

[habadr]

Hi David

You have to use 2 separate vPCs, each made of "2" interfaces and splitting vlans, or use server's TLB (Transmit Load balancing) as described in following design guide

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572829-01_Design_N5K_N2K_vPC_DG.pdf

Page 37 and figure 29

Thanks

Hatim Badr

[Nicholas Sheridan]

Hatim,

Good reference, many thanks.

Nik

[Andrew Sparkes]

Hi,

I had the same problem, but with slightly different models.

My setup used a Nexus 5548 and 2248 FEX.

The fix to the problem, shut down the uplink ports on the 5548 and disassociate the physical ports with the port channel and delete the port channel.

Next, reassign the two uplink ports to the FEX and recreate the port channel. This allowed me to apply the necessary configuration without error messages. I have provided an example below of the steps carried out.

show FEX - confirmation FEX is online and ports associated.

FEX: 104 Description: FEX104-Po104   state: Online
  FEX version: 7.0(2)N1(1) [Switch version: 7.0(2)N1(1)]
  FEX Interim version: 7.0(2)N1(1)
  Switch Interim version: 7.0(2)N1(1)
  E
  Fabric interface state:
    Po104 - Interface Up. State: Active
    Eth1/18 - Interface Up. State: Active
    Eth1/26 - Interface Up. State: Active
  Fex Port        State  Fabric Port

show port-channel - confirmation port-channel has been built, but only one port assigned

104   Po104(SD)   Eth      NONE      Eth1/18(D)  

initial configuration applied and error message when trying to add second port to PO.

interface Ethernet1/18
  description fex 104 port 1
  switchport mode fex-fabric
  fex associate 104
  channel-group 104

!

interface Ethernet1/26
  description fex 104 port 2
  switchport mode fex-fabric
  fex associate 104

conf t

int e1/26

channel-group 104

command failed: port not compatible [port dynamic logical-interface scale disable]

interface port-channel104
  description fex 104
  switchport mode fex-fabric
  fex associate 104

The above shows the configuration applied and the error message recieved when trying to add second port to PO.

The next set of commands resolved the issue.

conf t

int e1/18
shut
switchport mode access
int e1/26
shut
switchport mode access
exit

no int po 104

Reapply configuration, but using port range command.

!

int e1/18 , e1/26
switchport mode fex-fabric
fex associate 104
channel-group 104

!

show commands confirm FEX returned to service and both ports in PO

sh fex

104   FEX104-Po104                Online   N2K-C2248TP-E-1GE   FOX1744G5TC

show port-channel summary
104   Po104(SU)   Eth      NONE      Eth1/18(P)   Eth1/26(P)

Hopefully the above will assist and is easy to follow.