cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3642
Views
0
Helpful
3
Replies

Nexus 1000V - Port Suspended

DanielSheppard
Level 1
Level 1

Good Afternoon,

 

 

 

I am starting to work with the Nexus 1000V switch, however I am running into a little bump and would like some assistance if possible.

 

I have a dell R720 with:

2x Broadcom NetXtreme 1G

2xIntel Ethernet 10G 2P X540-T

 

I am included the models because I think that might somehow be part of the problem.  I am getting the following interface history and any help would be appreciated in deciphering this log:

 

show system internal ethpm event-history interface Eth4/3


>>>>FSM: <Ethernet4/3> has 46 logged transitions<<<<<

1) FSM:<Ethernet4/3> Transition at 330000 usecs after Mon Mar  9 08:31:12 2015
    Previous state: [ETH_PORT_FSM_ST_NOT_INIT]
    Triggered event: [ETH_PORT_FSM_EV_MODULE_INIT_DONE]
    Next state: [ETH_PORT_FSM_ST_INIT_EVAL]

2) FSM:<Ethernet4/3> Transition at 330000 usecs after Mon Mar  9 08:31:12 2015
    Previous state: [ETH_PORT_FSM_ST_INIT_EVAL]
    Triggered event: [ETH_PORT_FSM_EV_IE_PORT_INIT]
    Next state: [ETH_PORT_FSM_ST_SPAN_EVAL]

3) FSM:<Ethernet4/3> Transition at 330000 usecs after Mon Mar  9 08:31:12 2015
    Previous state: [ETH_PORT_FSM_ST_SPAN_EVAL]
    Triggered event: [ETH_PORT_FSM_EV_NON_SPAN_DEST]
    Next state: [ETH_PORT_FSM_ST_WAIT_PRE_CFG]

4) Event:ESQ_START length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq Type:SERIAL

5) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq:determine steps to skip

6) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_ETH_SPAN(174), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x0014600A

7) Event:ESQ_RSP length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_ETH_SPAN(174), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x0014600C

8) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_VSIM(716), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)

9) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_DHCP_SNOOP(360), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)

10) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_PVLAN(180), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)

11) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_FWM(602), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x0014601B

12) Event:ESQ_RSP length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_FWM(602), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x00146020

13) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_ETH_PORT_SEC(191), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x00146026

14) Event:ESQ_RSP length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_ETH_PORT_SEC(191), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x00146028

15) Event:ESQ_REQ length:38, at 330000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_ACLMGR(351), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x0014602E

16) Event:ESQ_RSP length:38, at 340000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_ACLMGR(351), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x00146030

17) Event:ESQ_REQ length:38, at 340000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_IPQOS_MGR(377), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x00146036

18) Event:ESQ_RSP length:38, at 340000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_IPQOS_MGR(377), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)
    RRtoken:0x00146038

19) Event:ESQ_REQ length:38, at 340000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_ASSOC_MGR(714), Opc:MTS_OPC_ETHPM_PORT_PRE_CFG(61441)

20) Event:ESQ_REQ length:38, at 340000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq:Pacer Check

21) Event:ESQ_REQ length:38, at 340000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_PORT_CLIENT(181), Opc:MTS_OPC_ETHPM_LCP_SET_PORT_CFG(61443)
    RRtoken:0x0014603F

22) Event:ESQ_RSP length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_PORT_CLIENT(181), Opc:MTS_OPC_ETHPM_LCP_SET_PORT_CFG(61443)
    RRtoken:0x00146041

23) FSM:<Ethernet4/3> Transition at 350000 usecs after Mon Mar  9 08:31:12 2015
    Previous state: [ETH_PORT_FSM_ST_WAIT_PRE_CFG]
    Triggered event: [ETH_PORT_FSM_EV_PRE_CFG_DONE]
    Next state: [ETH_PORT_FSM_ST_LINK_INIT]

24) FSM:<Ethernet4/3> Transition at 350000 usecs after Mon Mar  9 08:31:12 2015
    Previous state: [ETH_PORT_FSM_ST_LINK_INIT]
    Triggered event: [ETH_PORT_FSM_EV_LINK_UP]
    Next state: [ETH_PORT_FSM_ST_WAIT_BRINGUP]

25) Event:ESQ_START length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq Type:SERIAL

26) Event:ESQ_REQ length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq:determine steps to skip

27) Event:ESQ_REQ length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq:check for issu

28) Event:ESQ_REQ length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_REGISTRY(0), Opc:MTS_OPC_IM_PHY_LINK_STATE_CHANGE(62470)

29) Event:ESQ_REQ length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_VLAN_MGR(167), Opc:MTS_OPC_VLAN_MGR_GET_PORT_TRUNKING_MEMBERSHIP(23562)

30) Event:ESQ_RSP length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_VLAN_MGR(167), Opc:MTS_OPC_VLAN_MGR_GET_PORT_TRUNKING_MEMBERSHIP(23562)

31) Event:ESQ_REQ length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_UDLD(216), Opc:MTS_OPC_ETHPM_PORT_BRINGUP(61442)

32) Event:ESQ_REQ length:38, at 350000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_ETH_PORT_CHANNEL_MGR(378), Opc:MTS_OPC_ETHPM_PORT_BRINGUP(61442)
    RRtoken:0x001460A0

33) Event:ESQ_RSP length:38, at 370000 usecs after Mon Mar  9 08:31:12 2015
    Instance:620806272, Seq Id:0x1, Ret:port not compatible
    [E_MTS_RX] Src:MTS_SAP_ETH_PORT_CHANNEL_MGR(378), Opc:MTS_OPC_ETHPM_PORT_BRINGUP(61442)
    RRtoken:0x001460A0

34) FSM:<Ethernet4/3> Transition at 370000 usecs after Mon Mar  9 08:31:12 2015
    Previous state: [ETH_PORT_FSM_ST_WAIT_BRINGUP]
    Triggered event: [ETH_PORT_FSM_EV_SUSPEND_PORT]
    Next state: [ETH_PORT_FSM_ST_PORT_SUSPENDED]

35) Event: E_DEBUG length:42, at 370000 usecs after Mon Mar  9 08:33:30 2015
    [REINIT] Src:MTS_SAP_ETH_PORT_CHANNEL_MGR(378) Reinit-Type:0

36) FSM:<Ethernet4/3> Transition at 370000 usecs after Mon Mar  9 08:33:30 2015
    Previous state: [ETH_PORT_FSM_ST_PORT_SUSPENDED]
    Triggered event: [ETH_PORT_FSM_EV_EXTERNAL_REINIT_NO_FLAP_REQ]
    Next state: [ETH_PORT_FSM_ST_WAIT_BRINGUP]

37) Event:ESQ_START length:38, at 370000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq Type:SERIAL

38) Event:ESQ_REQ length:38, at 370000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq:determine steps to skip

39) Event:ESQ_REQ length:38, at 370000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    Seq:check for issu

40) Event:ESQ_REQ length:38, at 370000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_REGISTRY(0), Opc:MTS_OPC_IM_PHY_LINK_STATE_CHANGE(62470)

41) Event:ESQ_REQ length:38, at 370000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_VLAN_MGR(167), Opc:MTS_OPC_VLAN_MGR_GET_PORT_TRUNKING_MEMBERSHIP(23562)

42) Event:ESQ_RSP length:38, at 380000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_RX] Src:MTS_SAP_VLAN_MGR(167), Opc:MTS_OPC_VLAN_MGR_GET_PORT_TRUNKING_MEMBERSHIP(23562)

43) Event:ESQ_REQ length:38, at 380000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_UDLD(216), Opc:MTS_OPC_ETHPM_PORT_BRINGUP(61442)

44) Event:ESQ_REQ length:38, at 380000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:SUCCESS
    [E_MTS_TX] Dst:MTS_SAP_ETH_PORT_CHANNEL_MGR(378), Opc:MTS_OPC_ETHPM_PORT_BRINGUP(61442)
    RRtoken:0x0014648A

45) Event:ESQ_RSP length:38, at 380000 usecs after Mon Mar  9 08:33:30 2015
    Instance:620806272, Seq Id:0x1, Ret:port not compatible
    [E_MTS_RX] Src:MTS_SAP_ETH_PORT_CHANNEL_MGR(378), Opc:MTS_OPC_ETHPM_PORT_BRINGUP(61442)
    RRtoken:0x0014648A

46) FSM:<Ethernet4/3> Transition at 380000 usecs after Mon Mar  9 08:33:30 2015
    Previous state: [ETH_PORT_FSM_ST_WAIT_BRINGUP]
    Triggered event: [ETH_PORT_FSM_EV_SUSPEND_PORT]
    Next state: [ETH_PORT_FSM_ST_PORT_SUSPENDED]


    Curr state: [ETH_PORT_FSM_ST_PORT_SUSPENDED]

1 Accepted Solution

Accepted Solutions

Hi Daniel -

Thanks for the updates.

For port-channel, all the ports should have same NIC speed, just like regular physical switch port-channel config. requirement.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus1000/sw/4_2_1_s_v_1_4/interface/configuration/guide/n1000v_if/n1000v_if_5portchannel.html#wp1270393

"

Each port can be in only one port channel. All the ports in a port channel must be compatible; they must use the same speed and duplex mode

"

 

If you have multiple uplink port-channels (1 2G port-channel and 1 20G port-channel) from the same VEM/host, you need to avoid overlapping vlans issue.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus1000/sw/4_2_1_s_v_1_4/port_profile/configuration/guide/n1000v_port_profile/n1000v_portprof_4system.html

"

In an installation where multiple Ethernet port profiles are active on the same VEM, it is recommended that they do not carry the same VLAN(s). The allowed VLAN list should be mutually exclusive. Overlapping VLANs can be configured but may cause duplicate packets to be received by virtual machines in the network.

"

 

Regards,

Michael

View solution in original post

3 Replies 3

mizhang
Level 1
Level 1

hi -

A few more questions -

x- what version of N1Kv?  -> [show ver ]

x- status of the interface e4/3 -> [show interface e4/3 ] and [show run int e4/3 expand ]

x- [show logg log | in suspend ]

 

You can try the following to see if it could fix the issue:

x- create another temp uplink port-profile, and migrate the vminc(i.e. e4/3) to this temp port-profile;

x- if the port comes up fine, then migrate it back to the original port-profile.

 

Hope it helps.

 

Michael

 

I was thinking it might be due to 2 of the NICs being 10G and 2 being 1G, but it sounds like you don't think this is the case.

 

I did try creating two different port profiles and having the 2 10G on one profile and the 2 1G on another.  They both come up in that case (but due to not being 100% familiar with the Nexus line yet, I didn't know you couldn't have all VLANs out both, so it will take some re-engineering if I want to do that).  I moved it back to the original profile and it went into a suspended state again.

 

Can you have Port-Channels with different speed NICs?

 

 

show ver
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.html
Copyright (c) 2002-2014, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.

Software
  loader:    version unavailable [last: loader version not available]
  kickstart: version 5.2(1)SM3(1.1)
  system:    version 5.2(1)SM3(1.1)
  kickstart image file is: bootflash:///n1000vh-dk9-kickstart.5.2.1.SM3.1.1.bin
  kickstart compile time:  12/4/2014 10:00:00 [12/04/2014 19:07:52]
  system image file is:    bootflash:///n1000vh-dk9.5.2.1.SM3.1.1.bin
  system compile time:     12/4/2014 10:00:00 [12/04/2014 20:49:28]


Hardware
  cisco Nexus 1000V Chassis ("Virtual Supervisor Module")
  Intel(R) Xeon(R) CPU         with 4135820 kB of memory.
  Processor Board ID T155D358504

  Device name: SW-06
  bootflash:    3122988 kB

Kernel uptime is 3 day(s), 23 hour(s), 12 minute(s), 1 second(s)

plugin
  Core Plugin, Ethernet Plugin, Virtualization Plugin

 

 

Ethernet4/3 is down (suspended)
  Hardware: Ethernet, address: c81f.66b9.df80 (bia c81f.66b9.df80)
  Port-Profile is ACSNET-UPLINK
  MTU 1500 bytes
  Encapsulation ARPA
  Port mode is trunk
  auto-duplex, 1000 Mb/s
  5 minute input rate 361232 bits/second, 75 packets/second
  5 minute output rate 32 bits/second, 0 packets/second
  Rx
    5638365 Input Packets 2181 Unicast Packets
    5203238 Multicast Packets 432952 Broadcast Packets
    3244083521 Bytes
  Tx
    2363 Output Packets 378 Unicast Packets
    1363 Multicast Packets 623 Broadcast Packets 708 Flood Packets
    542998 Bytes
    112389 Input Packet Drops 0 Output Packet Drops

 

 

 

interface Ethernet4/3
  switchport mode trunk
  switchport trunk allowed vlan 1,5,11-14,100,110,120-122,125,130
  switchport trunk allowed vlan add 140-145,200
  channel-group auto mode on mac-pinning
  no shutdown

 

2015 Mar  9 12:20:27 SW-06 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: port-channel7: port-channel7 is suspended
2015 Mar  9 12:20:27 SW-06 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: port-channel7: port-channel7 is suspended

 

 

 

Hi Daniel -

Thanks for the updates.

For port-channel, all the ports should have same NIC speed, just like regular physical switch port-channel config. requirement.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus1000/sw/4_2_1_s_v_1_4/interface/configuration/guide/n1000v_if/n1000v_if_5portchannel.html#wp1270393

"

Each port can be in only one port channel. All the ports in a port channel must be compatible; they must use the same speed and duplex mode

"

 

If you have multiple uplink port-channels (1 2G port-channel and 1 20G port-channel) from the same VEM/host, you need to avoid overlapping vlans issue.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus1000/sw/4_2_1_s_v_1_4/port_profile/configuration/guide/n1000v_port_profile/n1000v_portprof_4system.html

"

In an installation where multiple Ethernet port profiles are active on the same VEM, it is recommended that they do not carry the same VLAN(s). The allowed VLAN list should be mutually exclusive. Overlapping VLANs can be configured but may cause duplicate packets to be received by virtual machines in the network.

"

 

Regards,

Michael

Review Cisco Networking for a $25 gift card