Re: Network connectivity not working (on port-group with LACP)

compassnl · ‎09-07-2010

I have this config in my nexus 1000v:

port-profile type vethernet Server-network
vmware port-group
switchport mode access
switchport access vlan 2
no shutdown
system vlan 2
state enabled
port-profile type ethernet Server-uplinks
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 2-1001
channel-group auto mode active
no shutdown
system vlan 2
state enabled

The server-uplinks port-group got 3 NICs and they are in LACP portchannelgroup on a 3560s stacked switches.

interface Port-channel37
description ESX1
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport mode trunk
!
interface GigabitEthernet1/0/33
description NLAMSESX01-trunk
switchport access vlan 2
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport mode trunk
channel-group 37 mode active
spanning-tree portfast
!
interface GigabitEthernet2/0/45
switchport access vlan 2
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport mode trunk
channel-group 37 mode active
spanning-tree portfast trunk
!

interface GigabitEthernet3/0/33
description NLAMSESX01-trunk *
switchport access vlan 2
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport mode trunk
channel-group 37 mode active
spanning-tree portfast
!

1. What am i doing wrong? since i cannot get network connectivity trough the (Server-network) NIC...

2. Is the system vlan 2 really neccesary? i dont really understand where that option should be used for, the usage in documentation is very cryptic for me.

Kind regards

Geoffrey

lwatta · ‎09-07-2010

Geoffrey,

On the VSM does the VEM module show up as connected in "show module"? I'm guessing that your port-channel never really comes up and thats why the traffic never passes.

Can you show us "show module" and "show int brief". We need to make sure the port-channel is really coming up.

Also can you add the following two commands to the interfaces on the catalyst switch

cat65k-1(config)# spanning-tree portfast bpdufilter
cat65k-1(config)# spanning-tree portfast bpduguard

Now a quick primer on system vlans.

The Nexus 1000V is a chassis based virtual switch. You have a supervisor (VSM) and line cards (VEM). In a real switch when a line card is inserted, power-on is detected and the supervisor programs the line card over the backplane. In the Nexus 1000V world the backplane is the network. The line card (VEM) needs the network to be up in order to get programmed, but gets it can't forward traffic until it gets it's programming. So we have a chicken-and-egg type issue. The VEM has a very small configuration that it keeps on the ESX host. When a system vlan is tied to a port-profile, that port-profile is now called a system port-profile and is stored locally on the VEM. When the VEM module loads, it looks at the local configuration and if it has any system port-profiles it starts passing traffic on those port-profiles even if it cannot get programmed.

The following port-profiles should always have system vlans

1. your uplink port - (should carry system vlans for Service Console, VMK, N1KV control and packet)

2. Any port-profile for Service Console

3. Any port-profile carrying a VMK interface

4. Any port-profile carrying the control or packet vlan

Essentially any network interface that truly needs to work even if the VEM and VSM cannot communicate.

louis

compassnl · ‎09-08-2010

I can indeed see that 2 of the 3 ports are in operating mode: down (on the catalyst switches)

Also when i look at the etherchannel trough cisco NA, that the only port that is up, is in standalone mode.

The commands you mention are only for non-trunking ports?? the ports are in trunk mode...

This is the output of show mod:

switch(config)# show mod
Mod Ports Module-Type                      Model              Status
--- ----- -------------------------------- ------------------ ------------
1    0      Virtual Supervisor Module        Nexus1000V         active *
3    248    Virtual Ethernet Module          NA                 ok

Mod Sw               Hw
--- --------------- ------
1    4.0(4)SV1(3a)    0.0
3    4.0(4)SV1(3a)    2.0

Mod MAC-Address(es)                         Serial-Num
--- -------------------------------------- ----------
1    00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA
3    02-00-0c-00-03-00 to 02-00-0c-00-03-80 NA

Mod Server-IP        Server-UUID                           Server-Name
--- --------------- ------------------------------------ --------------------
1    10.31.2.240      NA                                    NA
3    10.31.2.10       00000000-0000-435a-4338-303833565758 nlamsesx01

output of show int brief:

--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
mgmt0 -- up 10.31.2.240 1000 1500

--------------------------------------------------------------------------------
Ethernet      VLAN   Type Mode   Status Reason                   Speed     Port
Interface                                                                   Ch #
--------------------------------------------------------------------------------
Eth3/3        1      eth trunk up      none                       1000(D) --
Eth3/4        1      eth trunk up      none                       1000(D) --
Eth3/5        1      eth trunk up      none                       1000(D) --
Eth3/9        1      eth access up      none                       1000(D) 1
Eth3/10       1      eth access up      none                       1000(D) 1

--------------------------------------------------------------------------------
Port-channel VLAN Type Mode   Status Reason                    Speed Protocol
Interface
--------------------------------------------------------------------------------
Po1          1     eth access up      none                      a-1000(D) none
Po2          1     eth trunk link-up No operational members    a-1000(D) lacp

--------------------------------------------------------------------------------
Interface     VLAN   Type Mode   Status Reason                   MTU
--------------------------------------------------------------------------------
Veth1         2      virt access up      none                     1500
Veth2         2      virt access up      none                     1500
Veth4         1      virt access up      none                     1500
Veth6         2      virt access up      none                     1500
Veth7         2      virt access up      none                     1500
Veth8         1      virt access up      none                     1500

--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
ctrl0 -- up -- 1000 1500

2. is very clear, thanks

compassnl · ‎09-08-2010

I now have this up and running , after reboot of the ESX server it seems to be working..

But now i have another issue, i also have a iscsi network which have NICs connected to different switches

these catalyst switches do not support etherchannels so i configured it like this:

port-profile type ethernet iSCSI-uplinks
vmware port-group
switchport mode access
switchport access vlan 1
pinning id 0
channel-group auto mode on mac-pinning
no shutdown
state enabled

port-profile type vethernet iSCSI-network
vmware port-group
switchport mode access
switchport access vlan 1
pinning id 0
no shutdown
state enabled

This configuration does also not seem to work...

lwatta · ‎09-08-2010

Glad you got the other port-channel up.

For your iSCSI config, I suspect a mismatch between your physical network ports and the virtual uplink. How are the physical ports configured on the catalyst for those two nics? Are they trunk or access ports? If they are trunk ports then there is a mismatch.

Also remove the pinning id 0 command under the "ethernet iSCSI-uplinks" port-profile. That command really only works for veth port-profiles.

louis

compassnl · ‎09-09-2010

the Links on the catalyst switches have only VLAN1 configured and are in access mode, so that should be ok.

when i added sub-group-id 0 to the NICs connected to the iscsi switch, Ping to the VM with a NIC connected to the iscsi-network does work.

but a ping (and other connectivity) to other machines from the VM does not work.

lwatta · ‎09-09-2010

Based of what you described there is probably still something wrong with the "Server-uplinks" port-profile.

Are you evaluating the Nexus 1000V or do you have a valid license? If you have a license please open a support call. Opening up a support call at this point will be the fastest way to get resolution to your problem.

If you are evaluating the Nexus 1000V contact me directly and we can setup a webex to try and determine the problem. My email is lwatta@cisco.com.

louis

compassnl · ‎09-09-2010

ok i've send you an email about this... i can say that my first question is resolved so im marking this question as answered and we will continue in the background