Buy or Renew
Buy or Renew
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
4643
Views
1
Helpful
2
Replies

Problem with nexus 1000 and HP Flex-10

bashir.rabbani
Level 4
Level 4

‎04-08-2010 02:30 AM

Hi!

I have a weird behavior on almost all of mine ESX hosts.

Vmotion don’t work between all hosts though they use the same port-profile in the nexus 1000 switch. Servers can ping each other when they are in the same ESX-host but they cannot ping the DG or servers in other ESX-hosts.  On some ESX hosts this work fine but in others this doesn’t work at all. And the problem is not just the vmotion vlan, all other vlans seems to have the same problem.

Any help will be greatly appreciated.

I will attach the nexus config file.

Here is some show commands:

N1k-VSM-FortV# sh module
Mod  Ports  Module-Type                      Model              Status
---  -----  -------------------------------- ------------------ ------------
1    0      Virtual Supervisor Module        Nexus1000V         active *
2    0      Virtual Supervisor Module        Nexus1000V         ha-standby
3    248    Virtual Ethernet Module          NA                 ok
4    248    Virtual Ethernet Module          NA                 ok
5    248    Virtual Ethernet Module          NA                 ok
6    248    Virtual Ethernet Module          NA                 ok
7    248    Virtual Ethernet Module          NA                 ok
8    248    Virtual Ethernet Module          NA                 ok
9    248    Virtual Ethernet Module          NA                 ok
10   248    Virtual Ethernet Module          NA                 ok
11   248    Virtual Ethernet Module          NA                 ok
12   248    Virtual Ethernet Module          NA                 ok
13   248    Virtual Ethernet Module          NA                 ok
14   248    Virtual Ethernet Module          NA                 ok
15   248    Virtual Ethernet Module          NA                 ok
16   248    Virtual Ethernet Module          NA                 ok
17   248    Virtual Ethernet Module          NA                 ok
18   248    Virtual Ethernet Module          NA                 ok
19   248    Virtual Ethernet Module          NA                 ok
20   248    Virtual Ethernet Module          NA                 ok
21   248    Virtual Ethernet Module          NA                 ok
22   248    Virtual Ethernet Module          NA                 ok
23   248    Virtual Ethernet Module          NA                 ok

Mod  Sw               Hw
---  ---------------  ------
1    4.0(4)SV1(2)     0.0
2    4.0(4)SV1(2)     0.0
3    4.0(4)SV1(2)     1.11
4    4.0(4)SV1(2)     1.11
5    4.0(4)SV1(2)     1.11
6    4.0(4)SV1(2)     1.11
7    4.0(4)SV1(2)     1.11
8    4.0(4)SV1(2)     1.11
9    4.0(4)SV1(2)     1.11
10   4.0(4)SV1(2)     1.9
11   4.0(4)SV1(2)     1.9
12   4.0(4)SV1(2)     1.9
13   4.0(4)SV1(2)     1.9
14   4.0(4)SV1(2)     1.9
15   4.0(4)SV1(2)     1.9
16   4.0(4)SV1(2)     1.9
17   4.0(4)SV1(2)     1.9
18   4.0(4)SV1(2)     1.9
19   4.0(4)SV1(2)     1.9
20   4.0(4)SV1(2)     1.9
21   4.0(4)SV1(2)     1.9
22   4.0(4)SV1(2)     1.9
23   4.0(4)SV1(2)     1.9

Mod  MAC-Address(es)                         Serial-Num
---  --------------------------------------  ----------
1    00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8  NA
2    00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8  NA
3    02-00-0c-00-03-00 to 02-00-0c-00-03-80  NA
4    02-00-0c-00-04-00 to 02-00-0c-00-04-80  NA
5    02-00-0c-00-05-00 to 02-00-0c-00-05-80  NA
6    02-00-0c-00-06-00 to 02-00-0c-00-06-80  NA
7    02-00-0c-00-07-00 to 02-00-0c-00-07-80  NA
8    02-00-0c-00-08-00 to 02-00-0c-00-08-80  NA
9    02-00-0c-00-09-00 to 02-00-0c-00-09-80  NA
10   02-00-0c-00-0a-00 to 02-00-0c-00-0a-80  NA
11   02-00-0c-00-0b-00 to 02-00-0c-00-0b-80  NA
12   02-00-0c-00-0c-00 to 02-00-0c-00-0c-80  NA
13   02-00-0c-00-0d-00 to 02-00-0c-00-0d-80  NA
14   02-00-0c-00-0e-00 to 02-00-0c-00-0e-80  NA
15   02-00-0c-00-0f-00 to 02-00-0c-00-0f-80  NA
16   02-00-0c-00-10-00 to 02-00-0c-00-10-80  NA
17   02-00-0c-00-11-00 to 02-00-0c-00-11-80  NA
18   02-00-0c-00-12-00 to 02-00-0c-00-12-80  NA
19   02-00-0c-00-13-00 to 02-00-0c-00-13-80  NA
20   02-00-0c-00-14-00 to 02-00-0c-00-14-80  NA
21   02-00-0c-00-15-00 to 02-00-0c-00-15-80  NA
22   02-00-0c-00-16-00 to 02-00-0c-00-16-80  NA
23   02-00-0c-00-17-00 to 02-00-0c-00-17-80  NA

Mod  Server-IP        Server-UUID                           Server-Name
---  ---------------  ------------------------------------  --------------------
1    xxx.xx.x.21      NA                                    NA
2    xxx.xx.x.21      NA                                    NA
3    xxx.xx.x.131     35303933-3134-435a-4330-303134374650  al-esx101-prod.fortv.net
4    xxx.xx.x.132     35303933-3134-435a-4330-303134374753  al-esx102-prod.fortv.net
5    xxx.xx.x.133     35303933-3134-435a-4330-303134374750  al-esx103-prod.fortv.net
6    xxx.xx.x.134     35303933-3134-435a-4330-30313437484d  al-esx104-prod.fortv.net
7    xxx.xx.x.135     35303933-3134-435a-4330-303134374738  al-esx105-prod.fortv.net
8    xxx.xx.x.136     35303933-3134-435a-4339-343838365232  al-esx106-prod.fortv.net
9    xxx.xx.x.137     35303933-3134-435a-4330-303134374848  al-esx107-prod.fortv.net
10   xxx.xx.x.138     35303933-3134-435a-4330-303134374851  al-esx108-vdi
11   xxx.xx.x.139     35303933-3134-435a-4330-30313437464c  al-esx109-vdi
12   xxx.xx.x.140     35303933-3134-435a-4330-303134374838  al-esx110-vdi
13   xxx.xx.x.141     35303933-3134-435a-4330-30313437464b  al-esx111-vdi
14   xxx.xx.x.142     35303933-3134-435a-4330-303134374835  al-esx112-vdi
15   xxx.xx.x.145     35303933-3134-435a-4330-30313437464e  al-esx115-app
16   xxx.xx.x.146     35303933-3134-435a-4330-303134374830  al-esx116-app
17   xxx.xx.x.143     35303933-3134-435a-4330-303134374744  al-esx113-vdi
18   xxx.xx.x.144     35303933-3134-435a-4339-343838365230  al-esx114-vdi
19   xxx.xx.x.231     35303933-3134-435a-4330-30313437464a  al-esx201-vdi
20   xxx.xx.x.232     35303933-3134-435a-4330-303134374746  al-esx202-vdi
21   xxx.xx.x.233     35303933-3134-435a-4330-30313437474e  al-esx203-vdi
22   xxx.xx.x.239     35303933-3134-435a-4330-303134374837  al-esx209-app
23   xxx.xx.x.240     35303933-3134-435a-4330-303134374836  al-esx210-app


* this terminal session
N1k-VSM-FortV#

N1k-VSM-FortV# show svs neighbors

Active Domain ID: 5

AIPC Interface MAC: 0050-56a7-46fa
Inband Interface MAC: 0050-56a7-5a39

Src MAC           Type   Domain-id    Node-id     Last learnt (Sec. ago)
------------------------------------------------------------------------

0050-56a7-30b8     VSM         5         0201     1123035.87
0002-3d40-0502     VEM         5         0302     165458.22
0002-3d40-0503     VEM         5         0402     155390.46
0002-3d40-0504     VEM         5         0502     154634.60
0002-3d40-0505     VEM         5         0602     153954.96
0002-3d40-0506     VEM         5         0702     71454.35
0002-3d40-0507     VEM         5         0802     152928.87
0002-3d40-0508     VEM         5         0902     152509.86
0002-3d40-0509     VEM         5         0a02     93198.45
0002-3d40-050a     VEM         5         0b02     93032.88
0002-3d40-050b     VEM         5         0c02     69893.90
0002-3d40-050c     VEM         5         0d02     87083.88
0002-3d40-050d     VEM         5         0e02     87332.18
0002-3d40-050e     VEM         5         0f02      2284.95
0002-3d40-050f     VEM         5         1002      1851.31
0002-3d40-0510     VEM         5         1102     87661.64
0002-3d40-0511     VEM         5         1202     87888.06
0002-3d40-0512     VEM         5         1302     165458.22
0002-3d40-0513     VEM         5         1402     165458.22
0002-3d40-0514     VEM         5         1502     162594.17
0002-3d40-0515     VEM         5         1602      1401.75
0002-3d40-0516     VEM         5         1702       940.90

N1k-VSM-FortV#

Labels:
Preview file
21 KB
2 Replies 2

Robert Burns
Cisco Employee
Cisco Employee

‎04-08-2010 03:39 AM

Bashir,

At first glance there's a few config issues:

On all your uplink port profiles you define a native VLAN, but you don't allow this VLAN on your uplink.  All vlans including Native VLANs need to be allowed on the trunk.  Also caution using Native VLANs.  Using them allows for VLAN-mismatching easily.  Ex. If you set a Native VLAN on the 1000v, and set a different VLAN on the upstream Switch, you'll potentially bridge VLANs and cause issues.  Normally you should only be using ONE native VLAN, not multple - unless these are distinct networks.

As for not being able to even ping your default GW, ensure your IP subnet is correct on VLAN 1550.  Can you ping the GW from a vSwitch if you migrate your SC and tag that port group with VLAN 1550?


port-profile type ethernet uplink-prod
  description Uplink production
  vmware port-group
  switchport mode trunk
  switchport trunk native vlan 3004
  switchport trunk allowed vlan 100-102,200,203,300,710
  channel-group auto mode on mac-pinning
  no shutdown
  state enabled
port-profile type ethernet uplink-pvs
  description Uplink pvs
  vmware port-group
  switchport mode trunk
  switchport trunk native vlan 3003
  switchport trunk allowed vlan 301
  channel-group auto mode on mac-pinning
  no shutdown
  state enabled
port-profile type ethernet uplink-sc_vmotion
  description Uplink for vmotion sc/mgmt control packet
  vmware port-group
  switchport mode trunk
  switchport trunk native vlan 3002
  switchport trunk allowed vlan 202,1550,3040-3041
  channel-group auto mode on mac-pinning
  no shutdown
  system vlan 1550,3040-3041
  state enabled

Take a look at what I've pointed out.  If that doesn't fix your issue pleaes provide the following:

1. Configuration of the upstream switchports your "uplink-sc_vmotion" interface connects to.

2.  From one of the ESX host this is not working on, provide the output of "vemcmd show trunk", "vemcmd show port" and "vemcmd show vlan"

Regards,

Robert

0 Helpful

bashir.rabbani
Level 4
Level 4
In response to Robert Burns

‎04-08-2010 05:41 AM

Hi

Robert: Why do I need the native vlans in the trunk, we use native vlan just because we don’t want the vlan1 to flow in the network. The native vlan have no other function in our case.

Yes I can ping the DGW and vlan 1550 is correct. I can also ping all the ESX hosts from the nexus 1000 switch.

Now the problem is a bit hard to explain but a example is like this:

From VEM(Module) 8 I can migrate a server to VEM(Module) 9. But it doesn’t work to migrate the same server to VEM(Module) 7. There seem to be a network failure.

If I instead of this way use the standard VMware switch (bypass the nexus 1000) it works just fine.

I will attach the port-config from both nexus 5000 switches.

here is the output of "vemcmd show trunk", "vemcmd show port" and "vemcmd show vlan" on the ESX host that this don’t work on and the same output from ESX that this works on.

Not Working ESX-HOST

N1k-VSM-FortV# module vem 7 execute vemcmd show trunk
Trunk port 16 native_vlan 3002 CBL 1
vlan(1) cbl 1, vlan(1550) cbl 4, vlan(3002) cbl 1, vlan(3040) cbl 4, vlan(3041) cbl 4,
Trunk port 17 native_vlan 3002 CBL 1
vlan(1) cbl 1, vlan(1550) cbl 4, vlan(3002) cbl 1, vlan(3040) cbl 4, vlan(3041) cbl 4,
Trunk port 18 native_vlan 3004 CBL 1
vlan(1) cbl 1, vlan(102) cbl 4, vlan(203) cbl 4, vlan(300) cbl 4, vlan(3004) cbl 1,
Trunk port 19 native_vlan 3004 CBL 1
vlan(1) cbl 1, vlan(102) cbl 4, vlan(203) cbl 4, vlan(300) cbl 4, vlan(3004) cbl 1,
Trunk port 304 native_vlan 3004 CBL 1
vlan(1) cbl 1, vlan(102) cbl 4, vlan(203) cbl 4, vlan(300) cbl 4, vlan(3004) cbl 1,
Trunk port 305 native_vlan 3002 CBL 1
vlan(1) cbl 1, vlan(1550) cbl 4, vlan(3002) cbl 1, vlan(3040) cbl 4, vlan(3041) cbl 4,

N1k-VSM-FortV# module vem 7 execute vemcmd show port
  LTL    IfIndex   Vlan    Bndl  SG_ID Pinned_SGID  Type  Admin State  CBL Mode   Name
    8          0   3969       0     32          32  VIRT     UP    UP    4 Access l20
    9          0   3969       0     32          32  VIRT     UP    UP    4 Access l21
   10          0   3040       0     32           0  VIRT     UP    UP    4 Access l22
   11          0   3968       0     32          32  VIRT     UP    UP    4 Access l23
   12          0   3041       0     32           1  VIRT     UP    UP    4 Access l24
   13          0      1       0     32          32  VIRT     UP    UP    0 Access l25
   14          0   3967       0     32          32  VIRT     UP    UP    4 Access l26
   15          0   3967       0     32          32  VIRT     UP    UP    4 Access l27
   16   1a060000   3002 T   305      0          32  PHYS     UP    UP    1  Trunk vmnic0
   17   1a060100   3002 T   305      1          32  PHYS     UP    UP    1  Trunk vmnic1
   18   1a060200   3004 T   304      2          32  PHYS     UP    UP    1  Trunk vmnic2
   19   1a060300   3004 T   304      3          32  PHYS     UP    UP    1  Trunk vmnic3
   48   1b060000    202       0     32          32  VIRT     UP    UP    4 Access vmk0
   49   1b060010   1550       0     32           0  VIRT     UP    UP    4 Access vswif0
   51   1b060030   1550       0     32           0  VIRT     UP    UP    4 Access IOMeterTest7 ethernet0
  304   16000007   3004 T     0     32          32  VIRT     UP    UP    1  Trunk
  305   16000008   3002 T     0     32          32  VIRT     UP    UP    1  Trunk

N1k-VSM-FortV# module vem 7 execute vemcmd show vlan
BD 1, vdc 1, vlan 1, 6 ports
Portlist:
     16  vmnic0
     17  vmnic1
     18  vmnic2
     19  vmnic3
    304
    305

BD 102, vdc 1, vlan 102, 3 ports
Portlist:
     18  vmnic2
     19  vmnic3
    304

BD 202, vdc 1, vlan 202, 1 ports
Portlist:
     48  vmk0

BD 203, vdc 1, vlan 203, 3 ports
Portlist:
     18  vmnic2
     19  vmnic3
    304

BD 300, vdc 1, vlan 300, 3 ports
Portlist:
     18  vmnic2
     19  vmnic3
    304

BD 301, vdc 1, vlan 301, 0 ports
Portlist:
BD 1550, vdc 1, vlan 1550, 5 ports
Portlist:
     16  vmnic0
     17  vmnic1
     49  vswif0
     51  IOMeterTest7 ethernet0
    305

BD 1560, vdc 1, vlan 1560, 0 ports
Portlist:
BD 3002, vdc 1, vlan 3002, 3 ports
Portlist:
     16  vmnic0
     17  vmnic1
    305

BD 3004, vdc 1, vlan 3004, 3 ports
Portlist:
     18  vmnic2
     19  vmnic3
    304

BD 3005, vdc 1, vlan 3005, 0 ports
Portlist:
BD 3006, vdc 1, vlan 3006, 0 ports
Portlist:
BD 3040, vdc 1, vlan 3040, 4 ports
Portlist:
     10  l22
     16  vmnic0
     17  vmnic1
    305

BD 3041, vdc 1, vlan 3041, 4 ports
Portlist:
     12  l24
     16  vmnic0
     17  vmnic1
    305

BD 3967, vdc 1, vlan 3967, 2 ports
Portlist:
     14  l26
     15  l27

BD 3968, vdc 1, vlan 3968, 3 ports
Portlist:
      1  inband
      5  inband port security
     11  l23

BD 3969, vdc 1, vlan 3969, 2 ports
Portlist:
      8  l20
      9  l21

N1k-VSM-FortV#

Working ESX - HOST

N1k-VSM-FortV# module vem 8 execute vemcmd show trunk

Trunk port 16 native_vlan 3002 CBL 1
vlan(1) cbl 1, vlan(202) cbl 4, vlan(1550) cbl 4, vlan(3002) cbl 1, vlan(3040) cbl 4, vlan(3041) cbl 4,
Trunk port 17 native_vlan 3002 CBL 1
vlan(1) cbl 1, vlan(202) cbl 4, vlan(1550) cbl 4, vlan(3002) cbl 1, vlan(3040) cbl 4, vlan(3041) cbl 4,
Trunk port 18 native_vlan 3004 CBL 1
vlan(1) cbl 1, vlan(102) cbl 4, vlan(203) cbl 4, vlan(300) cbl 4, vlan(3004) cbl 1,
Trunk port 19 native_vlan 3004 CBL 1
vlan(1) cbl 1, vlan(102) cbl 4, vlan(203) cbl 4, vlan(300) cbl 4, vlan(3004) cbl 1,
Trunk port 304 native_vlan 3004 CBL 1
vlan(1) cbl 1, vlan(102) cbl 4, vlan(203) cbl 4, vlan(300) cbl 4, vlan(3004) cbl 1,
Trunk port 305 native_vlan 3002 CBL 1
vlan(1) cbl 1, vlan(202) cbl 4, vlan(1550) cbl 4, vlan(3002) cbl 1, vlan(3040) cbl 4, vlan(3041) cbl 4,
N1k-VSM-FortV#

N1k-VSM-FortV# module vem 8 execute vemcmd show port
  LTL    IfIndex   Vlan    Bndl  SG_ID Pinned_SGID  Type  Admin State  CBL Mode   Name
    8          0   3969       0     32          32  VIRT     UP    UP    4 Access l20
    9          0   3969       0     32          32  VIRT     UP    UP    4 Access l21
   10          0   3040       0     32           0  VIRT     UP    UP    4 Access l22
   11          0   3968       0     32          32  VIRT     UP    UP    4 Access l23
   12          0   3041       0     32           1  VIRT     UP    UP    4 Access l24
   13          0      1       0     32          32  VIRT     UP    UP    0 Access l25
   14          0   3967       0     32          32  VIRT     UP    UP    4 Access l26
   15          0   3967       0     32          32  VIRT     UP    UP    4 Access l27
   16   1a070000   3002 T   305      0          32  PHYS     UP    UP    1  Trunk vmnic0
   17   1a070100   3002 T   305      1          32  PHYS     UP    UP    1  Trunk vmnic1
   18   1a070200   3004 T   304      2          32  PHYS     UP    UP    1  Trunk vmnic2
   19   1a070300   3004 T   304      3          32  PHYS     UP    UP    1  Trunk vmnic3
   48   1b070000    202       0     32           1  VIRT     UP    UP    4 Access vmk0
   49   1b070010   1550       0     32           1  VIRT     UP    UP    4 Access vswif0
   50   1b070020    300       0     32           2  VIRT     UP    UP    4 Access Vmwi01 ethernet0
   51   1b070030    300       0     32           3  VIRT     UP    UP    4 Access Vmwi03 ethernet0
   52   1b070040    301       0     32          32  VIRT     UP    UP    4 Access Vmpvs02 ethernet0
   53   1b070050    300       0     32           2  VIRT     UP    UP    4 Access Xenfil01 ethernet0
   54   1b070060    300       0     32           3  VIRT     UP    UP    4 Access Vmddc02 ethernet0
  304   16000009   3004 T     0     32          32  VIRT     UP    UP    1  Trunk
  305   1600000a   3002 T     0     32          32  VIRT     UP    UP    1  Trunk
N1k-VSM-FortV#

N1k-VSM-FortV# module vem 8 execute vemcmd show vlan
BD 1, vdc 1, vlan 1, 6 ports
Portlist:
     16  vmnic0
     17  vmnic1
     18  vmnic2
     19  vmnic3
    304
    305

BD 102, vdc 1, vlan 102, 3 ports
Portlist:
     18  vmnic2
     19  vmnic3
    304

BD 202, vdc 1, vlan 202, 4 ports
Portlist:
     16  vmnic0
     17  vmnic1
     48  vmk0
    305

BD 203, vdc 1, vlan 203, 3 ports
Portlist:
     18  vmnic2
     19  vmnic3
    304

BD 300, vdc 1, vlan 300, 7 ports
Portlist:
     18  vmnic2
     19  vmnic3
     50  Vmwi01 ethernet0
     51  Vmwi03 ethernet0
     53  Xenfil01 ethernet0
     54  Vmddc02 ethernet0
    304

BD 301, vdc 1, vlan 301, 1 ports
Portlist:
     52  Vmpvs02 ethernet0

BD 1550, vdc 1, vlan 1550, 4 ports
Portlist:
     16  vmnic0
     17  vmnic1
     49  vswif0
    305

BD 1560, vdc 1, vlan 1560, 0 ports
Portlist:
BD 3002, vdc 1, vlan 3002, 3 ports
Portlist:
     16  vmnic0
     17  vmnic1
    305

BD 3004, vdc 1, vlan 3004, 3 ports
Portlist:
     18  vmnic2
     19  vmnic3
    304

BD 3005, vdc 1, vlan 3005, 0 ports
Portlist:
BD 3006, vdc 1, vlan 3006, 0 ports
Portlist:
BD 3040, vdc 1, vlan 3040, 4 ports
Portlist:
     10  l22
     16  vmnic0
     17  vmnic1
    305

BD 3041, vdc 1, vlan 3041, 4 ports
Portlist:
     12  l24
     16  vmnic0
     17  vmnic1
    305

BD 3967, vdc 1, vlan 3967, 2 ports
Portlist:
     14  l26
     15  l27

BD 3968, vdc 1, vlan 3968, 3 ports
Portlist:
      1  inband
      5  inband port security
     11  l23

BD 3969, vdc 1, vlan 3969, 2 ports
Portlist:
      8  l20
      9  l21

N1k-VSM-FortV#

Preview file
3 KB
0 Helpful
Customers Also Viewed These Support Documents

Review Cisco Networking for a $25 gift card