cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1460
Views
0
Helpful
11
Replies

SG550XG-8F8T - esxi ports up/down (no trunk)

A.Biedermann
Level 1
Level 1

I have two DELL Esxi 7.0b host on my SG550XG-8F8T 16-Port 10G

 

on each host are 2 VMs and each VM is using a own vSwitch which is dedicated to one 10G NIC. 

 

After a short while I can see in the logs that all ports on the switch on the copper with the VM's are go up / down, partially every minute and I have a massiv packet lost to the vm's so that the VM's are unreaceable for the clients.

 

But if I use the 1G NIC of the Dell esxi host and not the 10G NIC on this SG550XG I don't have any trouble. DELL already replaced my complet system board and 10G NICs but still the same behavior. 

 

Looks like there is a wrong configuration of the switch.

 

21474820222021-May-14 07:35:27Warning%STP-W-PORTSTATUS: te1/0/1: STP status Forwarding   
21474820232021-May-14 07:35:23Informational%LINK-I-Up:  te1/0/1   
21474820242021-May-14 07:35:22Warning%LINK-W-Down:  te1/0/1   
21474820252021-May-14 07:08:23Warning%STP-W-PORTSTATUS: te1/0/1: STP status Forwarding   
21474820262021-May-14 07:08:18Informational%LINK-I-Up:  te1/0/1   
21474820272021-May-14 07:08:13Warning%LINK-W-Down:  te1/0/1   
21474820282021-May-14 06:56:55Warning%STP-W-PORTSTATUS: te1/0/1: STP status Forwarding   
21474820292021-May-14 06:56:50Informational%LINK-I-Up:  te1/0/1   
21474820302021-May-14 06:56:50Warning%LINK-W-Down:  te1/0/1   
21474820312021-May-14 03:58:22Warning%STP-W-PORTSTATUS: te1/0/1: STP status Forwarding   
21474820322021-May-14 03:58:17Informational%LINK-I-Up:  te1/0/1   
21474820332021-May-14 03:58:11Warning%LINK-W-Down:  te1/0/1
11 Replies 11

marce1000
VIP
VIP

 

 - What is the port configuration(s) on the SG550XG-8F8T for the esxi connections, are the 10g and 1g same configured (for instance), apparently I also see STP related negotation messages, is it needed. Or perhaps you need something like ...port-fast trunk... (on ios). That being said , do you need trunk on the esxi connections (e.g.) ?

 M.



-- ' 'Good body every evening' ' this sentence was once spotted on a logo at the entrance of a Weight Watchers Club !

I don't need trunk (I don't use different VLANs on my VMs) or LAG ports for the esxi and all ports are configured the same, only the uplink port to my SG350 is configured as trunk. no special configurations. I also tried the configure the speed of my 10G port to a fix 1000 but then the port also goes up / down. I also tired and deactvied autosmart port configuration. but not help too

I'm not a real pro with cisco switches, so I don't know if I need STP messages or not. 

 

 

                                               - Have a look at this document : 

            https://www.cisco.com/c/en/us/support/docs/smb/switches/cisco-350x-series-stackable-managed-switches/smb5102-configuring-stp-interface-settings-on-the-sg350xg-and-sg550x.html

 For the ESXI-ports disable STP (you probably don't need it)  and enable edge.

Also have a look at vmkernel-logs on ESXI , look for events related to networking when your problem happens.

                      https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-832A2618-6B11-4A28-9672-93296DA931D0.html

 

                                                   Besides all that use , latest firmware on the SG

 M.

 

 



-- ' 'Good body every evening' ' this sentence was once spotted on a logo at the entrance of a Weight Watchers Club !

Hi M.

 

I disabled STP and enabled Edge on the esxi ports, but still the same problems.

 

in vmkernel.log I found this

2021-05-16T19:43:10.946Z cpu0:2100126)FSS: 148: Reservation state of LOCKER-5fe5188a-cf2c3a32-d9e8-4cd98f4a2a3f moved from Idle to Ready
2021-05-16T19:43:14.502Z cpu0:2097933)bnxtnet: hwrm_get_phy_cfg:917: [vmnic2 : 0x4520463c8000] AutoGrEEEn active: adv spd msk: 0x40 partner spd mask: 0x40 tx_lpi enabled: 1
2021-05-16T19:43:14.502Z cpu0:2097933)bnxtnet: bnxtnet_display_link:1715: [vmnic2 : 0x4520463c8000] NIC Link is Up, 10000 Mbps full duplex, Flow control: none
2021-05-16T19:43:17.153Z cpu0:2097291)NetqueueBal: 5113: vmnic2: device Up notification, reset logical space needed
2021-05-16T19:43:17.153Z cpu0:2097291)NetPort: 1784: disabled port 0x8800000d
2021-05-16T19:43:17.153Z cpu8:2688391)NetSched: 723: vmnic2-0-tx: worldID = 2688391 exits
2021-05-16T19:43:17.153Z cpu14:2688392)NetSched: 723: vmnic2-1-tx: worldID = 2688392 exits
2021-05-16T19:43:17.153Z cpu5:2688395)NetSched: 723: vmnic2-4-tx: worldID = 2688395 exits
2021-05-16T19:43:17.153Z cpu30:2688396)NetSched: 723: vmnic2-5-tx: worldID = 2688396 exits
2021-05-16T19:43:17.153Z cpu14:2688394)NetSched: 723: vmnic2-3-tx: worldID = 2688394 exits
2021-05-16T19:43:17.153Z cpu21:2688398)NetSched: 723: vmnic2-7-tx: worldID = 2688398 exits
2021-05-16T19:43:17.153Z cpu8:2688393)NetSched: 723: vmnic2-2-tx: worldID = 2688393 exits
2021-05-16T19:43:17.153Z cpu11:2688399)NetSched: 723: vmnic2-8-tx: worldID = 2688399 exits
2021-05-16T19:43:17.153Z cpu5:2688397)NetSched: 723: vmnic2-6-tx: worldID = 2688397 exits
2021-05-16T19:43:17.153Z cpu29:2688402)NetSched: 723: vmnic2-11-tx: worldID = 2688402 exits
2021-05-16T19:43:17.153Z cpu10:2688403)NetSched: 723: vmnic2-12-tx: worldID = 2688403 exits
2021-05-16T19:43:17.153Z cpu18:2688405)NetSched: 723: vmnic2-14-tx: worldID = 2688405 exits
2021-05-16T19:43:17.153Z cpu16:2688411)NetSched: 723: vmnic2-20-tx: worldID = 2688411 exits
2021-05-16T19:43:17.153Z cpu9:2688412)NetSched: 723: vmnic2-21-tx: worldID = 2688412 exits
2021-05-16T19:43:17.153Z cpu15:2688416)NetSched: 723: vmnic2-25-tx: worldID = 2688416 exits
2021-05-16T19:43:17.153Z cpu22:2688417)NetSched: 723: vmnic2-26-tx: worldID = 2688417 exits
2021-05-16T19:43:17.153Z cpu26:2688418)NetSched: 723: vmnic2-27-tx: worldID = 2688418 exits
2021-05-16T19:43:17.153Z cpu17:2688422)NetSched: 723: vmnic2-31-tx: worldID = 2688422 exits
2021-05-16T19:43:17.153Z cpu3:2688410)NetSched: 723: vmnic2-19-tx: worldID = 2688410 exits
2021-05-16T19:43:17.153Z cpu19:2688415)NetSched: 723: vmnic2-24-tx: worldID = 2688415 exits
2021-05-16T19:43:17.153Z cpu0:2097291)Uplink: 12079: enabled port 0x8800000d with mac 4c:d9:8f:3e:83:4a
2021-05-16T19:43:17.153Z cpu2:2688401)NetSched: 723: vmnic2-10-tx: worldID = 2688401 exits
2021-05-16T19:43:17.153Z cpu4:2688414)NetSched: 723: vmnic2-23-tx: worldID = 2688414 exits
2021-05-16T19:43:17.153Z cpu0:2688406)NetSched: 723: vmnic2-15-tx: worldID = 2688406 exits
2021-05-16T19:43:17.153Z cpu7:2688407)NetSched: 723: vmnic2-16-tx: worldID = 2688407 exits
2021-05-16T19:43:17.154Z cpu30:2688413)NetSched: 723: vmnic2-22-tx: worldID = 2688413 exits
2021-05-16T19:43:17.154Z cpu14:2688400)NetSched: 723: vmnic2-9-tx: worldID = 2688400 exits
2021-05-16T19:43:17.154Z cpu21:2688404)NetSched: 723: vmnic2-13-tx: worldID = 2688404 exits
2021-05-16T19:43:17.154Z cpu29:2688408)NetSched: 723: vmnic2-17-tx: worldID = 2688408 exits
2021-05-16T19:43:17.154Z cpu26:2688421)NetSched: 723: vmnic2-30-tx: worldID = 2688421 exits
2021-05-16T19:43:17.154Z cpu8:2688419)NetSched: 723: vmnic2-28-tx: worldID = 2688419 exits
2021-05-16T19:43:17.154Z cpu18:2688409)NetSched: 723: vmnic2-18-tx: worldID = 2688409 exits
2021-05-16T19:43:17.154Z cpu22:2688420)NetSched: 723: vmnic2-29-tx: worldID = 2688420 exits
2021-05-16T19:43:17.246Z cpu1:2097291)Uplink: 537: vmnic2: Driver claims supporting 25 RX queues, and 25 queues are accepted.
2021-05-16T19:43:17.246Z cpu1:2097291)Uplink: 533: vmnic2: Driver claims supporting 31 TX queues, and 31 queues are accepted.
2021-05-16T19:43:17.246Z cpu1:2097291)NetqueueBal: 2980: vmnic2: LRO on RSS enabled
2021-05-16T19:43:17.246Z cpu1:2097291)NetqueueBal: 3134: vmnic2: rxQueueCount=25, rxFiltersPerQueue=8, txQueueCount=31 rxQueuesFeatures=0x1a3
2021-05-16T19:43:17.246Z cpu1:2097291)NetPort: 1784: disabled port 0x8800000d
2021-05-16T19:43:17.246Z cpu9:2718023)NetSched: 723: vmnic2-0-tx: worldID = 2718023 exits
2021-05-16T19:43:17.247Z cpu1:2097291)Uplink: 12079: enabled port 0x8800000d with mac 4c:d9:8f:3e:83:4a

 

 -          Which firmware (version) is being used on the SG and what ESXI version is being used ?

 M.



-- ' 'Good body every evening' ' this sentence was once spotted on a logo at the entrance of a Weight Watchers Club !

SG Firmware Version (Active Image): 2.5.7.85

 

ESXI 7.0 Update 1 DEL-ESXi-701_16850804-A00

 

 - You may want to try the latest ESXI (too) :

           https://kb.vmware.com/s/article/2143832

  Sometimes cumbersome , but 'needed to try the latest version ' for that class of problems, and or engage  with Cisco TAC :

              https://www.cisco.com/c/en/us/support/web/tsd-cisco-small-business-support-center-contacts.html

 M.



-- ' 'Good body every evening' ' this sentence was once spotted on a logo at the entrance of a Weight Watchers Club !

Thank you for your help, I will open a TAC.

 

 - Ok, one thing you may also want to check  the interface-counters for the particular 10g-interfaces on the SG. Look for errors if any.

 M.



-- ' 'Good body every evening' ' this sentence was once spotted on a logo at the entrance of a Weight Watchers Club !

Hi M.

 

I think I found my problem.

In the esxi logs I always read "AutoGrEEEn active: adv spd msk: 0x40 partner spd mask: 0x40 tx_lpi enabled: 1" at the beginning when the problem happens, so I disabled "802.3 Energy Efficient Ethernet (EEE)" and I don't see any port flapping for 2 days.

I know why I hate all energy safing options.

 

Axel

 

                    >...so I disabled "802.3 Energy Efficient Ethernet (EEE)" and I don't see any port flapping 

  - Good to know for some reason then the 1g(dell)/10g(SG) solution was not impacted by this native setting.

 M.

 



-- ' 'Good body every evening' ' this sentence was once spotted on a logo at the entrance of a Weight Watchers Club !
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Switch products supported in this community
Cisco Business Product Family
  • CBS110
  • CBS220
  • CBS250
  • CBS350
Cisco Switching Product Family
  • 110
  • 200
  • 220
  • 250
  • 300
  • 350
  • 350X
  • 550X