Hi, I am having some issues configuring new Cisco switches for our datacenter infrastructure. I am fairly new to NX-OS, and am far from a network engineer, but I do have experience with IOS. I would love some ideas, if anyone has them!
We are migrating from some old Dell PowerConnect switches to Cisco Nexus 9300. Most of our routing is done on the firewall, so we use the switches primarily in layer 2 mode. However, we do have a few vLANs that need to pass through the switches to our vSAN infrastructure.
Currently, our PowerConnect switches are set up in a stack. I understand that the Nexus switches do not have a traditional stack like some of Cisco’s other switches do (SX550X for example). Currently, we have a 5 node vSAN cluster. Each host has 2 NICs, each with 2 ports. Each host therefore will be plugged into each switch twice to allow for redundancy. We also have two firewalls, with each one plugged into each switch, also for redundancy. I have included a document that gives a brief overview of our infrastructure.
I have been into our datacenter twice now to configure these switches for migration. Each time, I have failed, and barely gotten anywhere. I was able to successfully set up a vPC and port channel for the firewalls, as well as the failover between switches. My issue is with the vSAN connectivity. We have to pass multiple vLANs through for our different environments. They are as follows:
1 – a
10 – b
100 – c
101 – d
102 – e
103 – f
200 – iSCSI
300 – h
500 – vSAN1
501 – vSAN2
I have configured the vSAN port as follows:
interface Ethernet1/7
description vsan5-vsan
switchport mode trunk
switchport trunk allowed vlan 10,100-103,200,500-501
However, even though I am able to ping the switch just fine, the other vSAN hosts do not see this particular one. It appears that some of the vLANs are not passing through as I get errors saying the hosts are not able to connect on certain subnets. I did have a couple of weird/interesting events during my latest testing. Running a constant ping to the vSAN host, I had a successful connection on one switch. I then plugged the second connection to the second switch but lost all connectivity to the host itself. Even after unplugging the latter cable, the connection would not be restored. I would have to completely reset the switch and start over to reconnect.
Another issue I had, which seems even weirder was also while running a constant ping, and having a successful connection to vSAN host, I plugged my computer directly into a switch port to make sure I could access said host from the LAN. It was at that point that I again lost all connectivity to that host. Even after removing my computer connection, I received 100% packet loss.
I am at a loss because I feel like what I am doing should work. Again, we do not run anything special on our current switches. The only difference I can tell is that the Dell PowerConnect switches allow for a general port mode in addition to trunk and access modes, whereas the Nexus switches only allow the latter two.
