Modifications on HyperFlex Configuration

[TheoThym]

Hello,

I have some questions about modification on a HyperFlex infrastructure.

I have a 3 nodes cluster with VMWare and there is some modifications that i will have to do :

1. change the gateway of the entire infrastructure
2. allow some VMs to contact OOB managment

Some changes in our organisation force us to do the change of gateway, from an external device to a VM on the HyperFlex cluster.

So, is it possible to change the gateway of the cluster ? I saw that i can change the gateway of FI's through console or via the UCSM, and I can change the gateway of each ESX, but is there anything else to do ? On the Storage controller VM for exemple ?

Since the new gateway will be a VM nested in the cluster, I want to be able to contact UCSM, KVM... but these devices are on the default vlan (1)... So can I create this VLAN through the HyperFlex Data Plateform (HX Installer) without generating problems ?

Sorry for my poor english, don't hesitate to ask me if you don't understand.

thank in advance for your answers.

 Regards,

[Kirk J]

Rule #1: don't use vlan 1

Rule #2: don't use vlan 1

Rule #3: don't use vlan 1

While you may be able to manually adjust the default gateway (assuming you aren't trying to change subnet or actual storage controller or ESXi node IPs themselves) nesting the DG inside of hyperflex is potentially going to create a chicken and egg scenario.  Hyperflex and ESXi need to be sync'd with NTP servers for time, and have reachable DNS servers when booting up.

If your esxi and hyperflex nodes can't reach those required resources because the DG is nested in a guestVM hosted on the hyperflex platform, then Hyperflex cluster will not be able to start, guestVMs can't start,, DG can't start, etc,etc,etc,etc

If you were looking at only selectively doing DGs for guestVm level networks, and nothing that impacted the ESXi or hyperflex infrastructure, then I guess that would be ok.

Kirk...

[RedNectar]

Hi @TheoThym ,

[Edit: Just read Kirk's reply. He has good advice!]

This is a little hard to answer without knowing how your HX Cluster is configured, but it seems you are wanting to change the default gateway of the ESXi nodes' management interface.  There is also the question of the default gateway for the UCSM manager and CIMC IPs -which (if I understand you correctly) are currently configured on VLAN 1 via an external L2 switch.

This is also more of a VMware problem than a Hyperflex problem, but we are all here to help each other, so here goes.

Before I tell you how to do this, I would advise you not to do this unless you have considered the following scenarios and are happy you have mitigated any risk

• If the default gateway is on a guest VM on one of your ESXi (Hyperflex) nodes, what happens if the default gateway VM or the hosting ESXi host fails?
• Is HA (High Availability) enabled on your ESXi cluster so that a new instance of the VM will be deployed thus restoring connectivity? This of course will be impossible for vCenter to acomplish if vCenter is on a different subnet to the ESXi hosts, so the next question is
  • Is vCenter on the same subnet as the ESXi management IPs? (If not, you probably should NOT PROCEEED)
  • Do you have a management PC that is on the same subnet so that you will still have access to vCenter and KVM of the ESXi hosts in the event of a failure of the GW VM or ESXi host?
  • Have you considered how you are going to recover if say HA does NOT work as expected?
  • Have you considered running two default GW VMs (on two different ESXi hosts) simultaneously in a VRRP configuration?

So if you have considered all those points, here is what you can do.

Firstly, I'm going to assume that the default gateway IP for the UCS Manager (Fabric Interconnects) and CIMCs is ALSO going to be defined on this Default Gateway VM you are going to deploy.

Step 1: Configure VLAN 1 on the Fabric Interconnects, the vNIC profiles and the vm-network vSwitches

This is achived by running the following command from the installer VM

post_install --vlan

Step 2: Configure northbound switches for VLAN 1

Make sure that VLAN 1 on the northbound switches that the Fabric Interconnects

1. Has VLAN permitted on the trunk ports leading to the Fabric Interconnects 
2. The trunk ports leading to the Fabric Interconnects are NOT using VLAN as the native VLAN - in other words configure some other VLAN as the native VLAN on those ports.
3. The northbound switches are connected to the UCSM/CIMC switch via VLAN 1

Step 3: Configure your Gateway VM

Your gateway VM will need at least two NICs. In vCenter, assign one NIC to the portgroup you assigned to VLAN 1in Step 1, and configure the other to the hx-mgmt-vswitch (assuming you kept the default names suring the install)

I hope this helps

---

Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

---

[TheoThym]

Thanks a lot for your answers !

I'd like to say that i'd love not to use the VLAN 1, but is the VLAN applied to the FIs is fixed by the native VLAN configuration on the switch or is it linked to the default vlan configuration inside UCSM ?

Sorry for the dumb questions, I'm not really familiar with all the HyperFlex configuration.

If it's juste with the native VLAN on the swith, I won't use the VLAN 1, I'll be happy, every one will be happy :)

Regarding my gateway, I'm aware that it's not recomended, but I don't have a choice (it will be a temporary solution, we will get an external device in the future).

We have 2 guest VM in cluster(a-p) that will act as the default gateway, HA enable, vCenter on the same subnet.

I wanted to change the default gateway of my infrastructure smoothly one thing at a time but I may juste take the IP of my current gateway and apply it to the guestVM that will act as the gateway afterwards if it's to much trouble to modify on FI, ESX and others...