Creating HX cluster with compute-only nodes

[Markus5849]

I'm going to create a new standard HX cluster with 3 converged and 3 compute-only nodes. May I know is it possible for me to choose also the compute-only nodes during cluster creation? Or I have to expand the HX cluster after creation with compute-only nodes?

[Kirk J]

Initial cluster creation is always converged only nodes.  You will need a subsequent expansion to add the compute nodes.

Kirk...

[Markus5849]

I see, noted on this. Another question for the VMware ESXi host installation. Currently the compute-only nodes only have 2 local SSD without any RAID and OS installation. 

As I checked from the knowledge article I'm thinking of below steps:
1. Create service profile from the HX Service Profile Template
2. Edit the Local Disk Configuration Policy
3. Install ESXi host after service profile associated
4. Run Cluster HXDP Expansion Workflow

For the Local Disk Configuration Policy, may I know should I configure it for 2 disks to be RAID 1 or leave it as slot 1 for OS, slot 2 for Cache?

[Steven Tardy]

From compute only nodes (which don't have ESXi installed). . .

I would:

1. Run the HX installer to let the installer create the UCS Service profile and associate the SP the compute-only servers.
2. The HX installer will stop/fail at something like "can't access ESXi via SoL".
3. At this point open the KVM and manually install HX ESXi (from cisco.com) on the compute-only nodes.
4. [Retry] on the HX installer.

I'm assuming you have the M.2 boot drive, but I guess compute nodes could come with front facing / external SSDs.
There are two controller modules for M.2 boot on UCS:

• UCS-MSTOR-M2 If you have this one, then only one M.2 will be used. The second disk will go unused.
• UCS-M2-HWRAID If you have this one, then the two M.2 disks will create a RAID1/mirror. HX should detect this and configure the UCS boot policy accordingly.
  

In Hyperflex the compute only nodes don't do any local disk caching.

[Markus5849]

Hi Steven,

As checked, my compute-only node doesn't have any M.2 disks and above mentioned controllers are not present in the server. I can only see 2 local SSD and a Cisco 12G Modular RAID Controller. So in this case I believe the OS installation shall be on the local SSD but how about the RAID? Will the HX detects it and configure automatically or I have to do it manually?

[Steven Tardy]

Hyperflex installer should auto-detect the disks and add them to a RAID1/mirror.

https://www.cisco.com/c/en/us/td/docs/hyperconverged_systems/HyperFlex_HX_DataPlatformSoftware/AdminGuide/4-5/b-hxdp-admin-guide-4-5/m_expand_the_cisco_hx_cluster.html

Prerequisites for Adding a Compute-Only Node
..

Compute-only nodes are deployed with automatic detection and configuration of disk and boot policies based on the 
boot hardware.

Starting with HX Data Platform release 4.5(1a) and later, compute-only nodes are deployed with automatic detection 
and configuration of disk and boot policies based on the inventoried boot hardware. Users cannot directly select 
the UCSM policies. Instead, the boot device is automatically determined based on the first acceptable boot media 
discovered in the server. The tables below show the priority order for M4/M5 generation servers. Reading from top 
to bottom, the first entry that is a match based on the inventoried hardware are selected automatically during 
cluster expansion. For example, when expanding with a B200 compute node with a single M.2 boot SSD, the second 
rule in the table below is a match and used for SPT association. 

On that same page, your configuration should select: compute-nodes-m6-ldr1

[Markus5849]

Hi Steven,

Thanks for the information. I'll try with your provided steps and update you again on the outcome.

1. Run the HX installer to let the installer create the UCS Service profile and associate the SP the compute-only servers.
2. The HX installer will stop/fail at something like "can't access ESXi via SoL".
3. At this point open the KVM and manually install HX ESXi (from cisco.com) on the compute-only nodes.
4. [Retry] on the HX installer.

Thanks a lot

[Markus5849]

Hi Steven,

I'm trying to read the article and found below steps:

It mentions there will be an automatic association but how? I have the HXDP installer OVA deployed and may I know how should I achieve this? If I click on the standard workflow for cluster expansion, it won't allow me to select Compute-Only node. I've to click on the "I know what I'm doing" for adding compute-only node. But from there, if I tick ESXi configuration & Deploy HX Software, it won't allow me to select the compute-only node. So after I deployed the HXDP installer, what is the next action like which should I click into for having the service profile automatic association as I'm a bit confuse in this.

My current compute-only nodes have no ESXi installed and I have a lot of compute-only nodes and each of them need to be added to three different HX cluster.

[Steven Tardy]

Using HX 4.5(2d) installer in my lab. . .

1) Open Installer and [Expand Cluster] / [Standard Cluster]

2) Fill out UCSM / vCenter details

3) Choose cluster to expand [HX-01a] and enter credentials

4) Choose what (unassociated) (compute-only) servers to add to the cluster

5) [Continue] if CIMC IP Pools have enough IPs.

6) Give ESXi an management IP

7) Give ESXi a data/storage IP

Warning will show detailing boot configuration changes needing to be done

Will update more once the installer gets further along and the forums eats all of my screenshots.

[Steven Tardy]

HX installer queried UCSM and found this server had a 32GB FlexFlash (SD card).

Don't have proper DNS setup in the lab 'cuz "Cisco".

9) Eventually saw this server had old ESXi 6.5 on it instead of ESXi 7.0u3

10) Mount ESXi .iso via KVM vMedia / [Activate Virtual Devices] / [CD/DVD] / [Browse] / HX.iso / [Map Drive] / [Reset]

11) Install ESXi [continue] / [SD Cards/M.2] username/password = erase/erase

Twiddle thumbs while the installer .iso puts ESXi bits on the local disks.

[Steven Tardy]

ESXi install completed.

12) HX installer was still running in the background and applied the static IP

13) HX installer continued and complained about lack of A/PTR records (again)

14) Hit an issue where my lab cluster is in a non-typical configuration preventing the [Deploy] step from completing.

What step is your install failing?

[Markus5849]

Hi Steven,

I failed on the first step which is after selecting compute-only nodes from the standard cluster expansion workflow. My compute-only node is UCS C220 M5SX and with two local disks. Why would this error appear?

[KyoCode]

Morning Markus, 

it looks like it is taking the new server as a converged node instead of a compute-only node, that's the kind of error I've seen before, since Disk type is irrelevant for a compute-only node.

trying to find some relevant info to compare to.

AIV

[Markus5849]

Hi KyoCode,

May I know do you have any solution for this since you have faced it before?

Appreciate that.

[KyoCode]

quick question (maybe irrelevant), is your current browser in a different language than english? there's a defect in which workflow will complain about browser language CSCwe45153

AIV