iSCSI Boot with Multiple Fabrics

[vnick]

Hello, all,

I'm working with an older UCS Blade configuration - FI 6332s and some M5 blades - and I'm trying to get them properly configured to boot over iSCSI. I have it working - I've configured the vNICs, iSCSI HBAs, and boot policy, the blades do the iSCSI login and boot from the SAN volume presented, etc.

I'm running into one issue with the configuration that I could use some help with, specifically regarding multiple iSCSI HBAs. I've got two iSCSI HBAs configured on the service profile, one uses an overlay vNIC from Fabric A, the other uses an overlay vNIC from Fabric B. In the boot policy, I configure the one from Fabric A to boot, first, and then the one from Fabric B. The only trouble is that, as soon as it successfully connects to the target on the first one, it doesn't initialize and connect the second one. Since I'm using iBFT in my O/S, this means that, when the blade boots, all of the paths to the iSCSI target/boot volume are on one Fabric rather than being spread across both. And, of course, this means that, if I ever have to reboot the FI on the side where everything is booted, I'm going to lose all of the blades, or, at the very least, have to reboot all of them on the other Fabric before rebooting the FI - not to mention if one goes down unexpectedly, I'll lose all the blade.

Any hints on how to get the blade platform to initialize and log in on both iSCSI HBAs before actually selecting the boot device?

[Kirk J]

Checking https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/CLI-User-Guides/Server-Mgmt/4-3/b_cisco_ucs_manager_server_mgmt_cli_guide_4_3/m_cli_configuring_server_boot.html#concept_wcw_13w_d1b   ...

iSCSI Boot Process

Cisco UCS Manager uses the iSCSI vNIC and iSCSI boot information created for the service profile in the association process to program the adapter, located on the server. After the adapter is programmed, the server reboots with the latest service profile values. After the power on self-test (POST), the adapter attempts to initialize using these service profile values. If the adapter can use the values and log in to its specified target, the adapter initializes and posts an iSCSI Boot Firmware Table (iBFT) to the host memory and a valid bootable LUN to the system BIOS. The iBFT that is posted to the host memory contains the initiator and target configuration that is programmed on the primary iSCSI VNIC.

I read this to mean, by design, it is only expecting to have one active boot iSCSI NIC.
After boot completes, your OS multipathing module should take care of failing over to available alternate paths if needed for any of the LUNs including boot LUN.

I am not clear on what would happen if your Primary boot iSCSI NIC path goes down part way through boot... Would that require a reboot, or would it fail over to the 2ndary iSCSI boot NIC during boot.  I'm thinking it would require a reboot, which would re-evaluate the A/B iSCSI vnic target availability, and choose the 2ndary if primary is not available.

[Edit]  Confirmed that Primary iSCSI Boot vnic path loss during boot, would require a reboot.  Path availability would then be re-evaluated, with Primary being preferred, but 2ndary used if Primary detected as not available.

Kirk...