Manual Nexus 9k software update over the network

[Johannes Luther]

Hi board,

Let's assume a multi-pod environment and you cannot register your new spine switch in a remote pod to the network due to a SW bug or the SW does not support multi pod in the current version on the remote spine switch. And no - there's no APIC in the remote location :)

Challenge is to manually upgrade the remote spine switch over the management network (out-of-band / management0 interface).

I found the technote, which describes the process in very detail.

However, one aspect is missing: How to configure an IP address on the unititialized spine switch over the CLI (console) to copy the image to the spine filesystem? Anybody has an idea?

I tried Linux "ifconfig" on the eth0 interface (which is the mgmt0 port on the chassis). The command was accepted and the CLI output for ifconfig looked good. However a test ping resulted in strange error messages (hardware error etc.) - guess this is not the correct way :)

[Yannick Vranckx]

Hello,

Maybe it's better to revert the 9K Spine back to NX-OS and doing it old style. Configuring an IP via CLI and sending the file like that?

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/upgrade/guide/b_Cisco_Nexus_9000_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide_Release_7x/Converting_from_Cisco_NX_OS_to_ACI_Boot_Mode.pdf

[Johannes Luther]

Hi,

interesting point. One question regarding the conversion:

There is the statement in the bootloader to load a specific NX-OS image ("nxos-image-name").

loader> boot nxos-image-name

I guess in an ACI image delivered switch, there is no NX-OS image on the disk.

At least if I check the bootflash directory of my N9k, there is no NX-OS image.

So the original question remains: How to I copy an image (ACI or NX-OS) remotely (not USB) to an fabric uninitialized N9k switch?

[Yannick Vranckx]

Usually there is an NX-OS image and Cisco ACI image on the device, they should be both on that. Because you've paid for a switch in NX-OS mode, and you pay for the extra ACI License with image.

Have you checked the flash? To see wether the image is there on the flash?

If not, the fastest way is to get USB access to the device and copy the image from there. I think there is also a way to configure a TFTP server in 'Rommon', although I'm not sure anymore if it exists in Cisco Nexus. I can't find any documentation about it.

[Johannes Luther]

---

@Yannick Vranckx wrote:

Usually there is an NX-OS image and Cisco ACI image on the device, they should be both on that. Because you've paid for a switch in NX-OS mode, and you pay for the extra ACI License with image.

Nope... I (and I guess most of the other ACI users) order the switch with preinstalled ACI SW. During ordering (check CCW) you either chose an NX-OS or ACI software image option for $0 (USD). Normally if you deploy ACI you want a preinstalled ACI image, because there won't be a "plug and play" experience if the switch is delivered with a NX-OS image.

As a matter of fact, sometimes an ordered HW is ordered incorrectly (wrong ACI image selected) or the hardware is in stock for a long time (e.g. cold standby) with a pretty old image.

---

@Yannick Vranckx wrote:
Have you checked the flash? To see wether the image is there on the flash?

I did. However I'm not quite sure which directory.

I checked the /recovery and the /bootflash directory - both only contain an ACI image.

[Yannick Vranckx]

That's unfortunate to be honest, when we got our switches delivered it had both images on it.

But I can confirm that the images are sometimes really outdated.

If you can't get an IP on the device in ACI mode, you'll need physical access to do something and since your switch is not getting an IP assigned by the APIC due to bad firmware.

[Johannes Luther · Accepted solution]

I created a TAC case and got the answer now:

1.) Reboot ACI switch

2.) During reboot - at the very beginning ("...initializing fan controller") hit CTRL-C to enter the loader

3.) In the loader set the IP and DG for the mgmt interface

set ip <IPv4-ADDR> <NETMASK>
set gw <IPv4-ADDR-GW>

Note: It's not possible to ping the IP address. Even ARP resolution does not work for the IP. Communication is only possible if initialed by the ACI switch

4.) Boot from a remote location via TFTP.

boot  tftp://<IPv4-TFTP-SERVER>/[<PATH>/]<IMAGE-FILE>.bin

After reboot the image is hopefull in the /bootflash folder

Note: Only TFTP is possible. So bring along some games or stuff to pass the time to download the large images over a long fat link :)