Re: Nexus 9300 upgrade

KRISTOF SLACHMUYLDERS · ‎03-09-2020

Hi,

We're planning a software upgrade for the Nexus 9300 (93180YC-EX) switches in our 2 data centers. However, I'm having a hard time to predict downtime and/or data plane behaviour.

We're using a straight-through FEX design with N2K devices. Both vPC domains at each data center are connected back 2 back using a layer 2 DCI running over dark fiber. (4 links in total over 2 fibers). I've attached our topology to this topic for clarity. Peer keep-alive links are configured on the the dedicated mgmt interfaces of both N9K switches.

Show vpc brief output:

Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 2
Peer status                       : peer adjacency formed ok
vPC keep-alive status             : peer is alive
Configuration consistency status  : success
Per-vlan consistency status       : success
Type-2 consistency status         : success
vPC role                          : primary, operational secondary
Number of vPCs configured         : 12
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled, timer is off.(timeout = 240s)
Delay-restore status              : Timer is off.(timeout = 30s)
Delay-restore SVI status          : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router    : Disabled

My question is:
Our data centers are in an active-passive setup. Only one data center is actively routing(firewalls, routers) or load-balancing.
We've got several firewalls and/or routers however which aren't dual-homed at 1 data-center, however via dark fiber, a HA setup is achieved using either HA interfaces for firewalls (dual nics) or FHRP for upstream routers (ISP, data center lease lines,...)

What will happen if I start the upgrade at the passive site (DC2) on the primary vpc role switch? Will any vlans be suspended or how long will downtime last?

As I see it, it should be OK to upgrade the first Nexus at the passive site (but disable the DCI links to the other site first to avoid err-disabling them) Then, when upgrade is finished upgrade the other one and finally, the same for the active data center. Is this somewhat correct? Do I need to take into account other things?

Our currently installed release is 7.0(3)I5(2)
The current preferred nxos release is 7.0(3)I7(7)

Thank you very much for your feedback!

Kind Regards,
Kristof

tcmckay · ‎03-11-2020

I just upgraded my N9300 this past weekend. My process is to upgrade the "primary" role switch first then the "secondary" switch. I was upgrading from nxos.9.2.3 to nxos.9.3.3. The upgrade and reload of the switch took 15min and the single attached fex took another 10 min. During the upgrade the primary role is relinquished and all vlan traffic is paused on the the peerlink. Communication still travels through the peer-keepalive link to keep databases consistent. When the primary switch comes back on line it will take priority if configured or you can force the return to primary. Here is the configuration I use on both switches

vpc domain 2
peer-switch
role priority 1000
peer-keepalive destination 10.200.50.12 source 10.200.50.11
delay restore 150
peer-gateway
auto-recovery
ip arp synchronize

vpc domain 2
peer-switch
role priority 2000
peer-keepalive destination 10.200.50.11 source 10.200.50.12
delay restore 150
peer-gateway
auto-recovery
ip arp synchronize

Although I am not upgrades with a connection to a remote site I did the upgrade while attached to 2 other switch pairs that are configured in back-to-back vpc's. Connectivity through the entire environment was maintained during the upgrade. I have done this with 2 or the 3 switch pairs. All work was done without connectivity loss.