This connectivity was something that was in place prior to me. It was relayed to me that this was designed and installed by cisco and at the time was a Cisco "Best Practice" for the architecture in place. Not what the core architecture has turned into or as is currently stands. The 5K swap is part of that conversion. Hence the need to pull the old 5548P and replace it with a new 5548UP.
I will agree that the UCS connectivity does not meet current best practices based on the current Nexus designs. However let that not distract you.
From a Nexus standpoint if you lose a switch that should not stop the UCS from seeing the fabric drop and repining the VNICs to the other fabric. Then the fabric would send a GrARP to the switch that is still up and rehome those addresses to that switch.
Don’t confuse this with losing the peer link as that would indeed cause the same type of issue. Only the vPC role (secondary) switch would shut down vPC’s as well as all the L3 SVI's. The physical interfaces would remain up and the UCS would not see a down status. So it would keep trying to use the fabric to that Nexus. Essentially you would create a black hole for traffic leaving that fabric to the Nexus where it would have no L3 route to the next hop and just drop the traffic.
