I am setting up a ASA HA active passive implimentation with 4 Nexus 5596 on 2 physical locations for a customer.
of the 4 Nexuses, only 1 have SVI's for all VLAN's and is the default gateway for all servers and clients, and no HSRP is configured. (dont ask...)
The other 3 Nexuses actually have SVI's too, but there is no IP's configured on those for some reason.
Nexus 1 and 2 is in location 1 and Nexus 3 and 4 is in location 2.
Nexus 1 and 2 is configured as a vPC pair (vPC1) and nexus 3 and 4 is configured as their own vPC pair (vPC2).
I configured a regular port-channel interface in trunking mode on vPC1 and the same on vPC2.
Nexus1
interface port-channel5
description ASA location 1
switchport mode trunk
spanning-tree port type edge trunk
vpc 5
Nexus2
interface port-channel5
description ASA location 1
switchport mode trunk
spanning-tree port type edge trunk
vpc 5
Nexus3
interface port-channel5
description ASA location 2
switchport mode trunk
spanning-tree port type edge trunk
vpc 5
Nexus4
interface port-channel5
description ASA location 2
switchport mode trunk
spanning-tree port type edge trunk
vpc 5
Did a default port-channel config on both ASA's and hooked them up. All looks good. all interfaces is in (P) mode on the port-channels.
added vlan interface to the port-channel and routed internal subnets to the SVI in the same VLAN. initially things looked good.
was able to ping the SVI on Nexus1 from the primary and secondary ASA. Then strange things started happening.
The issue I am having is that traffic from the primary ASA is dropped when traversing the physical ports connected to the nexus2 switch, without an SVI in this VLAN, to hosts in other subnets with default gateway on Nexus1. This traffic must cross the vPC link in order to get routet to the hosts.
When I shut down the port-channel on Nexus2 so that the port-channel is only active on Nexus1, all is good.
I have read through documentation on vPC and best/good practice and limitaitons. None of these mirror the "not very supported" setup I have to get working.
From what I have read, if I set up HSRP on all 4 switches for ALL SVI's, I'm good to go.
But I need to understand why the packets get dropped in the setup I am facing here. I cant find any documentation pointing out a clear issue with the setup and why its behaving this way.
I'm guessing its because the traffic is using the Burned In MAC of the SVI on Nexus 1 and somehow that is being dropped because it is traversing the vPC peer-link from Nexus 2 to Nexus 1?
But that should only be dropped if it destined for an interface in another vPC port channel. And it is not. The only Port-channels I have going is to the ASA and for the failover link/state interface.
