I Have been testing some failover scenarios with 4 nexus 7000 switches with an M2 and an F2 card in each. Each Nexus has two supervisor modules.
I have 3 VDC's Admin, F2 and M2
all ports in the M2 are in the M2 VDC and all ports on the F2 are in the F2 VDC.
All vPC's are connected on the M2 cards, configured in the M2 VDC
We have 2 Nexus representing each "site"
In one site we have a vPC domain "100"
The vPC Peer link is connected on ports E1/3 and E1/4 in Port channel 100
The peer-keepalive is configured to use the management ports. This is patched in both Sups into our 3750s. (this is will eventually be on a management out of band switch)
Please see the diagram.
There are 2 vPC's 1&2 connected at each site which represent the virtual port channels that connect back to a pair of 3750X's (the layer 2 switch icons in the diagram.)
There is also the third vPC that connects the 4 Nexus's together. (po172)
We are stretching vlan 900 across the "sites" and would like to keep spanning tree out of this as much as we can, and minimise outages based on link failures, module failures, switch failures, sup failures etc..
ONLY the management vlan (100,101) is allowed on the port-channel between the 3750's, so vlan 900 spanning tree shouldnt have to make this decision.
We are only concerned about layer two for this part of the testing.
As we are connecting the vPC peer link to only one module in each switch (a sinlge) M2 we have configured object tracking as follows:
n7k-1(config)#track 1 interface ethernet 1/1 line-protocol
n7k-1(config)#track 2 interface ethernet 1/2 line-protocol
n7k-1(config)#track 5 interface ethernet 1/5 line-protocol
track 101 list boolean OR
n7k-1(config-track)# object 1
n7k-1(config-track)# object 2
n7k-1(config-track)# object 5
n7k-1(config-track)# end
n7k-1(config)# vpc domain 101
n7k-1(config-vpc-domain)# track 101
The other site is the same, just 100 instead of 101.
We are not tracking port channel 101, not the member interfaces of this port channel as this is the peer link and apparently tracking upstream interfaces and the peer link is only necessary when you have ONE link and one module per switch.
As the interfaces we are tracking are member ports of a vPC, is this a chicken and egg scenario when seeing if these 3 interfaces are up? or is line-protocol purely layer 1 - so that the vPC isnt downing these member ports at layer 2 when it sees a local vPC domain failure, so that the track fails?
I see most people are monitoring upstream layer3 ports that connect back to a core? what about what we are doing monitoring upstream(the 3750's) & downstream layer2 (the other site) - that are part of the very vPC we are trying to protect?
We wanted all 3 of these to be down, for example if the local M2 card failed, the keepalive would send the message to the remote peer to take over.
What are the best practices here? Which objects should we be tracking? Should we also track the perr-link Port channel101?
We saw minimal outages using this design. when reloading the M2 modules, usually 1 -3 pings lost between the laptops in the diff sites across the stretched vlan. Obviously no outages when breaking any link in a vPC
Any wisdom would be greatly appreciated.
Nick
