Re: Ask the Expert: Nexus Virtual Port Channel (vPC) - Page 3

ciscomoderator · ‎08-26-2011

With Hatim Badr and Iqbal Syed

Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn about design, configuration and troubleshooting of VPC with Cisco experts Hatim Badr and Iqbal Syed. Iqbal is a product manager and technical marketing engineer for the Cisco Nexus 7000 Series of switches. He is responsible for product road-mapping and marketing the Nexus 7000 line of products with a focus on virtual port channel design and training. Syed has been with Cisco for more than 8 years, which includes experience in Cisco Advanced Services and the Cisco Technical Assistance Center. His experience ranges from reactive technical support to proactive engineering, design, and optimization. He holds CCIE (Routing & Switching), CCDP, Cisco Data Center, and TOGAF (v9) certifications. Hatim is a network consulting engineer for Cisco Advanced Services in Toronto, where he supports Cisco customers across Canada as a specialist in data center architecture, design, and optimization projects. He has more than 10 years of experience in the networking industry. He holds CCIE certification #14847 in Routing and Switching and also holds TOGAF 9, VCPv4, and PMP certifications.

Remember to use the rating system to let Hatim and Iqbal know if you have received an adequate response.

Hatim and Iqbal might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the discussion forum shortly after the event. This event lasts through September 9, 2011. Visit this forum often to view responses to your questions and the questions of other community members.

habadr · ‎08-31-2011

Hi Adriaan,

Thanks for highlighting this case. In fact this is documented in defect CSCtl70133 and you will have packet loss if peer-keepalive followed by peerlink fail in vPC domain that does not contain the STP root which is N5K in your scenario.

It is also documented in the test scenarios in site to site vpc-vpc test results. (table 2-4 page 2-32).

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns949/ns304/ns975/data_center_interconnect_design_guide.pdf

Test Case	Test Details	Failure Ucast	Failure Mcast	Restore Ucast	Restore Mcast	Result
DC1-DCI-7K vPC Peer Keepalive Link and Peer Link Failure	Physically disconnected the cables connected to DC1-DCI-7K1's peer keepalive interface, Management 0, and entire vPC Peer Link, Ethernet 1/9 and 2/9 for failure. Links reconnected for restoration.	sustained loss	sustained loss	N/A	N/A	Fail

By failing all vPC peer link members plus the vPC peer keepalive link simultaneously, a vPC dual active condition is forced. If this scenario happens on a vPC domain that does not contain the STP root, the STP dispute mechanism inadvertently blocks links that cause intermittent traffic drop. While this is considered an unlikely scenario (triple failure within <3 seconds).

Please note that if peer-link fails before peer-keepalive then you should not have this issue since secondary vPC peer switch will shut down all vPC ports.

Thanks

Hatim Badr

Changing the format to show the table properly

dtran · ‎09-02-2011

Hi all,

I am looking into deploying a pair of N7010's running vPC and a few N2K's connecting to N7010's via port-channel (Please see drawing below). Most of my servers today are single homed to either N2K or directly to the N7K. My question is is there any issue/concern if I configure the vPC peer link to allow all VLANs ? in what scenario would you configure vPC vlan and none vPC vlan ?

Thanks !!!

D.

isyed · ‎09-02-2011

**Re-posting with minor edit**

Hi There,

To answer your question , by definition any vlan that is forwarded on the vpc peer link beocmes a vpc vlan ...If the devices are single homed ( i.e connected to only one peer either directly or via N2K) , then the question should be do you really want those vlans to be forwarded on the peer link and by doing so extend your L2 domain across the two peers ....because you can easily configure the vlan only on the peer where the device is connected to and use an SVI for further connectivity.That way there will be no need to extend these vlans across on the peer link essentially making those vlans 'non-vpc' vlans.

As for the Non-VPC vlan - any vlan which is not forwarded on the peer link is a non-vpc vlan and follows the regular STP rules , A lot of times customers use it for single homed devices or for the devices which arent capable of running etherchannel.

Please also note that single attached devices that are not connected via a vPC ( including single homed ) but still carry vPC VLANs are also known as orphan ports.In case of a peer-link shut or restoration, an orphan port's connectivity may be bound to the vPC failure or restoration process

Hope it is clear now.

Regards,

Iqbal

isyed · ‎09-02-2011

Also to add our best practice recommendation would be to dual connect all devices to both the peers because in case of failure scenarios , the single homed devices would be isolated and traffic from them would be blackholed.

dtran · ‎09-02-2011

Hi Isyed !!! I appreciate your help !!

In my scenario I will have servers on the same VLAN connecting across both N7Ks, so I am extending L2 domain across both N7K's. Do you see any issues / concerns allowing all VLANs across the vPC peer-link ? I would like to get your inputs on this !!!!

I agree that all servers should be dual homed and that's what I am planning to do moving forward. That's why I decided to use vPC between the N7K's.

Thanks !!

Danny

isyed · ‎09-06-2011

Hi Danny,

As mentioned earlier - It is strongly recommended to dual attach every device to vPC domain to avoid the isolation of the orphan ports in case of peer link failure.

I would suggest you to consider the following 3 design options before opting the option 4 which is the orphan port design you are currently considering. I have also listed down the pros and cons of each of the design options to help with your evaluation.

If after evaluating the first three options, option 4 still seems the only feasible option for you, then I would encourage you to reconsider the vpc implementation at this point since by single homing the devices and using vpc , you are not really gaining anything anyway. It would be best for you to configure vpc when you are ready to go dual homed (dual connect ) with your devices.

Hope that helps.

Regards,

Iqbal

********************

Here are the recommendations for connecting devices to vPC domain (in order of preference)

•1. ALWAYS try to dual attach devices using vPC (not applicable for routed links).

PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Ensures full redundant active/active paths through vPC.

CONS: None

•2. If (1) is not an option – connect the device via a vPC attached access switch (could use VDC to create a “virtual access switch”).

PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Availability limited by the access switch failure.

CONS: Need for an additional access switch or need to use one of the available VDCs. Additional administrative burden to configure/manage the physical/Virtual Device

•3. If (2) is not an option – connect device directly to (primary) vPC peer in a non-vPC VLAN and provide for a separate interconnecting port-channel between the two vPC peers.

PROS: Traffic diverted on a secondary path in case of peer-link failover

CONS: Need to configure and manage additional ports (i.e. port-channel) between the Nexus 7000 devices.

•4. If (3) is not an option – connect device directly to (primary) vPC peer in a vPC VLAN

PROS: Easy deployment

CONS: Generally Bad. Bound to vPC roles, Full Isolation on peer-link failure when attached vPC toggles to a secondary vPC role.

yue cheng · ‎09-07-2011

Hi Hatim and Iqbal

I have few questions about vpc

1)Do vpc peers use the mac add 00:23:04:ee:be:xx as their des mac to communicate with eachother,like the ospf use the multicast add 224.0.0.5 and 224.0.0.6？and at the scenario of double-side vpc，device must use the unique vpc domain id，if not ，they will get confuse with who is the primary vpc.

2)I confuse with the vpc role and the operational role，what is their responsibilities？and I use the show vpc role command at nexus 5000 found that there is olny vpc role.

3)The default vpc priority is 1024 ?

habadr · ‎09-07-2011

Hi Yue

Thanks for your questions please find my answers inline

Q1)Do vpc peers use the mac add 00:23:04:ee:be:xx as their des mac to communicate with eachother,like the ospf use the multicast add 224.0.0.5 and 224.0.0.6？and at the scenario of double-side vpc，device must use the unique vpc domain id，if not ，they will get confuse with who is the primary vpc.

A1) As you know this mac address is vpc system MAC address and derived from domain ID. The vPC peer devices use the vPC domain ID to automatically assign a unique vPC system MAC address . You MUST use utilize unique Domain id’s for all vPC pairs defined in a contiguous layer 2 domain.

vPC Peers use this MAC address as source MAC address in the following two cases

1- LACP neighbor needs to see the same System ID from both vPC peer. The LACP system ID is the combination of the LACP system priority value and the MAC address of the router. The vPC ‘system-mac’ is used by both vPC peers in LACP system ID to appear as single device to its neighbors.

2- When using peer-switch feature. Beginning with Cisco NX-OS Release 5.0.2a peer-switch feature is introduced to help both switches appear as single STP bridge and sending BPDU with same bridge ID, which is the vpc system MAC address, to ensure that the downstream device does not detect a spanning-tree misconfiguration.

Q 2)I confuse with the vpc role and the operational role，what is their responsibilities？and I use the show vpc role command at nexus 5000 found that there is olny vpc role.

A2) The vPC Primary is manually defined by the role priority. The switch with lower priority will be elected as the vPC primary switch. and in normal operation the primacy vpc switch is the “Operational primary” and secondary as “Operational Secondary” switch but it will show only as primary or secondary in the show vpc output as shown below

switch1# sh vpc

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 2

Peer status : peer adjacency formed ok

vPC keep-alive status : peer is alive

Configuration consistency status: success

Per-vlan consistency status : success

Type-2 consistency status : success

vPC role : primary

Number of vPCs configured : 3

Peer Gateway : Disabled

Dual-active excluded VLANs : -

Graceful Consistency Check : Disabled (due to peer configuration)

switch2# sh vpc

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 2

Peer status : peer adjacency formed ok

vPC keep-alive status : peer is alive

Configuration consistency status: success

Type-2 consistency status : success

vPC role : secondary

Number of vPCs configured : 3

Peer Gateway : Disabled

Dual-active excluded VLANs : -

However in case of primary switch failure the secondary switch will take over as “Operational primary” although it is configured as secondary and that is what you will see

switch2# sh vpc

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 2

Peer status : peer adjacency formed ok

vPC keep-alive status : peer is alive

Configuration consistency status: success

Type-2 consistency status : success

vPC role : secondary, operational primary

Number of vPCs configured : 3

Peer Gateway : Disabled

Dual-active excluded VLANs : -

And when the primary switch comes online it will appear as “operational secondary” although it is configured as primary

Switch1# sh vpc

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 2

Peer status : peer link is down

vPC keep-alive status : peer is alive

Configuration consistency status: success

Per-vlan consistency status : success

Type-2 consistency status : success

vPC role : primary, operational secondary

Number of vPCs configured : 3

Peer Gateway : Disabled

Dual-active excluded VLANs : -

Graceful Consistency Check : Disabled (due to peer configuration)

So The role is nonpreemptive, so a device may be operationally primary but secondary from a configuration perspective.

Q3)The default vpc priority is 1024 ?

A3) I think you are referring to vPC role priority. The default is 32667

Thanks

Hatim Badr