N5k Peer keepalive path

[minghui.qi]

Hi all,

A simple setup, 2 x Nexus 5548 connected by the peer link, 2 x 6500 in VSS. Each 5548 has two 10G uplinks to each of the 6500 in VSS.

My understanding is that the peer keep-alive should not be carried over a vlan that is allowed on the peer link trunk, so the choices are:

A: Use mgmt0 interface.

B: create a vlan and make sure it is not allowed on the peer link trunk, but allowed on the uplink (to 6500) trunk. then create one SVI in that VLAN on each N5k and use those two SVI for peer keepalive.

my questions is

1. I know Cisco recommends A, but why? what's the pros and cons of each method?
2. if I use A, my understanding is that the vlan that mgmt0 is in can be allowed on the peer link trunk, because the mgmt0 is a different VRF (management) than all other traffic in the default VRF, even if the VLAN is allowed, the peer keepalive traffic will not use the peer link. correct?

Thank in advance,

ming

[Lucien Avramov · Accepted solution]

A few reasons for this:

-mgmt0 is on a different vrf and doesnt interfere with your data traffic

-mgmt0 saves you from utilizing a 10 gb port from your switch

- management vrf traffic doesnt go trough the defaut vrf ( peer link)

[minghui.qi]

hi Lucient

Thanks for your reply.

I agree, but customer wants to keep the mgmt0 as pure out of band mgmt only, no important traffic uses it, so I guess the question comes down to what is the real impact of losing peer keepalive while peer link is still fine?

Using SVI won't be wasting a 10G ports because the that VLAN traffic will just go via the 10G uplinks rather than dedicated 10G ports, is there any problem doing this way?

thanks

ming

[habadr · Accepted solution]

Hi Ming,

Please find my answers

Q what is the real impact of losing peer keepalive while peer link is

still fine?

Answer: First to get your VPC up and running you have to have the

peer-keeplive working. However if you lose it after that traffic will

not be interrupted. Peer-keepalive is used as a backup path between both

switches in case of peer link failure to avoid split brain scenario. The

per-keepalive message size is very small and its default interval time

is 1 second.

As Lucient mentioned we recommend using the management interface because

it is carried in separate VRF (management). You can carry it in

isolated SVI but it will be in the default VRF. Since all traffic is

carried in Default you have to be very careful and make sure that this

traffic will not be carried peer-link. Our concern is that if you run it

in default VRF you "may" end up utilize the peerlink for peer-keepalive

message which is simply wiping out it is purpose as backup link.

http://www.cisco.com/en/US/partner/docs/switches/datacenter/nexus5000/sw

/layer2/502_n1_1/Cisco_n5k_layer2_config_gd_rel_502_N1_1_chapter8.html#c

oncept_47F7274E5FDA489884D0488BC491B066

Thanks

Hatim Badr

[minghui.qi]

hi all,

Thanks a lot for your help. I will leave the decision to customer, if using SVI, I will make sure that VLAN is not allowed on the peer link.

thanks,

ming

[michoco911]

Dear All,

Just one question related to this subject.

If we use the management port for the keepalive link connecting both Nexus switches, how the Two Nexus switches will be managed?

Will they have any virtual IP address to manage both, or each one will be managed through his own IP address?

Please clarify.

[minghui.qi]

hi michoco911,

I think you got confused with the peer link and peer keepalive link, peer link is a physical link between two N5k for all sync traffic bewteen them, peer keepalive link is usually routed via either mgmt0 interface or SVI as the two option I described above. it's there in case the peer link drops. refer to the doc.

The mgmt0 port will still connect to whatever switches for mgmt purposes, you will just be sending the peer keepalive traffic using the mgmt port. the N5k switches can be managed using the mgmt0 interfaces or you can create SVI on the switch as inband mgmt interface.

Hope it helps

ming

[e-chuah]

Alternatively, for your option B, you can have a direct link between the two 5548 for vpc keepalive. Since the 5548 is layer 2, you will need to create a SVI for that and make sure that vpc keepalive vlan is not allowed in the peer link. Ofcourse, this will consume 1 interface on each 5548. If you look at the cisco networkers slides, this is the preferred method. I also think this is a neat way to do it as well. Rgds Eng Wee