connecting the FIs to the N5ks with layer-3

[sandevsingh]

Hi, we have a setup where we are connecting the UCS FIs to nexus 5548UP with layer3 modules. All layer 3 for the UCS domain will be handled by the n5ks. The n5ks are in a vpc domain and each FI is dual connected to both the n5ks. Now I will have to create layer-3 subinterfaces on the n5ks (similar to a router-on-the-stick design) in order to facilate layer3 routing. FI-A connects to both n5ks with PO1 and FI-B connects to both n5ks also with PO1.

config as below -

N5k-A:

int eth1/1

des to FI-A

no switchport

channel-group 1 mode active

int eth1/2

des to FI-B

no switchport

channel-group 1 mode active

int po1.100

ip add 10.110.100.2/24

encap dot1q 100

hsrp 100

ip 10.110.100.1

prempt

priority 110

N5k-B has exactly same config, except that it is hsrp secondary. I am doing the right way? Please advise?

[David Grocke]

Hi There!

I hadn't thought of doing it that way, but I think you need to use VLANs as the associated vlan tag of a sub-interface will not traverse a VPC link.

It may be best to post this in the switching and routing or Nexus forums, but my understanding of a subinterface is that it simply reads the VLAN tag being sent over the wire and isn't a layer 2 VLAN on the local device. If it's not a layer 2 VLAN on the device your VPC peer link will never pass the traffic, plus I can't think how the VPC address tables would work with sub-interfaces.

You could tell easily as I wouldn't expect your HSRP to come up, even if your VPC peer link is traversing the related VLAN.

I would recommend creating L3 SVIs for each of your assiciated VLANs and prune both on the VPC peer link and your port-channels to your UCS. This also means you aren't restricted to using these VLANs exclusively for UCS.

My rule of thumb would be if you are using a L3 switch, like a N5K with a L3 module, then there is no reason to use subinterfaces. Only use subinterfaces on routed ports where you cannot trunk and use L3 VLANs everywhere else. I don't think this would be accepted everywhere because some security aware people do prefer subinterfaces.

Cheers

Daivd Grocke

[sandevsingh]

Thanks David, you are right.... my hsrp groups did not come up perfectly, each 5k was making itself active as those vlans never run through the vpc peer-link even though they are created and tagged there.

I am gonna try SVIs and see how it goes..