MGMT Interface on Nexus 7010

[mohankumarm]

Hi everybody,

We have a couple of Nexus 7010's split into Core and Distribution VDCs. MGMT0 interfaces on each of the Nexus VDC's (including the Admin VDC) are configured with different IP address, but on the same subnet i.e 10.10.10.1/24 for admin, 10.10.10.2/24 for Core and 10.10.10.3/24 for Distribution. The MGMT 0 physical port on each Nexus is connected to a physical gig port on a 3750 X switch, and the 3750X has uplinks back to the Nexus configured for vPC.

When i ssh to the VDC MGMT0 IPs from the 3750X, i can access each of these VDCs without any problems. But if i enable routing on each of these links(OSPF) and advertise it to the WAN, i cannot see these routes advertised and also cannot see any of these routes in the local routing table.            Just wondering if i have to enable these links on a VLAN and then advertise it to the WAN..But if this the case, VLANs cannot be created  on the Admin(default VDC).

Thanks and Regards,

Mohan

[Steve Fuller]

Hi,

The mgmt0 interface is typically in the Management VRF and so the addresses will not be visible within the default VRF that's probably connected to your WAN. You can see the VRF these interfaces are associated with using the show ip interface brief vrf all command.

Regards

[Reza Sharifi]

As Steve said, the management interface is in a separate vrf and it is not visible to the regular routing table.  Now, since all the mgmt interfaces connect to the 3750, if you could connect the 3750 to your WAN router and not back to the 7k, then these routes would be visible on you WAN side. As for admin VDC, you are correct, you can not create any vlan on it.  The Admin VDC has limited functionality.

HTH

[mohankumarm]

Thanks a lot Reza and Steve. Currently we are connected to the WAN via OSPF. I was thinking that if i extend the OSPF area to 3750 ( at the moment this is vPC L2 trunk link to Nexus) and establish a common mgmt vlan on the Nexus and 3750X, will this work?

Best Regards,

Mohan

[Reza Sharifi]

Hi Mohan,

Here is what I think should work.  All your mgmt0 interfaces are connected to the 3750 and are all in one vlan.  lets call it vlan 100.  If you run OSPF on the 3750, you simply add vlan 100 to OSPF. Now, have a layer-3 connection from the 3750 to your distro VDC using a /30 and advertise this /30 into OSPF on both 3750 and 7k. Now, you should be able to advertise these routes including your management subnet (vlan 100) to your WAN.  Am I understanding your design correctly?

HTH

Reza

[mohankumarm]

Hi Reza,

Thanks and you are absolutely right..My only worry now is because the mgmt0 interfaces are in Management VRF on the Distribution VDC, these mgmt0 interfaces have to be in the same VLAN 100 as above correct? or is that just advertising OSPF on VLAN 100 on the Distribution VDC as well as 3750 will suffice? so technically we are bringing all the managment routes into the OOB switch and pushing it back into the WAN?

Thanks and Regards,

Mohan

[Reza Sharifi · Accepted solution]

Hi Mohan,

That is correct, We are bringing vlan 100 which is a management vlan and on the 7ks the mgmt0 ips are part of vrf management back to the 7k and put them in global routing table where they will be visible to the rest of the network.

Basically, we are connecting the management vrf to the global routing table through the 3750.

HTH

Reza

[mohankumarm]

Hi Reza,

Thank you very much indeed and will try and get this work. Also, another point being, do we need separate Layer 2 and Layer 3 links between 3750 and Nexus or a couple of layer 3 links will suffice in this design? I was thinking Layer 2 vPC/Port channel for the mgmt information to come in from Nexus to 3750 and Layer 3 to push it back out to Nexus (preferably HSRP VLAN)

Thanks and Regards,

Mohan

[Reza Sharifi · Accepted solution]

Hi Mohan,

You just need redundant layer-3 links (/30s) from the 3750 back to the Nexus, no need for layer-2. vPC, etc...

The management interfaces connecting to the 3750 are just host ports.  So, you should be good to go with just the layer-3 links.

HTH

Reza

[mohankumarm]

Hi Reza,

Thanks again and its very clear now and will get this going soon.

Best Regards,

Mohan