Where do I find the "management" VRF in ACI?

[RedNectar]

[Apologies if you have already seen this: I accidently posted it in /Service Providers/MPLS/ and found no way to move it] 

Hi,

Let me start by saying I suspect I know the answer, but am hoping someone with more knowlwdge than I have will be able to elaborate.

Fist of all, let me define where I see the allusive management VRF.  The easiest is from the CLI on a brand-new ACI fabric. Run the show vrf command on any switch and you will see:

apic1# fabric 201 show vrf
----------------------------------------------------------------
 Node 201 (Spine201)
----------------------------------------------------------------
 VRF-Name                           VRF-ID State    Reason
 black-hole                              3 Up       --
 management                              2 Up       --
 overlay-1                               4 Up       --

Now curiously, I don't see any VRF with a VRF-ID of 1, nor do I see the pre-defined VRFs called mgmt:inb or mgmt:oob - which are seen when I issue the same command on the APIC. (I may have chrated a little here, because my fabric is NOT brand new, I have configured inb mangement)

apic1# show vrf
 Tenant      Vrf         Consumed Contracts    Provided Contracts    Description
 ----------  ----------  --------------------  --------------------  ----------------------------------------
 common      copy        -                     -
 common      default     -                     -
 infra       ave-ctrl    -                     -
 infra       overlay-1   -                     -
 mgmt        inb         -                     -
 mgmt        oob         -                     -

Now my suspicion is that the management VRF is in fact an alias for the mgmt:oob VRF - based on the fact that if I issue a command like: fabric 201 show ip route vrf management, I see the default route of the OOB VRF, in my case 172,16.12.1.

apic1# fabric 201 show ip route vrf management
----------------------------------------------------------------
 Node 201 (Spine201)
----------------------------------------------------------------
IP Route Table for VRF "management"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

0.0.0.0/0, ubest/mbest: 1/0
    *via 172.16.12.1/32, mgmt0, [0], 10:12:05, local

So here is what I hope someone can tell me:

• Is the mangement VRF ALWAYS the same as the mgmt:oob VRF?
  1. If so, why does ACI have two names for the same thing? (Again I suspect that if this is the case it just another of the MANY inconsistencies in ACI put there to confuse us)
  2. If not, what exactly IS the relationship between the mangement and the mgmt:oob VRF?  And how do I verify that?

One of the places where I have found this particularly confusing is when configuring NTP. When I issue the command  fabric 201 show ntp peers (or fabric 201 show ntp peer-status), I see the VRF listed as management

apic1# fabric 201 show ntp peers
----------------------------------------------------------------
 Node 201 (Spine201)
----------------------------------------------------------------
-----------------------------------------------------------------------------
  Peer IP Address                         Serv/Peer Prefer KeyId   Vrf
-----------------------------------------------------------------------------
  172.16.1.5                              Server    no     None    management

My problem is that I have no idea where to configure this allusive mangement VRF!

[Marcel Zehnder · Accepted solution]

Hi Chris

That's correct: mgmt:oob == management.

I think this is a "Nexus-thing": The out-of-band mgmt interface (mgmt0) Interface is always bound to the VRF "management". So even if there is an ACI image running - it's still Nexus-hardware running a special NX-OS.

HTH

Marcel

[RedNectar]

Thanks Marcel,

I can't count the hours I've wasted looking for relationships in ACI because of inconsistencies of naming. And although I accept the "Nexus-thing" explanation, I really don't think it is a good enough excuse for the inconsistency.

And for the record there is another inconsistency.

If I use the GUI to navigate to Fabric > Inventory >> Pod 1 > Leaf101 > Interfaces > Management Interfaces > mgmt0, I do see the IP address of the OOB interface.

BUT, if I issue an ifconfig command on Leaf101, mgmt0 interface exists, but has no IP address and a different MAC address to that shown in the GUI.  The IP and MAC corresponding to the mgmt0 interface shown in the GUI is associated with interface eth0

Thanks again for taking the time to respond.

[CristianP]

this could be because one view (ifconfig) is the Linux kernel view of the interfaces while show interfaces shows the view of the NXOS for the same. One (mgmt0) could simply be an alias of the other (ethX -in your case x=0).
this is proved by the fact that they share the same MAC address

F1P1S1# show int mgmt0
mgmt0 is up
admin state is up,
Hardware: GigabitEthernet, address: e4c7.22bd.e5e8 (bia e4c7.22bd.e5e8)
Internet Address is 10.85.53.41/28
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is routed
full-duplex, 1000 Mb/s
Beacon is turned off
Auto-Negotiation is turned on
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
EtherType is 0x0000
30 seconds input rate 4480 bits/sec, 6 packets/sec
30 seconds output rate 12408 bits/sec, 7 packets/sec
Rx
376651 input packets 141027 unicast packets 234257 multicast packets
1367 broadcast packets 112626826 bytes
Tx
207051 output packets 207043 unicast packets 4 multicast packets
4 broadcast packets 46241130 bytes

F1P1S1# ifconfig eth6
eth6 Link encap:Ethernet HWaddr e4:c7:22:bd:e5:e8
inet addr:10.85.53.41 Bcast:10.85.53.47 Mask:255.255.255.240
inet6 addr: fe80::e6c7:22ff:febd:e5e8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:376709 errors:0 dropped:0 overruns:0 frame:0
TX packets:207091 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:111126739 (105.9 MiB) TX bytes:44055740 (42.0 MiB)

F1P1S1#