Create Cisco UCS FI Cluster without connecting mgt ports to network switches

[IC2317]

Hi,

I would like to know if we can create FI cluster without connecting MGT ports to network switch. Reason, for this is that I just want to configure UCS FI cluster with Subnet from Different Site. Site where I'll be configuring this cluster does not have that subnet range and then later use it in other site where it already has that subnet configured.

I know there could be ways to have same subnets across these sites, however that will require extra efforts so don't want to go that way.Right now just want to have it configured and then connect-> power on at the other site so that its available on the network.

[RedNectar]

Hi @IC2317 ,

---

@IC2317 wrote:

Hi,

I would like to know if we can create FI cluster without connecting MGT ports to network switch.

---

Not really. You DO need to assign some IP addresses to the Mgmt ports to enable you to configure FI Cluster

---

Reason, for this is that I just want to configure UCS FI cluster with Subnet from Different Site.

Then why not assign a "temporary" private subnet to allow the intitial configuration to be completed. BTW - did you know that there are more IP addresses than just the regular 3 (10.x.x.x 172.16-31.x.x and 192.168.x.x)?  Like 198.18-19.x.x? [Edit: I should mention that this range is reserved for "Network Interconnect Device Benchmark Testing". But I'd consider this legitimate use.]

Then if you later extend your regular management subnet to that site, you can change the IPs to match.

I hope this helps

---

Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

---

[IC2317]

Thanks Chris.

Yes, I know we need to assign IP address to the FI. However, My query was that if I don't have the MGT port connected/cabled then can I still assign the IP address from management subnet to the FI and configure cluster ? or do we need to have that subnet configured in advance on network switches and connect the MGT ports to those switches for cluster configuration ?

Site where I am configuring cluster does not have that Management subnet configured and we don't want to configure that because both sites have different subnets. After the cluster is created, devices will be shifted to other site and powered on there. This is just to have initial configuration ready because we don't have anyone on other site who can do this configuration and right now we cannot travel :p.

[RedNectar]

Hi @IC2317,

Yes, I know we need to assign IP address to the FI. However, My query was that if I don't have the MGT port connected/cabled ...

If the MGT port is not connected, you won't be able to configure anything. You can't get to the management interface via any other interface - remember that Fabric Interconnects are pass-through devices, not like normal switches. Although I wrote this answer in relation to ACI and UCS, you might find it worth reading for the diagrams if nothing else.

then can I still assign the IP address from management subnet to the FI and configure cluster ? or do we need to have that subnet configured in advance on network switches and connect the MGT ports to those switches for cluster configuration ?

Well, how you do your subnet planning is up to you. And I don't know where your management PC is - is it local to the FIs or do you want to configure remotely?

My advice is that if you want to configure the FIs remotely, you assign the MGT interface an IP address that is local to the site and able to a) have access to a DNS server that can resolve svc.intersight.com, and b) access svc.intersight.com, then use Intersight to manage the FIs

Site where I am configuring cluster does not have that Management subnet configured and we don't want to configure that because both sites have different subnets. After the cluster is created, devices will be shifted to other site and powered on there. This is just to have initial configuration ready because we don't have anyone on other site who can do this configuration and right now we cannot travel :p.

---

Sounds like you are getting bogged down in worrying about the the IP addressing that will be used to do the configuring, and the IP address that will ultimately be the management IP address.  You could assign temorary IP addresses local to the site where you are doing the initial configuration, and when that is complete, change the IPs to the planned IP addresses and ship them out.

I'm not sure if you can claim the device in Intersight, then change the IPs and still have the device recognised by Intersight at another location, but I'd reckon the chances are pretty high that you could.  If these are new FIs there is every chance that you were sold an Essentials Licence for Intersight whether you knew it or not (the Base licence is free, but you can't do the remote configuration with Base). And if that is the case, especially in these time of restricted travel, I'd advise going down that path i.e

1. Make sure you have an Essentials (or better) licence for Intersight.
2. Assign IP addresses local to the site where you are doing the actual configuration. Make sure you configure DNS as well.
3. Access UCS Manager using the locally assigned IPs and get the devices claimed in Intersight
4. Change the IP addressing (and DNS if different at the new site) to the addressing to be used at the destination site
5. Ship the FIs and when they are installed you should be able to see them in Intesight, and continue any further configuration via Intersight

I hope this helps

---

Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

---

[IC2317]

Hi Chris,

Many thanks for you explanation and it all definitely makes sense. Probably, I was not able to explain what I am not looking for.

Let me explain it again. Lets say I have 2 Sites (X & Y) where X being the actual site where FI will be used in production and Y being the one where FIs are currently available.

Now, I want to use VLAN A (Available in site X but not in Y) for MGT of FIs, so that after configuration when I ship FIs to site X and connect it to switch ports already configured with VLAN A they should be reachable. Is it possible to do this without connecting MGT port to any upstream switch. ? 

NOTE - I cannot use X site's VLANs in site Y and vice versa and I don't care as of now if FIs configured with VLAN A is reachable or not at Site Y. Idea is just to configure with required subnet, so that when its connected at its desired site then it should be reachable.

[RedNectar]

Hi @IC2317 ,

I doni't quite understand. This paragraph has two parts that seem to contradict each other.

---

... when I ship FIs to site X and connect it to switch ports already configured with VLAN A they should be reachable.

That is possible - see top half of diagram below

Is it possible to do this without connecting MGT port to any upstream switch. ? 

---

No - this is not possible - see bottom ahlf of diagram below

[IC2317]

Hi Cris,

Thanks. As per bottom half of your diagram, I meant that at Site Y I want to configure FI(With IP Addresses of Site X) without any connection to Upstream Switch. After this ship it to X and connect it to Upstream switch. So, in my case your diagram bottom half will have FI connected to upstream switch in Site X not in Site Y.

I just want to know if I can configure FIs with no cables connected to Upstream switch in Site Y?

[Kirk J]

You are going to need have some sort of connectivity to mgmt ports setup to make mgmt cluster services happy.  If you are trying to configure your UCSM, without ever having had mgmt ports up to the FIs, I do not believe you will be successful, even if you have a console connection active.

With active mgmt ports, after configuration is completed,  you can use a console connection (which is out of band)  to apply the IP address changes that will match your 2nd site, where the cluster is going to be shipped, without needing local switch or routing accommodations for your changed node or VIP IPs.

In your diagram, you are showing no management port connections on FIs at the destination.

How are you planning on managing your FIs?  Also this will cause the FIs to to have HA/cluster status issues with mgmt ports being down.  What you are proposing does not meet FI connectivity requirements at the 2nd site.

Kirk...

[RedNectar · Accepted solution]

Hi @IC2317 ,

---

Thanks. As per bottom half of your diagram, I meant that at Site Y I want to configure FI(With IP Addresses of Site X) without any connection to Upstream Switch.

Yes you can, BUT you will need to have

• Both FIs powered on simultaneously, and
• The L1 and L2 cables from each FI connected to each other.

You do the initial FI configuration of:

• System name
• IP addresses
• Default Gateway
• DNS server IP address
• domain name
• admin password

using a console cable, so you can configure any IP address you like, so configuring the IP addresses ready for site X.

BUT

• The cluster will not form properly until
  • you configure some server port
  • connect a UCS B series chassis OR
  • a ucs C series server configured for single cable management
• and you will not be able to configure any service profiles or the like

After this ship it to X and connect it to Upstream switch. So, in my case your diagram bottom half will have FI connected to upstream switch in Site X not in Site Y.

I just want to know if I can configure FIs with no cables connected to Upstream switch in Site Y?

---

Yes, you will be able to configure all of the things I mentioned above without any connections to any switch, just a console cable.
And if the FI MGMT ports are connected to the VLAN with the correct default gateway that gives you connectivity to the rest of the world, then that will be fine.

Remember as @Kirk J says, you'll need that mgmt port connected once it is at Site X

I hope this has finally nailed it. It's been a lot of work for what in the end was a simple problem!

---

Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

---

[IC2317]

Many Thanks Chris for your support :). Probably, I was not able to explain in my first post :p

[RedNectar]

H1 @IC2317 ,

---

@IC2317 wrote:

Many Thanks Chris for your support :). Probably, I was not able to explain in my first post :p

---

More likely I got caught up in the detail about the subnets.  If I just read your first sentance

"I would like to know if we can create FI cluster without connecting MGT ports to network switch."

and analysed that a bit more, I may have get your answer much quicker.

Anyway, thanks for sticking with it, hope it all goes wel for you.