HX240C M4- use as stanalone with Ubuntu 20.x - networking

[alanraczek11]

We have 3 HX240C M4's that were logically disconnected from the UCS manager (although you can still see them there). We need to use these servers independently to build a Kubernetes cluster. See the image below: 

Upon installation of Ubuntu it seems Ubuntu does not have drivers for the built in NIC's on the UCS rack servers. So it seems we need to use the fiber interconnects after all. So I believe I have to re-integrate the servers back into UCS somehow and then set up their IP addresses. I am absolutely a novice with UCS, seems a little daunting to me. Would anyone be able to detail how this all works and guide me on how to accomplish this? 

Once the networking is in place the developers building this K8 cluster will also install ceph for storage. Any help would be appreciated. I DO have a TAC case pending but want a little more insight. Just telling me to read the doc doesn't help much.

Thank you VM.

...ar

[alanraczek11]

Kirk-

Did more reading and research and came across an article explaining EXACTLY what you are saying here.

https://community.cisco.com/t5/unified-computing-system/ucs-native-vlan-question/td-p/1601185

So I drilled down in the Servers section and down to the vNICS under the rack-unit-x servers. The VLAN we need for lab access is to

VLAN 110. 

So if I check this as a native VLAN I assume this is a step in the right direction? Makes sense. I just get confused on WHERE I should set the native VLAN as if I do it here I will need to go to each server and set this option. In our case no big deal, only 3 servers. But if it were 100 servers then where does this get set? Probably Service Profile Templates ?

I know a lot of questions, sorry. Last one is if I look at rack 1,2 or 3 how do I know what service profile is in use?

[Kirk J]

Yes, you are on the right path.

Where you make the changes just depends on whether you are using service profile templates or vnic templates, or neither.

If you have updating templates, then a change to the vnic made at template will auto cascade down to the service profiles.

Same thing would apply to changing an updating vnic template, that is in turn embedded in either service profile template, or service profile directly.

Kirk...

[alanraczek11]

I need to know how to check what templates I am using. Or Service Profile(s) I am using. How do I know where to look? I tried marking a VLAN as "native" and it did nothing. All I did was mark it active and saved changes. Right now I am just hunting and pecking to see where these things are.

These "updating" templates are templates called:

Service Template compute-nodes

Service Template compute-nodes-M5

Service Template hx-nodes

Service Template hx-node-M5

I made the change under the first compute-nodes. Obviously not the right spot.

[Kirk J]

The actual object you are changing (assuming you are still using the HX related service profiles) is a vnic template.

Here is the order to confirm what is linked to what.

• equipment (rack server # in question), 
• click on the service profile in middle (blue link) on general tab.
• In service profile general tab, click on Template instance
• In Service Profile template, click on Network tab
  • This should show it's using a Lan connectivity policy
  • click on lan connectivity instance
  • if you expand the listed vnics, it should list the vlans, as well as the native vlan tab.
  • If you highlight the vnic in question, click modify, and it will show you the vnic template name that is in use.
    • You will need to go find this same vnic template (typically it will be under you LAN tab, policies, your suborg, vnic templates.
    • Adjust the vlan in question for your A side vnic.  The B side should be 2ndary template and auto update, if not, make sure you modify the B side template for the same type vnic.
    • In the vnic template, click on 'show policy usage'.  this should show both the service profile template and the service profiles that are linked to this.
    • Also make sure it is set to 'updating template'

This takes about 30 seconds to show on a webex, but 10 minutes worth of writing ; )

Kirk...

[alanraczek11]

Getting closer, I can feel it

Seems to me the VLAN which I want Ubuntu to connect to (110) is not using a vNIC template if I did this correctly. It takes up space but I wanted to show what I did.

[Kirk J · Accepted solution]

So in that picture, I think it looks like you are using vnics directly added to a service profile template, and not vnics tied to vnic template.

IF, the service profile template is set to 'updating template' type, then marking that 110 vlan as native (which it is not in your picture), should get pushed down to your servers.

Kirk...

[alanraczek11]

OMG! Good news, I got rack 2 & 3 servers pinging the lab network successfully. I followed your instructions and looked and set native VLAN under Service Profile Templates for the HX-NODES template which was the one in use for that VLAN. I also set the native VLAN in connection policies. Rack 1 server is "out of the loop" only because Cisco TAC was working with it creating new policies and such. So I need to bring that machine back in. Is it just a matter of setting the policies and templates the same as the other 2?

I still have the boot issue but this is a big deal. Networking actually working.

...Alan

[alanraczek11]

I bound the rack1 server to the HX-NODES Service Profile Template and was hoping the boot issue would get fixed. Still need to make sure I did everything I needed to bring rack 1 back into the fold.

...Alan

[Kirk J]

Did we ever get the 'SD card' Boot item created in the boot policy?

Sent you an email...

Kirk...