UCS B200 M3: cannot create vHBAs

[Evgeniy Lotosh]

Cisco UCS B200 M3 server with VIC 280 and VIC 240 cards installed. I try to create two vHBAs connected to default vSAN. I have a hardware profile accociated with the server, and I was able to create a singe vNIC. However, when I try to create vHBAs for the server, UCS manager displays the following error message:

There are not enough resources overall

Not enough vHBAs available

I'm new to managing Cisco blade servers, so I'm confused. What does this message mean? According to datasheets, VIC 1280 & 1240 are able to create a lot of vHBAs. I want to create only two of them. What's wrong?

Firmware bundles 2.1 are used for both the server and 6248UP interconnects. The blade chassis has single 2208 IOM installed, it's connected to 6248 with two links.

[padramas]

Hello,

Welcome to UCS.

Can you please share complete error message  and screen shot of your service profile ( HBA section )  ?

Padma

[Evgeniy Lotosh]

Padramas,

I'm not sure what do you mean by "screenshot of HBA section". I attach three screenshots of different areas. Is that what you need?

[padramas]

Hello,

Can you please check available identities left in pool used by  WWPN and WWNN ( as defined in vHBA1  ) ?

Padma

[Evgeniy Lotosh]

Padma,

I have only default WWPN and WWNN pools, and they are populated with 256 addresses each. I can see that the system borrows addresses from those pools, and there are 250+ addresses avaliable in each of them.

I've found fault references here: http://www.cisco.com/en/US/docs/unified_computing/ucs/ts/faults/reference/UCS_SEMs.html

According to the manual, error code F0169 I see in event descriptions has the following explanation: "This fault typically occurs when the Cisco UCS Manager could not place the vNIC on the vCon". Solution methods:

Step 1 Verify that the server was successfully discovered.

Step 2 Verify that the correct type of adapters are installed on the server.

Step 3 Confirm that the vCon assignment is correct.

Step 4 If the above steps do not resolve the issue, execute the show tech-support command and contact Cisco Technical Support.

---

All the servers are successfully discovered, and VIC 1240 & 1280 support vNIC and vHBA creation. How could I verify that vCon assignment is correct?

[Evgeniy Lotosh]

OK, here is the answer.

I've tried to do it by the book, and that was the main problem. All the user's manuals implicitly presume than there are TWO IOM modules in the chassis. So all the procedures are described in a way which requires connecting a server to both fabrics (A and B) simultaneousely. However, if there is only one IOM, the server connects to only one fabric, and any attempt to insert a vNIC or a vHBA assigned to other fabric leads to the failure described above.

So, the correct way to work with a single IOM is as follows:

1) Equipment tab -> find and click the server -> General tab in the right pane -> Connection details area -> look to which fabric the server is connected.

2) Add vNICs and vHBAs as needed connecting them to that fabric only and disabling failover.

Thanks to Cisco technical support specialist who was able to point to that pecularity.

[Robert Burns]

I've never seen a UCS (production) deployment with a single fabric.  There's no redundancy so its a rare option outside of testing environments.

Once you learn the basics of UCS, it will become obvious why fabric failover will not work with a single fabric - therer's only one!

You can still create multiple vnics/vhbas, but you just have to point them to the appropriate fabric, and avoid using FF.  We could ask for an enhancement to disable the FF option when there's a single fabric present, but to be honest it would likely get deferred because there's not enough use cases/asks for it.

Robert

[Evgeniy Lotosh]

Robert,

I work with different kind of servers and build SANs many years, so I do know how failover works, thank you. However, before I met Cisco blades, I supposed that those vNICs are similar to usual ("physical") HBAs: if there is connection to a fabric, they work with it, if not, then link is down, and that's all. I did not expect the system to not be able to create them at all.

Of course I do not ask for a new functionality. In fact, we are still getting used to new blades, and I do not intend to put them in production without redundancy. However, I would appreciate clarifications added to UCS Manager user's guide. It won't require great efforts - just put some phrases here and there,

[Robert Burns]

Learning UCS from scratch does require somewhat of a learning curve.  I'll see if I can update the config guides to account for single FI limitations.  Glad to see you were following the manual - the least we can do is ensure its complete.

Let us know if you have any other questions as you go.

Regards,

Robert