full width blade-Q

[K P SIM]

Hi, Cisco Folks:

Can someone please help me to clarify the following doubts?

Q1. What is the pinning order of B250-M1 to IOM  if I have 2 Server Ports with a fully populated chassis of 4 B250-M1?

I know that for a half-width, the pinning order will look like:

                                                  Pinned Blade #

-----------------------------------------------------------------------------

Server Link 1 or Port 1 of IOM       Blade 1,3,5,    7

Server Link 2 or Port 2 of IOM       Blade 2,4,6, & 8

OR

                Pinned Blade #

--------------------------------------------------------------------------

Server Link 1 or Port 1 of IOM      Blade 1,3,5,    7

Server Link 2 or Port 2 of IOM      Blade 2,4,6, & 8

Q1.1

If I have the Server Link 1 connects to Port 1 of IOM, but Server Link 2 connects to Port 3 of IOM, will this change the pinning order?

Q1.2

If I have the Server Link 1 connects to Port 1 of IOM, but Server Link 2 connects to Port 4 of IOM, will this change the pinning order?

How about other combinations Like Port 2 of IOM (Sever Link 1) and Port 3 of IOM (Server Link 2)?

Q2. If I only install a Single M81KR in B250-M1, do I have to care about which Adapter Slot to install my PALO Card? My concern here is really Slot 1 is tied to IOM A and Slot B is tied to IOM B, is there such concept at all??

If there is such concept, then I would make sure that for a 4 blades setup, I would have 2 blades with PALO inserted into Slot 1 and the other 2 blades with PALO inserted into Slot 2,

Thanks.

SiM

[abbharga]

Sim,

The piining is based on the slot, so if there are two links the following slots will be pinned:

Link 1: 1,3,5,7

Link 2: 2,4,6,8

With the full width blades each blade occupies two slots, e.g blade number 1 will be using slot 1 and slot 2 (left adapter slot-1 and right slot-2).

Q1.1

If I have the Server Link 1 connects to Port 1 of IOM, but  Server Link 2 connects to Port 3 of IOM, will this change the pinning  order?

[abbharga] there will be no change in the piining, its always the links which are live and ack'd

Q1.2

If I have the Server Link 1 connects to Port 1 of IOM,  but Server Link 2 connects to Port 4 of IOM, will this change the  pinning order?

[abbharga] answer same as above.

Q2. If I only install a Single M81KR in B250-M1,  do I have to care about which Adapter Slot to install my PALO Card? My  concern here is really Slot 1 is tied to IOM A and Slot B is tied to IOM  B, is there such concept at all??

[abbharga] in such scenario, it would be good to stagger the adapters across the slots (i.e left and right) among the blades in the same chassis.

[K P SIM]

Hi, Abbharga:

Thank but I need further clarifications.

EXCERPT from your answer:

The piining is based on the slot, so if there are two links the following slots will be pinned:

Link 1: 1,3,5,7

Link 2: 2,4,6,8

With the full width blades each blade occupies two slots, e.g blade number 1 will be using slot 1 and slot 2 (left adapter slot-1 and right slot-2).

Q1. So assuming I have a full-width blade in Slot 1 which in this case will ALSO include Slot 2 as it is a Full-Width Blade, WHICH Server Link will be used-JUST Link 1 or 2 or BOTH Link 1 & Link2???

Thanks.

SiM

[abbharga]

It will be both Link 1 and 2.

./Abhinav

[K P SIM]

./Abhinav or other Cisco Gurus:

May I ask again?

q1. what is the exact command to find out which adapter slot is currently being occupied by M81KR as customer is having a single PALO on a full-width blade?

They are using UCS 1.3(1#) and UCS 2.0(##) for the 2 sites.

Appreciate it.

SiM

[abbharga]

You can select the blade or the SP in the UCSM and in the work space (right hand side) you can select the VIF Path tab and it will show you the entire path -> IOM Port -> FI Server Port -> FI Uplink port

./Abhinav

[K P SIM]

./Abhinav

Unfortunately, it DOES NOT show me which Mezzanine Slot of B250 is being used to install that single M81KR.

The confusion is if I have 1 M81KR installed into the LEFT or RIGHT Mezzanine Slot, the UCSM does not tell me clearly iwhere exactly is my M81KR being installed. I know very well that for 1 or 2 B250, the M81KR is being installed differently, but UCSM shows identical output, etc.

Frustrating. Is there a command I can run to tell me very clearly beside open up the blade casing which causes downtime to customer?

Thanks.

KP

[abbharga]

KP,

As far as I can remember adapter 1 is LEFT and Adapter 2 is right.

Here is one quick way to figure out which slot the adapter is installed at:

UCSM CLI : connetion iom x (x - is chassis number)

show platform software redwood sts

You will get an output like this:

Look at the conenction satus for each blade. A full width blade will occupy two slots (1+2, 3+4, 5+6, 7+8)

If there are both the adapter, you'll see ' | ' in the connection status (right above the ' v ' ) for both the slots. However in your case you should see ' I ' only for one slot. In case your blade is in slot 1 (i.e using slot 1 & 2 both) and you see ' | ' for slot 2 then the adapter is in installed on the right side otehrwise if its for 1 then the adapter is on left side.

./Abhinav

[K P SIM]

./Abhinav

Let me try out once I have the opportunity and update you.

I am beginning to understnad more now esp. with your deep technical expertise. I appreciate your input.

1 more question lingering on my mind:

If I have only 1 PALO installed in one of the 2 Mezzanine Slots of a Full-Width Blade, AM I really using only 1 Server Link at all assuming I have 2 Server Links i.e. I made use of 2 FEX Ports?

Example: B250-M2

2 Server Links Per IOM/FEX to Each FI

My full Width blade is in Slot 1 & 2 or the TOP of the Chassis.

For half-width, I know it is very clear as in:

2-Server Link     Blade (Assuming I have 8 blades)

Server Link 1     1, 3, 5, 7 - Blade 1,3,5 & 7 will make use of this 10G link only

Server Link 2     2, 4, 6, 8

4-Server Link     Blade (Assuming I have 8 blades)

Server Link 1     1, 5 - Blade 1 & 5 will make use of this 10G link only

Server Link 2     2, 6,

Server Link 3     3, 7

Server Link 4     4, 8

Please help me to explain further as I know Cisco blades, but there is a lot I need to learn everyday.

Appreciate it.

KP

[abbharga]

KP,

With one adapter you only use one link per side.

./Abhinav

[K P SIM]

./Abhinav:

Refer to your FIRST reply to my post, I am giving it a deeper thought and have the question below.

Q2. If I only install a Single M81KR in B250-M1,  do I have to care about which Adapter Slot to install my PALO Card? My  concern here is really Slot 1 is tied to IOM A and Slot B is tied to IOM  B, is there such concept at all??

[abbharga] in such scenario, it would be good to stagger the adapters across the slots (i.e left and right) among the blades in the same chassis.

Q.

May I know what is the exact technical reason to stagger the adapters across the slots? Assuming I have 4 full-width blades with a single adapter each, I would install the adapter on the LEFT slot for 2 servers and RIGHT slot for the other 2 servers.

The more I think, sometimes I get more confused.

I know that with a single adapter will have 2 internal traces to both IOM-i.e. 1 to the IOM 1 and the other to IOM 2. With 2 Single Adapters, I will have 4 internal traces to both IOM, albeit with a total of 40G in this case.

R u able to shed some light into this?

Again, I think you are able to answer all my questions.

Thanks.

SiM

[abbharga · Accepted solution]

KP,

In very simple words, its to share the load on different IOM to FI links.

./Abhinav

[K P SIM]

Hi, ./Abbharga:

As I slowly crawl through and trying to understand all your replies, I begin to understand your answers related to Cisco FWidth blades as really I seldon deal with it, so obviously, its internal architecture is hard to co-related, but with your helps, I am better off now.

Great Job.

SiM