UCS Memory Configuration

idnetter · ‎08-12-2021

Hello all,

Hoping someone in this community can offer advice on memory configuration options for UCSB-B200-M4.

I have UCSB-B200-M4’s with Intel Xeon E5-2683 v3 CPUs (UCS-CPU-E52683D) currently populated with 8 x 32GB (UCS-ML-1X324RU-A) modules providing 256GB total memory in each blade and am looking at options to upgrade. According to the Cisco spec sheet, the recommended memory configuration for 512GB is to add an additional 8 x 32GB. However, if I want to go further and increase to e.g. 768GB total if I upgrade with 16 x 32GB (UCS-ML-1X324RU-A) maxing out each blade at 24 x 32GB DIMMS, the DIMM max speed is reduced from 2133MHz to 1866MHz - memory speed is extremely important for my workloads so a reduction in speed is not something I can consider.

My question relates to choosing 2400MHz DIMMs (UCS-ML-1X644RV-A) instead and their compatibility with the above Xeon E5-2600 v3 CPUs.

Are the 2400MHz DIMMs (UCS-ML-1X644RV-A) supported if I populate the server with e.g. 16 x UCS-ML-1X644RV-A bringing total memory to 1024GB?

What speed can I expect this memory to run at?

This is the Cisco spec sheet I have been referencing:

https://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/b200m4-specsheet.pdf

I appreciated any help or guidance.

Thank you!

Steven Tardy · ‎08-14-2021

Find your CPU on the old spec sheets (have to use archive.org as Cisco Engineering removes PIDs from the spec sheet):

https://web.archive.org/web/20160331143243/http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/b200m4-specsheet.pdf

To find the details about the CPU you have installed in table:

Table 3 Available Intel CPUs: E5-2600 v3 Series Processor Family CPUs

2DPC (DIMMs per channel) 2133MHz DIMMs will run at 2133MHz.

3DPC (DIMMs per channel) 2133MHz DIMMs will run at 1866MHz.

2400MHz DIMMs (slow down to match the CPU frequency) on the 2683 v3 CPU will run at 2133MHz max per the Intel docs:

https://ark.intel.com/content/www/us/en/ark/products/81055/intel-xeon-processor-e5-2683-v3-35m-cache-2-00-ghz.html

Unless you've really done a LOT of testing, I have doubts your real-world application would notice the slower 1866MHz memory frequency. In a synthetic/drag-race test maybe, in a real-world service you'd see better gains from other OS/BIOS tunes then you would from only a change in memory speed.

Also older DIMMs aren't manufactured at volume (and maybe even at all) so finding DIMMs for older servers may require using refurbished DIMMs.

idnetter · ‎09-02-2021

Many thanks for this, Steven. I think on that basis we'll proceed with 2400MHz DIMMs running at 2133MHz.

Do you happen to know if there a good reference document for the procedure to upgrade the memory on each blade? I came across the following article which notes a fairly significant gotcha:

RAM Upgrade Procedures on Cisco UCS Servers

Steven Tardy · ‎09-02-2021

That link does cover two common UCS specific tasks:

Upgrading the UCS capability catalog if PIDs were added to the catalog after a given firmware version.
Server decommission/re-commission if hardware changes or the PID is unknown.

These steps are likely buried somewhere in the UCS documentation, but 10 minutes of google'ing and scrolling did NOT uncover a good end-to-end walk-through of the steps.