UCS C220 Very Slow RAID 10

[Arne Bier]

Hello

Why would a brand new UCS C220 with 4 x 1.8 TB (10K) HDD in RAID 10 give me only 50 MB/s write speeds on a lab system that is under no load at all (see ISE Alarm messages below)?

I have installed ESXi 6.7u3 (using the vendor Cisco .iso) on the 32 GB flash disk. The RAID 10 volume was used as a single data store. I did not initialise the RAID10 volume (I didn't think it was required, as opposed to say RAID 5)

I have run disk read and write tests on the Cisco ISE VM, as well as Windows 2019 VM - the results are terrible. The data sheet for one of the 1.8TB 10K drives says 260 MB/s sustained write speeds.

Wow do I go about fixing this? The server came fresh from Cisco on 4.1(2a) firmware? Does that imply all components are up to date?

I have not applied any ESXi patches yet. That would be the next step - but I have a feeling it's not an ESXi issue.

All the BIOS settings are still at default and by all accounts, are set optimally for virtual/hypervisor workloads.

The RAID 10 volume says it's running optimally.

[Kirk J · Accepted solution]

Try changing your VD's write caching policy from writethrough to 'write-back with good bbu', and see if those numbers perk up quite a bit.

You can change that on the fly, and doesn't require a reboot.

Kirk...

[Arne Bier]

Hello Kirk

I will give that a try as soon as I can get remote access again to the customer's VPN.

By the way, can you confirm whether or not it's required to initialise the UCS's RAID 10 volume when taking receipt of a new server from Cisco factory?

There seems to be a slightly newer firmware 4.1.2(b) - I am on 4.1.2(a). The RAID Controller has an update in that firmware.

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/release/firmware_files/4_1/b_UCS_C-series_Firmware-Files_4_1/m-firmware-files-for-4-1-2.html#Cisco_Reference.dita_682ad914-2a4a-466e-87ca-e089cc3f0295__firmware_4_1_2a

Can't be sure whether that would have any effect - but I might just update the whole lot

Unrelated to the ESXI datastore issue, I had an issue with the UCS FlexFlash (32GB SD card for the ESXi) - the install didn't work first time - I had to format the flash and then disable/enable the flash. Not sure why Cisco doesn't ship those things formatted from factory. But at least the answer was just a google search away.

regards

Arne

[Kirk J]

The initialize function should actual clear the drive.

Is the server shipped with an OS that are are wanting to wipe?

Initializing a Virtual Drive

All data on a virtual drive is lost when you initialize the drive. Before you run an initialization, back up any data on the virtual drive that you want to save

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c/sw/gui/config/guide/4_1/b_Cisco_UCS_C-series_GUI_Configuration_Guide_41/b_Cisco_UCS_C-series_GUI_Configuration_Guide_41_chapter_01100.html 

Kirk...

[Arne Bier]

I don't want to wipe the ESXi datastore. The single RAID 10 volume is used exclusively by the ESXi data store. The ESXi software is not on the data-store - I boot ESXi off the 32 GB SD card.

The question was whether or not to initialize a RAID 10 volume before presenting it to the OS/application (like vSphere ESXi). I thought I had read somewhere that RAID 5 volumes must be initialized before they can be used (to spread the parity around?). I could not find any documentation that stated that RAID 10 volumes needed to be initialized before they could be used by the operating system - it's probably a non-issue. 

[Kirk J]

Well, the VDs do get initialized, but when they are first created, prior to putting data on them.

How is the write-back caching change working out?

Kirk...

[Arne Bier]

Hi @Kirk J 

Your recommendation was 100% spot on. What an amazing difference. This should be enabled by default in my opinion.

Before (with Write Policy set to default 'Write Through')

After (with Write Policy set to Write Back Good BBU) - I did this while the ESXi was running - no downtime. Nice.