Cisco C240 M3 LSI 9271-8i Poor Raid10 Performance

[ibrahimd1]

Hi;
 

Im using a Cisco UCS 240-M3 server, I have 24 x 600 GB SFF 10K rpm disk, and Im build a Raid10 2 span 10 + 10 disk. 4 global hot spare. 

Raid Config;
Strip Size : 256K
Drivers per span : 10
Span depth : 2
Access-Policy: Read-Write
Cache Policy : Cached I/O
Read Ahead : Always
Req.Write Cache Policy : Write Back good bbu
Current Write Cache policy: Write Back
Disk Cache : Enable
 

Im tested our system 1 x centos 6 virtual machine. No load, no operation, only create 1 vm just tested. 

Our Disk performance Outputs;

[root@84 ~]# hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 500 MB in  1.04 seconds = 480.01 MB/sec

[root@84 ~]# sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.21286 s, 885 MB/s

[root@84 ~]# dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.38449 s, 776 MB/s

that disc instead, I get better performance wear 3 SSDs ( 3 x 120 GB ssd = $500). a product, poor performance failed. My friends are using older Dell server, they get higher performance. the C240 M3 is useless.

If you claiming otherwise please write.

Best Regards.

[Manuel Velasco]

Hi

What version of CentOS are you using?

What firmware is your server running?

In the past I've seen performance issues that are related to drivers.

Note that the first CentOS tested version for this server is 6.5, but your server must be running the latest firmware version (2.0.3).  Also according to the compatibility matrix below the only tested raid controller is the SAS 2008M-8i.  I would recomend that you try to install another Operating System that is supported by your server and the installed raid controller. 

UCS compatibility Matrix

http://www.cisco.com/web/techdoc/ucs/interoperability/matrix/matrix.html

[ibrahimd1]

Im Using a complete your system, not additonal upgrade hardware. 

Cisco Firmware 2.0(3i) 

Raid Controller LSI 9271-8i (Latest firmware 23.33.0-0018)

Im formatted our system and Create Raid50, test system. 

Raid50 Config
Strip Size : 128K
Drivers Per Span: 7
Span Depth : 3
Access Policy : Read-Write
Read Ahead : Always
Req.Write Cache: Write Back Good BBU
Current Write Cache: Write Back
Disk Cache : Enable

Make a Full Init. ~2 Hour. 

Create 1 virtual machine Centos 6.4 64bit, Install vmware tools. 

Test Results;

[root@84 ~]# hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads:  500 MB in  1.04 seconds = 479.76 MB/sec

[root@84 ~]# sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.14755 s, 936 MB/s

[root@84 ~]# dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.03335 s, 528 MB/s

working with exactly the same settings, there is another C240 M3, even under load, giving about 2 times more value. The features of the device settings raider everything exactly the same, the same day we received 2 hardware.

[ibrahimd1]

Another older DELL server R710

6 x 2 TB SATA, Writeback, read ahead, 128k strip size. 
Raid10, 50 active vm's. %70 cpu load, %85 memory load. 

Test results;
/dev/sda1:
 Timing buffered disk reads: 500 MB in  1.51 seconds = 331.30 MB/sec

I dont understand, older dell server include 6 sata disk, raid10, working 3 years, running 50 vm's. load full, hdparm test is 331.30 MB/S. Cisco 1 year server, new hardware, no load, fresh install, totaly sources free, include SAS disk, for example raid 10 : 480.01 MB/sec  and Raid 50: 479.76 MB/sec 

Whats the problem? 

[Manuel Velasco]

Wait, what is the OS installed on the server? (VMware esxi or Citrix CentOS?) note I am not talking about the virtual machine OS?

[ibrahimd1]

Hypervisor is ESX-i, Virtual Machine is Centos 6.5

[Manuel Velasco]

What is the ESXi version running running on this server?

What is the driver version installed for the raid controller on this server?

ssh into esxi host and run the following commands

vmware -lv

vmkload_mod -s megaraid_sas | grep Version

[ibrahimd1]

VMware ESXi 5.1.0 build-2323236
VMware ESXi 5.1.0 Update 3

Version: Version 6.602.54.00.1vmw, Build: 472560, Interface: 9.2 Built on: Nov 20 2013

[Manuel Velasco]

Take a look at this white paper below

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/unified-computing/whitepaper_c11-728200.html