A free efficiency boost for ISE in a VMWare deployment

[Arne Bier]

Hi

I am interested in ensuring that my ISE instances are running optimally in the VMWare hypervisors.  In my own labs I use VMWare Workstation and Player, as well as ESXi.  I don't always have an unlimited supply of CPU resources, and therefore I am keen to try and get away with using resources sparingly.  BTW I apply this to all my VM's (Windows and Linux)

VMWare Tools should be a no-brainer

It's fair to say that VMWare provides a reliable and robust hardware abstraction that allows our VMs to run quite nicely.

If possible, I always install the VMWare Tools in all of my VM's to ensure that the hypervisor can hook itself into the guest OS and enhance its performance.  I even do this for Linux guests because the open-vm-tools don't allow folder sharing and don't play nicely with vCenter.

But simply installing the Tools is often not enough.  There are two enhancements that lie dormant unless you do something to enable them

1) Networking paravirtualisation (VMXNET3)

2) SCSI paravirtualisation (pvscsi)

Why paravirtualise these things?

The answer is simple - CPU offload.  Because the hypervisor has to work hard at pretending to be the hardware adapter and it burns CPU cycles to achieve that.  By default VMWare emulates the Intel e1000 network adapter because this adapter is universal to most OS's - it works out of the box.  But it comes at a price.  The answer is to install the VMWare Tools and then switch from e1000 to vmxnet3.

In Workstation/Player I do this by editing the VM's .vmx file while the VM is shut down (in ESXi you can do this via the vSphere GUI).

After restart you'll see the vmxnet3 driver installed and also the link speed advertised as 10Gbps instead of 1Gbps.  I don't expect it to behave like 10Gbps but the performance improvements reported to double the throughput - and to lower the ESXi's CPU load.

The other often overlooked enhancement is the paravirtual SCSI adapeter - pvscsi.  Most default VM installs use lsilogic or something similar (depending on whether it's a Windows or Linux OS).  If you have a Linux OS and VMWare Tools installed, then you can replace the

scsi0.virtualDev = "lsilogic" with

scsi0.virtualDev = "pvscsi"

VMWare reports a 10% reduction in CPU and around 10% performance improvement.

Be careful when doing this on Windows OS's because they first need to install the driver before boot up - the typical trick is to install a dummy SCSI controller (i.e. scsi1:0:0) and change that controller's type to pvscsi.  This forces Windows to load the driver (automatically)while still keeping your primary disk operational.  Once the driver is installed you can shut down the guest and make the primary scsi controller of type pvscsi.  Remove the dummy controller and boot the OS.

VMXNET3 works on ISE

As far as ISE is concerned, the VMXNET3 adapter is officially supported and documented.  I have used it and it works well.  Check the install guide about the interface naming when using VMXNET - you have been warned

What about PVSCSI on ISE?

But I was unable to get the pvscsi working (ISE didn't boot up).  Does this mean that the VMWare Tools are not completely installed as part of the application bundle or is the kernel support not enabled for that?

I am keen to see an improvement because when ISE boots up it pegs the CPU at the same time that the HDD is being hammered.  It would be interesting to see the impact of implementing pvscsi in ISE.

[hslai]

If I read it correctly, you have only one question and it's on pvscsi. Now my question back to you... did you re-image ISE with its install ISO after the adjustment?

ISE has added this support for VMware Paravirtual SCSI in release 1.2.1. And, I've been able to use it since.

[Arne Bier]

Hi

I didn't reimage ISE after I edited the .vmx - I just started the VM up again.  This works for my other Linux VM's (CentOS 7) as well as Windows Server 2012 R2.

I was expecting the kernel module to be loaded during boot up ?

I have seen the pvscsi mentioned in the ISE installation guide - but it's mentioned in the context of ESXi (and I am using VMWare Player in my test lab)

Too bad I cannot get to the Linux CLI to perform a modprobe :-(

[Arne Bier]

I added a dummy SCSI controller to my ISE node and then edited the .vmx and changed the type to pvscsi

At least this time I was able to boot ISE because I didn't touch the primary SCSI controller.

Looks promising ...

But after setting this on the primary SCSI controller the VM doesn't boot up

But the new dmesg logs still indicate that the driver was loaded (or so it appears to me - the timestamps don't have a date on them).

[hslai]

ahci appears needed for pvscsi but blacklisted when the installation using other SCSI controller types. Installing ISE on HW appliances, such as SNS 3415, does require to blacklist this particular driver.

On an ISE VM imaged with pvscsi, I was able to flip the controller type to LSI Logic and to boot ok. It's possible to log a bug for lsilogic -> pvscsi but I can't see it valuable enough to push for the fix.

[Arne Bier · Accepted solution]

thanks for confirming this - I guess the alternative as you suggested earlier, is to setup the virtual hardware with all the paravirtual IO devices (VMXNET3 and pvscsi) before running the install from the .iso - I tried that and the install worked.