C240 10GB network latency peaks - vsan

[Andreas Linde]

Hi,

On two C240 M4 hosts with VIC1225 and 1227 connected to a Cisco Catalyst over 10GB we get continuous high network latency warnings from VMware VSAN Health check.

When running vmkping between vsan vmk's it shows latency peaks over 20ms from time to time. Average latency is low (>1ms) but the peaks trigger cluster level warnings several times per day even though everything is working fine.

Any suggestions on what could cause this are welcome! Overall load on the hosts is low in regards to CPU, RAM and network.

[Walter Dey]

Hi Andreas

Questions

- is ESXi installed on local disks ?

- any Raid configuration ?

- which raid Controller

- NFS ? iSCSI ?

- which storage Array ?

- do you also see loss of data stores ?

Walter.

[Andreas Linde]

Hi Walter,

Details on the platform, note that the problem is with network latency:

- ESXi (6.0U2) installed on SD Cards

- 12G Raid Controller set to JBOD

- VMware VSAN

- 2*SSD, 8*HDD

- Datastores are working fine, no issues detected there.

[Walter Dey]

Hi Andreas

Do you separate storage and IP traffic, by means of the two I/O adaptors ?

Is the storage traffic L2 or L3 ?

Without further details, I think it could be caused by the server (raid) or the network.

I had a similar case, which was caused by a failing raid Controller.

see e.g. http://www.cisco.com/c/en/us/support/web/tsd-products-field-notice-summary.html

http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-c-series-rack-servers/products-field-notices-list.html

Walter.

[Andreas Linde]

Hi, storage traffic is L2 and dedicated to 1 of 4 vNICs. IP traffic however flows on all 4 vNICs.

We're also thinking network and have seen som crc errors on one of the switches, will follow that up.

Kind regards

[iachick]

Why not Nexus switches in this design?