On two C240 M4 hosts with VIC1225 and 1227 connected to a Cisco Catalyst over 10GB we get continuous high network latency warnings from VMware VSAN Health check.
When running vmkping between vsan vmk's it shows latency peaks over 20ms from time to time. Average latency is low (>1ms) but the peaks trigger cluster level warnings several times per day even though everything is working fine.
Any suggestions on what could cause this are welcome! Overall load on the hosts is low in regards to CPU, RAM and network.
- is ESXi installed on local disks ?
- any Raid configuration ?
- which raid Controller
- NFS ? iSCSI ?
- which storage Array ?
- do you also see loss of data stores ?
Details on the platform, note that the problem is with network latency:
- ESXi (6.0U2) installed on SD Cards
- 12G Raid Controller set to JBOD
- VMware VSAN
- 2*SSD, 8*HDD
- Datastores are working fine, no issues detected there.
Do you separate storage and IP traffic, by means of the two I/O adaptors ?
Is the storage traffic L2 or L3 ?
Without further details, I think it could be caused by the server (raid) or the network.
I had a similar case, which was caused by a failing raid Controller.
see e.g. http://www.cisco.com/c/en/us/support/web/tsd-products-field-notice-summary.html
Hi, storage traffic is L2 and dedicated to 1 of 4 vNICs. IP traffic however flows on all 4 vNICs.
We're also thinking network and have seen som crc errors on one of the switches, will follow that up.