Post a node reboot the datastores look disconnected from HX and vCente

[josip-kelava]

Hi all,

we are facing same issue (described in CSCvk25616) with ESXi 7.0.3 and HyperFlex 5.0.2.g. In our setup we have 2x datastores and after reboot only one is visible where we have ESXi system logs. The datastore which is missing after reboot holds VM files. 

We reported it to Cisco but still no news about it. Does anybody knows if same bug is somehow present again in 7.0.x versions ?

[Kirk J]

Your referenced bug actually points to an ESXi bug.

Does the ESX cli show/not show the NFS datastores?

esxcfg-nas -l

and

df -h

should show if the ESXi hosts actually see the datastores or not.

IF the Cli command do show the datastores, than something is out of sync with ESXi and Vcenter.

If you DON'T see the nfs datastores listed in cli commands, then see below troubleshooting commands:

Try running the following esx command:

configstorecli config current get -c esx -g storage -k nfs_v3_datastores

You may see something like:

   {

      "iorm_state": 0,

      "latency_threshold": 30,

      "volume_name": "YOURdataStoreName"

   },

Typically, on a working system you would see 5/6 lines of attributes for each NFS datastore.

In your scenario you will only likely see the 3 lines like what I posted above.

"IF" you have no functional NFS datastores mounted, try running the following command:

configstorecli config current delete -c esx -g storage -k nfs_v3_datastores --all

Now re-attempt to remount the datastores via the HX connect, datastore UI.

There are a lot of environmental variables, that I can't cover in a single post like this.

Things like incorrect or missing HX VIBs in esxi hosts can also contribute to datastore mount issues.

These kinds of issues are normally best resolved in a TAC case.

Kirk...

[josip-kelava]

HI Kirk,

many thanks for your reply. We've contacted TAC and we have open case (SR 698203664), but unfortunately your colleagues didn't find still the root cause. We've involved Broadcom as well and they founded that problem can be in HX VIBs or in timing when they are activated. Therefore is Cisco now checking that.

What we found strange that after adding new third datastore (before upgrade we had two datastores), the problem is not coming. But again, we are still waiting on Cisco to provide us more informations why is this happening.

We founded this bug and because is almost same issue which we have, we thought to ask it here as well.