try to modify  Network failure detection to becon probing this way ESXi will be able to detect when a link is down from SW side.   when ink fails to 100 this will take it out of PO, but from esxi perspective, its still up so no failover will be done ...

Hi, I have seen similar behavior were the  MGMT PG did not inherit the Vswitch load balancing. try to check MGMT port Group configuration  and check if it match Vswitch0