Start with the following steps:
1) Navigate to the service profiles within UCSM (Servers/Org/Profile.)
2) In the network tab, note the MAC address of the vNIC(s) in question.
3) Double check the MAC in use at the OS level. Method will differ depending upon the OS.
4) SSH to each fabric interconnect directly.
5) Run 'connect nxos'
6) Run 'show mac address-table | grep <insert MAC here>'
The show mac address-table command can be used across the entire L2 network to ensure that the MAC addresses are being learned through the L2 domain. There should be a MAC address entry at every hop between these two end points.
You'll need to verify that each MAC (transmitter and receiver) are learned on the proper fabric interconnect in the correct VLAN.
If you are correct and one VM is passing traffic through FI-A and one through FI-B, the UCS needs to leverage the upstream network in order to complete the transaction of data.
If the two VMs are in different VLANs, each VLAN should have a gateway on the router that is responsible for inter-VLAN connectivity, determine this gateway and attempt a ping from each VM.
If each VM can ping the respective gateway but not one another, your issue is with routing. This is a simple example of troubleshooting this sort of connectivity issue. The steps listed above should give you a clearer picture of the situation.
My best guess would be that the two VMs are not passing traffic in the same VLAN and the issue lies upstream of the UCS domain.
