Cisco UCS B200M5 with FI6454 - private non-routable IP not reachable

[JeddGo80482]

Hello community

Hope someone can shed some light on this topic.

So we have a UCS B200M5 with FI6454. We are using this in a FlexPod environment with a NetApp AFF NFS storage. We have configured for service profile with 2 vNIC templates for NFS storage. We have assigned VLAN ID - 2001.

The IP ranges we are using for storage vmkernel are non-routable IP's.

Issue we are encountering

1. we can not reach any other host on that vmkernel VLAN, hence not reaching the storage also. 

2. 

QUestions:

1. using non-routable IP's. is this considered a PVLAN implementation? i dont think so, but please do correct me

2. does anyone have any documentation on how they have implemented using non-routable IP's for NFS storage vmKernel

3. our network team has defined the VLAN ID from uplink ports all the way to LEAF switch. Is there anything else that we are missing?

Any help would be greatly appreciated. We are at a standstill on this roll-out. 

Thanks

[Kirk J · Accepted solution]

Greetings.

Is your netapp attached to the FIs, or to your upstream switches?

If everything NFS is L2 adjacent, and using the same subnet, then there would be no need for L3 connectivity.

If your various NFS VMK ports on esxi can't ping each other, then I would do a sanity check on your VNIC and VMNIC numbering to see if your vnics (and the allowed 'NFS' vlan) actually match up to the VMNICs associated with your NFS VMK/vswitch.

Log into the UCSM, and 'connect nxos a|b'

#show mac add vlan xx  (where xx is your NFS vlan you expect you esxi host's NFS vmk's to be on)

If you don't see the mac's of your NFS vmk's in the right vlan, then you know something is off with VMNIC IDs and the VNICs you think they are associated with.

Kirk...