UCS with ISCSI storage issue

[engmo83]

Hello All,

I am new to UCS

we have an issue to reach ISCSI  storage form a ESXI 7u2  installed on  a  UCS

I am using appliance port configured as access mode and connected directly to ISCS storage.

each fabric is connected to one SP directly (no switch no router and MTU is 1500 one both sides ).

appliance port using VLAN 10.

I add also VLAN10  to the VNIC template which is attached to service profile for every server.

the storage is VNX5300.

can nay one help me to solve this issue

[Steven Tardy]

What are you trying to do with the iSCSI storage?

Sounds like you are simply trying to connect to the iSCSI target on the VNX as a VMware ESXi datastore.

If you connect directly to each Fabric Interconnect (FI) and the destination IP is connected to the "other" Fabric Interconnect (FI), then the traffic must go upstream. Was VLAN10 allowed upstream?

Can you `ping` the iSCSI targets?

What are the MAC addresses of each endpoint? Are those MAC addresses learned on each FI?

As a best-practice for a "HA" iSCSI topology you should have two VLANs/IP-ranges one on FI-A and one on FI-B. That provides redundancy in case one endpoint fails and has the added benefit of keeping the network traffic "local" to each FI.

After that it's just configure from the two endpoints (VMware ESXi and VNX) like you would any other iSCSI target.

[engmo83]

Hello Steven,

yes right we are trying to attach datastores form  VNX storage.

your second question: do you mean uplink port by upstream ?

no I can't ping the storage

No mac address is not added to the appliance port. is this mandatory ?

in ESXi host I add 2 vkernal ports and I bind it to software iscsi

Thanks

[Steven Tardy]

Sounds like a Layer 2 issue.

You should not need to manually configure the MAC Address on the Appliance Port.

Does arp complete? Does ESXi have an arp entry (`esxcli network ip neighbor list`) for the iSCSI IP?

If not then maybe the cables are flipped?

Or maybe the VNX is trying to "tag" to VLAN and UCSM is expecting "untagged" (or vice versa).

Have you gone through this doc?

UCS Appliance Port Connectivity and Troubleshooting:

https://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-5100-series-blade-server-chassis/116075-ucs-app-connectivity-tshoot-00.html

[engmo83]

Hello Steven,

yes I already read  this Doc

this command : 

show mac address-table vlan 10

 on FI - A there is no any MAC address

in FIB  it shows only one MAC for the storage

I think I have two issues here first is FI - A is not learning MAC for storage and second is I should see  VINCs MAC address in this VLAN right ?

[amobrown]

If you are using VLAN 10 for appliance port traffic, you have done the right thing by adding VLAN 10 to the VNIC that needs to reach the Appliance port.

The step many overlook is that you also need to have VLAN 10 on your network uplink(s), even if traffic for VLAN 10 will never traverse the uplink, and always be directed to the Appliance Port.  It sounds counter-intuitive but it's a requirement for the VNICs to pin and come online - once the pinning happens, the MACs for the VNICs should be learned on the FI on the proper VLAN.

VLAN10 does not need to be configured on the northbound switchports connected to the Fabric Interconnect for pinning to work.  If not configured on the northbound switch, traffic tagged with VLAN10 would likely be dropped on ingress to that device.  If you want VLAN10 to traverse that northbound switch, you'd need to configure the VLAN/Trunk there as well.

You might also want to look at Network Control Policy for the VNICs.

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/gui/config/guide/2-0/b_UCSM_GUI_Configuration_Guide_2_0/b_UCSM_GUI_Configuration_Guide_2_0_chapter_010100.html#d67076e1739a1635

Many change the 'Action on Uplink Fail' to 'Warning' - this will keep the appliance port connection online even if the network uplinks go down.  

The caveat here is that the network uplink must be up for the vnics to come online initially, but after that, the uplinks can change state without downing the appliance port connections.