Direct connect NFS/CIFS with 1.4

[cweinhold]

I see how 1.4's appliance ports can help with direct connect multipath iSCSI. Each fabric gets a VLAN/subnet, iSCSI has multiple targets via each fabric, etc.

But I don't see how appliance ports help with direct connect NFS/CIFS. E.g.,

In this diagram, traffic from hosts active on fabric B must go over the northbound LAN to reach the NAS. Even if every host and appliance has its NIC failover configured to prefer fabric A and perform preemption, you'll still have failures like NAS NIC's and IOM's that will cause some/all NAS traffic to go across the northbound LAN. Thus, you've got one of two situations:
    1) The LAN can handle the NAS traffic. If so, why not plug the NAS into the LAN in the first place?
    2) The LAN cannot handle the traffic, in which case you haven't built real fault-tolerance and, worse, a UCS problem can impact the LAN.

Am I missing something here? How are appliance ports better than using switch mode, shown below?

[Manish Tandon · Accepted solution]

Yes - you have a valid point on the traffic pattern depending on what fails etc.

Just to back up for a moment..

Version 1.3 and below has 2 types of Ethernet ports - Uplink and Server port.

Version 1.4 has 4 types of Ethernet ports - Uplink, Server, Appliance, Monitoring port

In 1.3, to directly connecting a NAS to the FI's meant that you move to switch mode on the FI's and then set the port connecting to the NAS as an uplink port. What that did is that the port connecting to the NAS was a a trunk port at the FI end allowing all VLANs (i.e no VLAN filtering), no QoS settings etc.

So if you wanted to stay in End Host mode AND not liking the above caveats, you connected the NAS to an upstream switch and not UCS and that option remains wit you today.

What the appliance port gives you now is that the VLAN(s) the NAS belongs to can be filtered, QoS settings possible and most importantly, it works in End Host Mode (most deployments are based on it).

The above is the rationale for the Appliance port and the port type was neeeded even if it works today in switch mode.

Now the question comes, appliance port in EHM or Switch mode (which is what the question is).

In EHM you are right, east-west traffic between NAS-blades could utilize the upstream network.

You can design efficiently by specifying the fabric id (A or B as primary) or set fabric affinity if using a soft switch but  guaranteed total localization (not using upstream network) cannot be made as you correctly said depending on "what" fails.

If all the uplinks on A fail, yes the whole thing should fail over but if a link between the IOM and the FI fails, then the servers pinned to that link will start using the external network.

So yes, the network needs to be designed keeping the flows and what if scenarios in mind. East-west traffic not hitting upstream at all cannot be assumed.

The long term solution is to have data links between the FI's in EHM or they are vPC peers and hence both links to the NAS from the FI's will be active/active.

Appliance port in switch mode can be used but that also depends on which links are STP blocked etc to guarantee that.

The topology you mentioned does that ..but then you also need to keep in mind on failurea what happens etc i.e the ISL between the FI's should always be forwarding for that VLAN.

Thanks

--Manish

[cweinhold]

Thanks, Manny. My takeaways are that:

• appliance ports in EHM are only suitable for multipath iSCSI.
  
• appliance ports in switch mode can be pretty handy. But for STP sanity's sake, try to keep the appliance VLANs local to UCS.
• (it's too bad there's no checkbox to control BPDUguard, because appliance ports would be great for dealing with the disjoint network problem...)
  

There's a 95-slide UCS 1.4 slide deck going around that you've probably seen. It repeatedly states that UCS 1.4 supports direct connect NAS without switch mode. These claims are misleading, at best, and will generate months of wrong assumptions. I've already heard of UCS designs with GE LAN uplinks and 10GE direct connect NetApp NFS. That could be a disaster.

If you have any pull in the SAVBU, could you try to get this caveat called out? Maybe make a distinction between multipath iSCSI and NFS/CIFS, or stress the need for a robust upstream LAN that is at least equal in capacity to the appliances.

Thanks again,

-Craig

[Manish Tandon]

Craig

Thanks for the feedback.

I do disagree with you on few points you mentioned.

If a network is poorly planned, it is poorly planned.

The 1 Gig uplink connectivity option was added later in UCS to get around the fact that some customers did not have the 10 Gig density upstream.

It is an additional knob provided and if used, traffic pattern needs to be kept in mind.

Setting aside appliance ports for a minute, what prevents a user to set up a NAS server on a blade (or any other bandwidth intensive app in or out) and have just one 1 gig uplink? Similarly, if appliance ports make sense to you in switch environment what prevents a user from having one 1 gig as the ISL between the two? That link can fail too.

Appliance ports can be used for external (non UCS) servers to access the data store. They could be generating a lot of traffic even if you disregard the east-west storage-UCS blade traffic.

UCS provides you QoS capabilities. You can specify what bandwidth NAS traffic should get capped to when congestion hits so that your other traffic is not affected. Using thresholds/alarms/snmp etc you can monitor the contention etc.

This topology we discused is single-vif. Then you have multi-vif etc.

Network needs to be designed with everything in mind.

I do work for the BU and have visibility into the roadmap. You did mention disjoint L2 etc. Feel free to contact me offline (mtandon@cisco.com) and I should be able to fill you in on what is slated down the road.

Thanks

--Manish

[dani_bosch]

Someone inside Cisco told me a NAS appliance directly connected to UCS would only be accessible from a UCS blade server, never from outside, the same way as directly connected FC storage. From the aforementioned scenario though, I see NAS appliance IS accessible from the outside world. Am i right?

Thanks,

[Nuno Filipe Alves Ferreira]

A NAS Apliance connected to the UCS WILL be accessible from the outside world if the UCS system is connected northbound to your LAN and the VLANS on the NAS are also going Northbound ....

So, assuming you have the correct routing and switching in place to access the NAS Appliance / Shares then UCS will not block access from outside devices / workstations

Cheers

Nuno Ferreira

[dani_bosch]

I assume that's also correct for End Host Mode.

Thanks a lot Nuno,

Dani

[Nuno Filipe Alves Ferreira]

Correct

Nuno Ferreira