Introduction
Niles Pyelshak is a Customer Support Engineer for Cisco TAC works in Server Virtualization Team. She has several years of networking and virtualization experience. She recently passed her CCIE Certification in Data Center (#44608). As a Network Engineer, Niles was accustomed to getting on the command line interface (CLI) to configure and build her network. In server virtualization it is a little different. She is always amazed at how servers are mostly managed by Graphical User Interface(GUI) interfaces and CLI. One can not underestimate the power of both interfaces and in this Webcast, Niles will leverage both GUI and CLI to demonstrate the UCS life of a packet.
The following expert was helping Niles to answer few of the questions asked during the session: Robert Burns. Robert is and expert in Data Center technologies and has vast knowledge in this topic.
You can download the slides of the presentation in PDF format here. The related Ask The Expert sessions is available here. The Complete Recording of this live Webcast can be accessed here.
Traffic Switching
Q: Traffic is never switched by IOM. Even for Vm in the same vlan on the same blade ?
A: Correct. IOMs are essentially MUXes. No local switching functionality.
Q: So FEX is the same link N2K which use vnTag technology, all switching must done at FI?
A: Correct.
Q: The switching between VMs are done by vswitch or dvs not by IOM or FI unless VM-FEX is configured in that case you have a virtual cable from FI directly to VM itself?
A: Correct. If there is a southbound switch before the IOMs, in the case of VMware vSS or vDS, then yes local switch is possible.
Q: Switching between vlans happens on upstream switch even if source vnics pinned to same FI?
A: Correct. There's no routing on the FI's, they are L2 only.
Hardware
Q: Niles said we can have 20 chassis or 160 C-series servers. What if I have both? What is the maximum for B-series chassis and C-series servers as well as the maximum number of FEX?
A: The 20 chassis limit includes the 5108 B-series chassis as well as FEXes.
Q: Do the FEXes count as 1 chassis each, meaning I can have a mix of 20 chassis and FEXes, correct?
A: Yes, FEXes would count as a Chassis. Keep in mind, though a FEX counts as a chassis, each can support another 32 10G connections. The reason we limit it to 160 servers or 20 chassis, is to limit the possible failure domain.
Q: What is the difference between the VIC 1240 with Expander and the VIC 1280?
A: The VIC1240 + Expander expands the bandwidth without creating another adapter on the host. The 1240 + 1280 would show up as two separate adapters on the host. Functionally they operate at the same total speed, just across one single or multiple ints.
Q: What kind of cable/connectivity we use between IOM and FI?
A: Twinax (copper) or Optic OM3 10G cable
Miscellaneous
Q: What if I have both B and C series?
A: Just choose B in that case. Not a big deal.
Q: Can we manage VM pinning for which FI?
A: Depends on the host. The OS will have an two interfaces, one to FI-A, one to FI-B. In the case of VMware, it would depend which Uplink you attach to the vSwitch or vDS. But yes you can manage this.
Q: Any idea does customer can run with YOSEMITE Y2S Y4S?
A: Can you elaborate what you're asking?
Q: I read some documents on Cisco UCS , but i am not sure does we cover these UCSM Y2S and Y4S int his session?
A: Are you talking about the 2 socket and 4 socket blades? or the blade we codenamed as "Yosemite"? I assume you're referring to the full width Intel Brickland platform known as the UCS B260 M4. This will not be specifically covered in this session, but we can speak to full width considerations.
Q: Port channel 1280 is the port channel between the IOM and the CNA ? (Traces) right ?
A: Correct. The internal traces are channeled, unknown to the host. That's what gives the 10G, 20G or 40G total bandwidth (depending on the IOM model)
Q: Do we loose connectivity during Chassis ReAck ?
A: Only to any blades that need to be re-pinned. If the pinning doesn't change, the server's networking will be unaffected.
Q: Port channels are also valid with the 2204 IOM and 1240/1280 adapters?
A: Correct. The IOM <--> adapter Port channels aren't configurable (automatic), but the FI <--> IOM is configurable. Discreet vs. PC mode.
Q: In what cases Discreet is preferred choice over PC ?
A: It's not. PC is a fat shared pipe, whereas discreet will guarantee at least 10G dedicated, but I've never seen any customer saturate 80G of bandwidth sustained. Always use PC if the HW supports it.
Q: How does the load balancing work if you have 2 or 4 link connection (IOM -> FI)?
A: If you have 2 IOM -> FI links, blades it would go Odds/Evens across the two links. 1,3,5,7 -> Link1, 2,4,6,8 -> Link 4. With four links its 2 per link, round robin.
Q: In which order do you usually check the path ? from the FI to the vNIC or the opposite way ?
A: I usually start from the host, and work outward up the IOM, then FI, to external from UCS.
Q: Since all FC/FCoE traffic comes down to the blade over FCoE, and is then broken out to either FC or FCoE to the host, how can you tell from the CLI or looking at the interfaces whether it's a FCoE or FC flow?
A: The VIF# will identify it as a vnic or vhba (eth vs. fc) traffic.
Q: Where is the emulator available?
A: The UCSPE (Platform Emulator) can be found on the developer network.
Q: There is an automatic port channeling between a 2208 and a vic 1280 right ? You can't specify which traces are you going to use right ?
A: Correct.
Q: FC/FCoE question. I realize Ethernet will come down differently, but doesn't a FCoE VLAN need to be tagged specially for the CNA to know it's a FCoE VLAN not ethernet VLAN?
A: Its auto tagged. When you assign an FCoE VLAN to a VSAN, it will take all the storage traffic with this VLAN tag for you. The FCoE VLAN ID is only significant with the UCS domain, it will never leave the UCS domain.
Q: I would like to integrate C series into ucsm.
A: That's completely possible. You just need a compatible Adapter.
http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm2-1/b_UCSM2-1_C-Integration/b_UCSM2-1_C-Integration_chapter_01.html
Q: Any plans of moving away from JAVA for UCSM? mostly after java upgrade UCSM stop working .
A: YES. UCSM 3.0 is HTML5 based.
Q: What is a proper way to do the upgrade ? from host side to FI or in an opposite way? Is it a good idea to use a Firmware Autoinstall in case of upgrading production system.
A: FW auto install is the preferred way, but otherwise we do an inside-to-outside manner. Starting with the inner most device (adapters) and work outwards to the FI's being last. The FW Upgrade/Downgrade guide details all the best practices.
Q: When can we expect UCSM 3.0 on FI62xx?
A: Hopefully by this summer.
Related Information
Cisco UCS Faults and Error Messages
Ask the Expert: Demystifying Unified Computing System (UCS) Interfaces for troubleshooting
Webcast Slides: Demystifying Unified Computing System (UCS) Interfaces for troubleshooting
