Welcome to this Cisco Support Community Ask the Expert conversation. This is an opportunity to learn and ask questions about utilizing Cisco data center technology and solutions with subject matter expert Ramses Smeyers. Additionally, Ramses will answer questions about FlexPOD, vBlock, Unified Computing Systems, Nexus 2000/5000, SAP HANA, and VDI.
Ramses Smeyers is a technical leader in Cisco Technical Services, where he works in the Datacenter Solutions support team. His main job consists of supporting customers to implement and manage Cisco UCS, FlexPod, vBlock, VDI, and VXI infrastructures. He has a very strong background in computing, networking, and storage and has 10+ years of experience deploying enterprise and service provider data center solutions. Relevant certifications include VMware VCDX, Cisco CCIE Voice, CCIE Data Center, and RHCE.
Remember to use the rating system to let Ramses know if you have received an adequate response.
Because of the volume expected during this event, Ramses might not be able to answer every question. Remember that you can continue the conversation in the Data Center Community, under the subcommunity Unified Computing, shortly after the event. This event lasts through August 1, 2014. Visit this forum often to view responses to your questions and the questions of other Cisco Support Community members.
I have dozen questions but will try to restrain myself and start with the most important ones :)
1. Can cables between IOM and FI be configured in a port-channel? Let me clarify what I"m trying to achieve: if I have only one chassis with only one B200M3 blade inside, will the 2208 IOM and FI6296 allow me to achieve more than 10Gbps throughput between the blade and the Nexus 5k? Of course, we are talking here about clean ethernet environment.
B200M3 --- IOM2208 --- 4 links --- FI6296 --- port-channel (4 links) --- Nexus5548
2. Is it possible to view/measure throughput for Fibre Channel interfaces?
3. Here is one about FlexPod: I know that in case of vBlock there is the company that delivers fully preconfigured system and offers one universal support point so customer don't have to call Cisco or VMware or storage supports separately. What I don't know is how it works for FlexPod. Before you answer that you are not sales guy, let me ask you more technical questions: Is FlexPod Cisco product or is NetApp product or this is just a concept developed by two companies that should be embraced by various Cisco/NetApp partners? As you obviously support Datacenter solutions, if customer/partner calls you with are FlexPod related problem, does it matter for you, from support side, if you are troubleshooting fully compliant FlexPod system or you'll provide same level of support even is the system is customized (not 100% FlexPod environment)?
4. When talking about vCenter, can you share your opinion about following: what is the most important reason to create the cluster and what will be the most important limitation?
5. I know that NetApp has feature called Rapid Clones that allows faster cloning than what vCenter offers. Any chance you can compare the two? I remember that NetApp option should be much faster but didn't understand what is actually happening during the cloning process and I'm hoping you can clarify this? Maybe a quick hint here: seems to me it will be helpful if I could understand the traffic path that is used in each case. Also, it will be nice to know if Vblock (i.e. EMC) offers similar feature and how it is called.
6. Can I connect Nexus 2000 to the FI6xxx?
7. Is vBlock utilizing Fabric Failover? Seems to me not and would like to hear your opinion why.
Thanks for providing us this opportunity to talk about this great topic.
1: yes, between IOM and FI you can use a port-channel, this will allow 1 blade to have more then 10Gbit/s bandwidth, although off course 1 flow can only use 10Gbit/s
2: yes, you can do this on the nxos part of the FI (login to UCSM via ssh and type 'connect nxos a' or b) Here you can do 'show interface fc x/y' or 'show interface vfc x' (use 'show interface brief' to get the correct interface)
3: FlexPOD is a Solution created between VMware, Cisco and NetApp based on Cisco CVD's, see for example http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/esxi51_ucsm2_Clusterdeploy.html If you buy normal Smartnet support, Cisco will support all your Cisco components and collaborate with VMware and NetApp to provide the support if it's outside the Cisco stack. If you buy SSPT support for DCI we will prime FlexPOD support and provide full frontend FlexPOD support. FlexPOD allows for customization but we encourage customers to stick to the CVD's.
4: Creating a cluster will allow features such as HA and DRS to function, this will greatly impact your SLA, so hence, cluster functionality is required in regards to availability of your workload
5: when cloning we can use 2 methods, either the host pulls in all the data and then pushes it back to the storage, hence, the host clones it. The other way is sending a signal to the storage through VAAI and hence tell the storage device to clone a block or file, this scenario will off course be a lot faster
6: Yes, you can do this for UCS C-Series integration into a UCS-B series setup, see also http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm2-1/b_UCSM2-1_C-Integration.html
7: vBlock is primarily designed to run VMware and as we have a best practise to not use adapter-failover with using VMware, hence, vBlock is mainly not using this feature. This is too allow visibility of the event into vCenter
I'm especially grateful to the answer related to FlexPod as it is not completely technical. I'll appreciate if you can ealborate how to create port-channel between FI and IOM as I didn't get that part. If it is too much for you to write, give me some key commands or the link that explains it and I'll hopefully figure out the rest.
Returning back to the VMware cluster: if I have UCS domain with 5-6 chassis, each fully populated with B200M3, what will be your choice of clustering? Where do you see problems if there is too many small clusters and similar, what will be the problem having only one (or two) cluster(s) for all the blades? If you need more realistic scenario, let's imagine it is a Vblock or FlexPod.
And the last question for this round :) If you need to capture traffic inside UCS system (for example 6 chassis, FI6296, IOM2208, VMware distributed switch instead of N1K etc, let's suppose it is Vblock solution), what will be the perfect place to do it? I'm not sure if VMware distributed switch can do port mirroring (let's suppose it is vSphere version 5.5). Another issue I see is vCenter that is sitting behind Cat3560 which provides "only" 1Gbps interface so in case when two VMs inside same VLAN are exchanging huge amount of traffic it would saturate the port on the switch and effectivelly block the vCenter.
To enable port-channel between IOM and FI, have a look @ http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/gui/config/guide/2-0/b_UCSM_GUI_Configuration_Guide_2_0/b_UCSM_GUI_Configuration_Guide_2_0_chapter_01100.html#d108099e588a1635
For the VMware cluster there is not an easy answer. Things such as recovery time, sla, licensing, failure domain, mixing env (dev/qa/test/prod), EVC mode, ESXi cluster restrictions etc... come into play.
Interesting reads on this subject are: http://www.amazon.com/VMware-vSphere-5-1-Clustering-Deepdive/dp/1478183411/ref=la_B002YJMRCY_1_2?s=books&ie=UTF8&qid=1406771337&sr=1-2, http://www.yellow-bricks.com/2014/02/19/dont-create-frankencluster-just-can/, http://www.yellow-bricks.com/2012/04/10/cluster-sizes-vsphere-5-style/
From Cisco side any size works, as long as you take VMware rules into account.
To sniff traffic you can either use a SPAN on UCS level (you can SPAN up to veth level if it's a B-Series). Else you need to sniff on VMware level with a promiscuous port-group, see also http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1000880 and http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1038847. Off course with the use of N1K this will be a lot easier.
When you sniff you typically will do this on an internal VM, hence you should not have a traffic limitation. And even when you sent it external (think with an FI span), you can span to a 10Gbit port so hence you should not oversubscribe the interface.
We upgraded Nexus 5548p from NX-OS 5.2(1)N1(4) to
NX-OS 5.2(1)N1(7). We have a dual-homed Nexus 5k to 2k fex. When the primary nexus 5548 did a reboot from a disruptive software upgrade we seen network disconnects for about 40 secs before the secondary took over. We did not see this behaviour during a reboot from a bug in NX-OS 5.2(1)N1(4) of show version. Is there a difference in the upgrade reboot? What would cause the network disruption? We cannot look at logs because did a reload power-recycle to upgrade power seq.
When performing an update, you should verify "show vpc" and look for the following lines:
vPC domain id : 53
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 273
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Also make sure you upgrade through the "install all" command.
If all of the above is being followed, I wouldn't expect any issues as you described and if so, probably you want to open a TAC service request to have a further look into this.