With Matt Blanshard and Jane Gao
Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to ask your toughest layer 2 questions to two of the technical leaders of the San Jose LAN Switching team, Matt Blanshard and Jane Gao. Learn more about Spanning Tree, VTP, Trunking, Resilient Ethernet Protocol, IGMP Snooping, Private VLANS, Q-in-Q Tunneling, QoS, various switching platforms including all desktop switches, Metro Ethernet switches, 4500 and 6500 switches, Blade Center switches, and Nexus 7000 switches.
Matt Blanshard began his Cisco career as an intern in 2007. He is now a technical leader at the Cisco Technical Assistance Center on the LAN Switching team. He holds a bachelor's degree from the University of Phoenix in computer science, and has CCNA certification.
Jane Gao is a technical leader in the Lan Switching Technical Assistance Center (TAC) team in San Jose. She has been working with LAN switching technologies and supporting Cisco switching platforms Jane's Bio since 2009. Ms. Gao was previously a technical leader in the Wireless TAC team in San Jose. Prior to joining Cisco Ms. Gao was working in software development. She has a Master of Science degree in Computer Science from DePaul University in Chicago.
Remember to use the rating system to let Matt and Jane know if you have received an adequate response.
They might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Lan Switching and Routing discussion forum shortly after the event. This event lasts through August 12, 2011. Visit this forum often to view responses to your questions and the questions of other community members.
Hello Matt and Jane,
It is a pleasure meeting you.
I have a question about the Catalyst 2960 platform. Over the past years, many features typical for 3550/3560/3750 have been brought to the 2960 series, such as Dynamic ARP Inspection, IP Source Guard, stacking (for 2960-S), static routing, etc. Considering this, I would like to ask:
Thank you very much for any information!
Good to meet you as well! Currently at this time there is plan to backport private vlan functionality to the 2960 switches. For question number two it looks like the VACL funtionality being slipped in is a mistake. VACL is still unsuported on the 2960 switches though it does appear to work, it is an untested feature.
Thank you very much for your answers. And please consider this as my personal expression of a huge interest in having both Private VLANs and VACLs supported on the 2960, should there be any doubt whether there is a customer interest for these features on 2960 Catalysts
There are currently enhancement requests in place for both feature in place. I will forward your interest as well along with this thread to the product team who handles the roadmap for these products and see if we can get them added. I suspect VACL wouldn't be a big deal for them to add as it looks like it already has happened on accident it would just require a commitment to test it.
Private vlan's is entirely a different matter as it introduces functionality that might not be possible with the current hardware. Again I will forward this request on and get it added to the enhancement request.
Thank you very much for passing these suggestions along. I am aware that both functionalities may require a specialized hardware support. In any case, thank you for clarifying this, and I am looking forward to see what functionality can and will be implemented.
Firstly, thanks for hosting this session.
I am just starting to read up on Fabric Path on the Nexus switches. I can appreciate the huge benefits it offers not just in terms of bandwidth but also in the flexibility of being able to extend L2 and have a much larger flat broadcast domain. I have always found L3 in the DC to be a limiting factor in terms of flexibility.
Where i am not clear is how you would intergrate services into this architecture. If you have one very large L2 flat network interconnected using Nexus switches this means you can literally have 1000s of servers L2 adjacent. So how do you firewall, load-balance etc. between the servers.
Take firewalling as an example. It is very common in a DC to segment servers eg. database servers. Now i can see how you could use a firewall in transparent mode but -
1) even your most powerful firewalls pale in comparison to the bandwidth we are talking about with fabric path
2) Nexus switches do not support service modules. So how would you "insert" a firewall between servers without losing the advantages of fabric path.
I may well have misunderstood some of the architecture so please feel free to tell me
I am really glad you asked this question since fabricpath is one of the things I love talking about
First when it comes to segmentation at layer 2 most of the reasons that there is segmentation at layer 2 in a traditional DC design is to limit the size of the l2 domain due to the many undesirable downsides of having a large flat l2 design. Since fabricpath is a way to eliminate these downsides it opens up the possibility to have any application talk to any other application without the need to involve layer 3 routing.
In a situation where you would need firewalling between the segments there is no way currently to firewall inside the fabricpath domain, so you will need to fallback to traditional layer 3 routing between segments if you want to firewall in between. As you pointed out even using a firewall in transparent mode would not be practical due to the huge amount of bandwidth fabricpath provides us with, along with the low latency switching which injecting a firewall into would kill.
As for your second question when it comes to load balancing fabricpath supports equal cost multiple path routing between multiple switches, so it can load balance between switches, up to 16 paths per mac address. If you are talking more like traditional Microsoft NLB and the like it fully supports multicast, but you would need to be sure to use IGMP mode on the servers to ensure that they are sending their IGMP joins so they get the traffic.
Hope this answers your question and I am clear on this post. If not reply on the parts you are unclear on and I will be happy to clarify further!
I am really glad you asked this question since fabricpath is one of the things I love talking about
Matt, you may regret that after my 100th question on it
Seriously though many thanks for the answer but could we go into this a bit more. When i talked about load-balancing i was thinking more server load-balancing ie. ACE module etc. To that list you could also IPS/IDS etc.
One of the key aspects in a DC is not just the sheer bandwidth/latency but services such as the ones mentioned above. Because one of the primary reasons for fabric path, at least as far as i can see, is to allow a much larger flat L2 network this means you can scale up to many 1000s of servers into one vlan in effect. But, speaking from purely personal experience, in all the DCs security is one of the major components. A simple example of a virus spreading for example is much easier on a L2 vlan with no protection.
So i may be missing something here, because to me a L2 domain that large without any sort of IPS/IDS/firewalling seems like a disaster waiting to happen. Yes you could fall back to L3 segregation but then you seem to me to be losing a lot of the advantage that fabric path gives you.
So i guess my follow up questions would be -
1) Do cisco see this as an issue and are there plans to look at firewalling/IPS integration into the Nexus switches. I suspect not because i'm not sure whether IPS or firewalling could ever actually keep up with that bandwidth ie. potentially it could always be the limiting factor
2) Appreciate you may not be able to answer this but what is the take up on fabric path so far and is security seen as an issue by these people
3) If Cisco do see this as an issue and traditonal stateful firewalling/IPS etc. are not the solution are there any other technologies on the horizon that could address some of the issues.
Or if this isn't an issue then please feel free to tell me !
Jon, keep the questions coming
As to address your question about firewalling this is my opinion and not Cisco's official position, but in order for firewalling to keep up we need a major technology leap to keep up with the bandwidth that fabricpath is allowing in the network. I don't know what/if plans are being made to address this, but I do see it as a potential problem. I just don't see in the near future producing a firewall that's capable of keeping up with terabits of traffic.
For the ACE side of things I do think there are ways to achieve load balancing without necessarily putting the traffic through the ACE engine so I do see that technology keeping up to some degree, but by using alternative methods of load balancing (round-robin arp and a few others).
Also to point out is that service modules for the Nexus are being developed, but I do am not privy to any of the performance details so my prediction on the firewall could also be totally wrong
Keep the fabricpath questions coming!
Service modules for the Nexus switches ! - I won't push you for any more information on that one
I agree about the firewalling/bandwidth issue and i suspect this will put more emphasis on end point securiity rather than using the network to secure the devices.
So moving on to my next set of questions !
I'm finding it difficult to find any docs that go into the specifics of how fabric path works. I have an overiew in the sense that it is basically implementing routing within L2 networks. The routing protocol it uses is IS-IS and in effect it uses switch tags or addresses as the source and destination ie. a packet from hostA to hostB would have a header added to the packet with the source address of the switch hostA is attached to and the destination address of the switch hostB is attached to. Is that correct so far ?
My initial thoughts were that IS-IS was used to exchange information about all available mac-addresses to all switches within the fabric domain with switch tags added to "routes" for each mac-address. But from one of the docs i have read on CCO -
Cisco FabricPath needs to learn at the edge of the fabric only a subset of the MAC addresses present in the network, allowing massive scalability of the switched domain.
I'm not sure i understand how this works. Does this imply
1) that the core switches within the fabric path "domain" need to know all mac-addresses present in that L2 network
2) if the edge switches only know a subset then how does an edge switch know which destination switch to use as the destination address in the packet. When an edge switch receives a packet from a device on the edge it presumably does a lookup for the destination mac-address and then assigns a destination switch tag to the header it adds. If it only has a subset of mac-addresses then how does it know how to address the packet if the destination mac-address is not part of this subset ?
Do you have any links that i may have missed to a more detailed explanation of how all this works ?
Once again, many thanks for answering all my questions.
This is going to be a long post
ISIS is used to establish routing adjacencies between switches. What is actually being advertised though is a bit different. Inside fabricpath we use what is called conversational mac learning. Broadcast frames do not trigger mac learning. Inside the core of the fabricpath there is no mac learning at all since all frames will have a destination switch id. The fabricpath core is very similar to a MPLS core where all forwarding being done there is label switching only. First I am going to go over how flooded frames are handled.
Inside the topology there will be two loop-free trees built which are used for multi-destination frames (broadcast, multicast, unknown unicast). One tree is typically used for all broadcast and unknown unicast while multicast is balanced across both trees. When a frame is received with an unknown destination mac it is sent out to the respective tree. All of the switches in this tree will receive this packet but only the switch which has the destination mac learned as a local mac will install the remote source mac of the frame. This means the edge switches will only learn mac addresses for traffic it is interested in instead of from every broadcast frame and greatly reduces the amount of mac learning required.
So a real scenario will look like this topology:
In this scenario even though the frames will be traversing both Sw1 and Sw2 they will never learn the mac of either Host A or Host B. ISIS is mostly used to establish the trees, handle multicast forwarding, and create switch id's to forward the frames to. You can also see that in a true core situation where there are dedicated core switches they will never learn any mac's because they will never have any mac's as local.
Unfortunately all the documentation that I have is internal to Cisco, but I know there are some white papers coming out on this which will explain it further. As you can see though this greatly reduces the overhead on switches and since fabricpath uses ECMP it will load balance traffic from Host A and Host B through sw1 and sw2.
Excellent reply, thanks very much.
I'm going to take a bit of time to absorb it but just to be absolutely clear in my head a few quick questions -
The fabricpath core is very similar to a MPLS core where all forwarding being done there is label switching only
i understand what you mean in the sense that with MPLS the routes outside the MPLS cloud are not known to the P routers in the same way the core switches in fabric path do not need to know the mac-addresses of hosts attached to the fabric path edge.
However in an MPLS network the tags are actually changed at each P router hop as they go through the MPLS network. I'm assuming there is no actual tag changing in fabric path ie. each core switch does not swap a tag, it simply looks at the destination switch tag and forwards it ?
One other thing is not entirely clear. Although fabric path builds a loop free topology using IS-IS which means no STP needed, broadcasts etc.must still be flooded to every switch within the fabric topology. So every switch must still process every broadcast sent. So when you talk of greatly reducing the overhead of switches are you talking purely about the cam tables switches maintain or is there some additional benefit i am not seeing ?
No tag switching inside the core just looks at the destination switch id and forwards it accordingly. As for the reduction it's only in the amount of mac's that have to be learned since there's no way to stop broadcast/unknown unicast without breaking basic network functionality.
Hi Matt & Jane,
You had been assigned as subject matter expert till August 12th in public forum also you are from the TAC. Your intro page start out as questions to raise in this area "Spanning Tree, VTP, Trunking, Resilient Ethernet Protocol, IGMP Snooping, Private VLANS, Q-in-Q Tunneling, QoS." I know Cisco had tons of information in the web but you must have favorite link for your area of expertise to support TAC issue.So, to benefit audience, Please provide URL pointer in a single page for your technology above for the following questions:
1) How is the handshaking protocol for "X"?
2) What you need to configure to work for X?
3) How do you verify with show command and debug to check your protocol X handshake?
4) Recover outagage issue in shorter time for each technology to follow the "Flow Chart" poster.
You may already have this or you have a good two weeks to research and put in one ducument that will help entire audience for any questions to arise. Update often as any NEW features introduce. Your TAC manager will appreciate your hard work as it will cut down tons of un-warranted tickets.