Microsoft NLB on 6509s

chris.smailes · ‎11-14-2012

We have a working MS-NLB cluster (I don't like MS-NLB but it wasn't my choice) which uses multicast with igmp. In reading on how to set this up I came across the fact that unless static mac address entries for the multicast mac address are added to the 6509 (with the disable snooping option), then if an SVI exists for the cluster's ip net on the 6509, traffic will be punted to the cpu. I have seen this behaviour when testing. My question is just - why does the 6509 have this behaviour ? Is it considered a bug ? The traffic clearly doesn't need to go to the cpu as adding the static mac entry restricts it to just the ports which have cluster members attached which have joined the igmp group. Thanks for your time.

danabersoch · ‎11-14-2012

I have had to confiure static ARP and mac entries in an attempt to make NLB be easier on the switched network quite a few times.

My understanding is that because the mac address is learnt from different ports (downlinks to you access switches) the switch will never keep a mac entry in its mac-address table so whenever it has traffic that is destined to the NLB mac it is always treated as a unknown unicast and the frame is sent down every forwarding link in that vlan (flooded).

When you add a static mac entries they will always stay in the mac-address table so the switch has a path to reach that host and so easier doesn't need to replicate every frame and flood it down every link in that vlan.

There are a load of docs on this exact subject if you google MS NLB Cisco network but I think you already know this information and your query is more 'why', try looking at the below doc to see if that helps your understanding:

http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00801d0808.shtml

**** Remember to rate helpful replies :-)

chris.smailes · ‎11-19-2012

Thanks for the reply.
When you use igmp snooping, the traffic is constrained as expected
except that the router port is added:

vlan mac address type learn age ports
------+----------------+--------+-----+----------+--------------------------
Active Supervisor:
* 130 0100.5e7f.83c9 static Yes - Po1,Po2,Router

The igmp-snooped mac addresses are learnt but added as statics - which can be misleading.

When a manually added mac entry is made (with disable snooping) the table shows:

vlan mac address type learn age ports
------+----------------+--------+-----+----------+--------------------------
Active Supervisor:
* 130 0100.5e7f.83c9 static Yes - Po1,Po2

So my question is, why in this case does igmp snooping add the Router port ?
As far as I can see this shoudn't happen because the cluster members joining
the group have otherwise been correctly snooped as being on Po1 and Po2 - so why add the router port ?

Kevin Dorrell · ‎11-14-2012

The problem with NLB is that it (ab)uses a multicast MAC address to get the switch to flood the traffic to all ports.

If you are accessing the service from another VLAN, then you have a problem because the router will ARP for the service IP address, but when the service replies with a multicast MAC, the router does not believe it. That is why you have to add a static ARP entry in your router. The MAC address is 01:00:5e:7f:xx:yy, where xx and yy are the hex expression of the last two octets of the IP service address.

If you have IGMP snooping, then the NLB tries to restrict the scope of the flooding by faking an IGMP in order to set up CAM forwarding entries for the multicast address. In this case the physical servers will generate IGMP reports for 239.255.xxx.yyy, where xxx and yyy are the last two octets of the service address.

That is fine on a 4500 and (I presume) a 6509, where IP multicasts are forwarded on the basis of the destination MAC address. It does not work on a 3750 or 2960, where IP multicasts are forwarded on the basis of the IP address.

Hope this helps

Kevin Dorrell

Luxembourg

chris.smailes · ‎11-19-2012

Hi Kevin, Thanks for the reply. It is the latter part of your answer that I'd like to take up: Do you know why it is that (on the 6509) igmp snooping leads to this traffic being forwarded to the router cpu ? I know that using a multicast mac with a unicast ip as Microsoft does is 'wrong' but is this why the traffic is punted ? Thanks again.

Julio Garcia · ‎11-19-2012

had a lot of problem with implementing this and wasn't comfortable with this going on the network -- so instead used cisco's SLB on the 6500s which worked really well -- im not sure if this is an option for you.

chris.smailes · ‎11-19-2012

Thanks for the tip Rob, I'll look into it.

Rolf Fischer · ‎11-19-2012

Chris,

I think this document gives a very good explanation:

https://supportforums.cisco.com/docs/DOC-2071

Short answer in summary: IP Packets with TTL=1 have to be processed by the CPU.

http://www.cisco.com/en/US/customer/products/hw/switches/ps708/products_tech_note09186a00804916e0.shtml#solution

HTH

Rolf

chris.smailes · ‎11-27-2012

Hi Rolf - thanks for the reply. It's an interesting theory but doesn't seem to be correct in this case as the cluster packets have a TTL of 111, not 1. Regards, Chris.