Multicast and Microsoft NLB question

[p.caforio]

Hello,

I just configured our first cluster of Microsoft servers using Multicast NLB.

The gateway of the VLAN is a Cisco 5540 and the servers are connected on a Cisco 3750 stack.

In order to let the ASA forward the frames for this multicast group I had to set the static arp resolution to the Multicast MAC address of this cluster.

No static MAC entry on our switches.

All works fine and multicast is working but...

1) Using wireshark on our NLB nodes I found that frames are destinated to the unicast mac address of the machine. IF I ping from the ASA to the Cluster  IP, the ICMP is sent just to ALL the NLB nodes.

2) All frames originated from the NBL nodes has the unicast (physical) MAC address.

2) Other hosts in the same VLAN do not receive any broadcast related to this cluster (multicast is working)

3) on our Cisco 3750 stack, the Multicast MAC address is not in the CAM.

Since just NLB nodes are receiving this traffic, my question is:

HOW can the multicast work if the switch doesn't know anything regarding this cluster?!

thanks,

Paolo

[Rolf Fischer]

Hi Paolo,

3) on our Cisco 3750 stack, the Multicast MAC address is not in the CAM.

Which show command did you use?

Remember that the CAM table of a switch is populated by learing the source-addresses of the received frames. In a valid Ethernet frame, the source address will never be a multicast address.

However, the way a switch can dynamically learn multicast MAC addresses is by snooping IGMP messages received on its ports. Try if a "show mac address-table multicast" provides the desired output.

Btw: What mode of NLB have you enabled? Multicast with or without the "IGPM option"?

Best regards,

Rolf

[p.caforio]

Dear Rolf,

I'm using "show mac address-table" and "show mac address-table multicast".

The NLB Multicast "without" IGMP.

What do you think should be the correct configuration in our scenario?

thanks,

Paolo

[Rolf Fischer · Accepted solution]

Paolo,

I really thought you were using the IGMP option because of your statement:

"2) Other hosts in the same VLAN do not receive any broadcast related to this cluster (multicast is working)"

Without the IGMP option, NLB uses a locally administered Multicast MAC address with the format 03:BF:<IP-Address-of-the-Cluster>. Thus this is not an IANA-assingend multicast MAC address (01-00-5E), IGMP snopping cannot avoid the flooding of those frames throughout the entire VLAN, which is the only way a switch can handle such frames. The recommendation for avoiding/containing this flooding is the configuration of static MAC entries for the multicast Cluster MAC (binding it exclusively to the required ports). Those static entries then also will be listed in the "show mac address-table" output.

With the IGMP option, you can make use of IGMP snooping in order to avoid the flooding, so static MAC entries are not required in this case and the multicast cluster MAC can be learned dynamically by IGMP snooping. It should then be listed in the "show mac address-table multicast" output.

HTH

Rolf

[p.caforio]

Dear Rolf,

The strange behaviour is that frames sent to nodes in the cluster has the physical mac address in the destination field and not the 03:BF...  as expected.

Even if the ASA ARP table forces the cluster IP address to the 03:BF... for some reason a single frame sent from the ASA is doubled to the two nodes of the cluster, each with different mac address in the destination field.

If you have to suggest the best practice, what is your opinion? With or without IGMP?

What about the configuration?

Regarding simple multicast, since we use trunks and ESXi on switches, is a static entry on our stacks CAM table riskless for current traffic?

Consider that we have the following setup:

ASA - (trunk) - single 3750 -- (stack on port-channel) --- 3750 stack --- 4 ports to ESXi  (many servers and NBL cluster on two of them)

For what you wrote I should map the 03:BF MAC address on the first 3750 on the port-channel interface and on the 3750 stack on each port that connects  to the ESXi.

thanks a lot,

Paolo

[Rolf Fischer]

Hi Paolo,

that's really strange. Unfortunately I'm not an ASA expert. On Cisco routers a static ARP entry is needed because the router would drop an ARP reply for an unicast IP address answered with a multicast MAC address (RFC 1812 - 3.3.2). With a static ARP entry configured, the router does not longer need to resolve the address and uses the configured MAC. Perhaps an ASA behaves differently...

Whether or not the IGMP option is the better choice depends on the number and distribution of the cluster members in the network and the changes to be expected in the future. I think in your case the dynamic approach with IGMP snooping wouldn't be a big advantage but that's something you know best.

The best and easiest way to limit the flooding scope would of course be a separate VLAN for the NLB cluster.

Useful link: Catalyst Switches for Microsoft Network Load Balancing Configuration Example

HTH

Rolf

[p.caforio]

Thanks Rolf,

You VLAN suggest is correct but I have not enough ports on the ASA to configure a gateway for it.

If I configure the static entry for the multicast MAC on our switches, will it prevent the flood?

In this case, What about my second question?

Quote --

Regarding simple multicast, since we use trunks and ESXi on switches, is a static entry on our stacks CAM table riskless for current traffic?

Consider that we have the following setup:

ASA - (trunk) - single 3750 -- (stack on port-channel) --- 3750 stack --- 4 ports to ESXi  (many servers and NBL cluster on two of them)

For what you wrote I should map the 03:BF MAC address on the first 3750 on the port-channel interface and on the 3750 stack on each port that connects  to the ESXi.

Paolo

[Rolf Fischer]

Paolo,

well, a static entry (Link: c3750 Configuration Guide) tells the switch to stop its default behavior (flooding multicast through all ports except the one, the original frame was received) for the specified destination MAC and instead forward those frames only to the configured port(s).

If such a configuration is needed at all for the first c3750 depends on whether it has other active (trunk) ports in that VLAN. I guess on the stack this is the case, if so you could limit the flooding scope to the two ESXi ports with the NLB cluster members.

Personally, I wouldn't recommend to do such changes during the business hours even if the risk is acceptable according to my experience.

HTH

Rolf

[srinivas1683]

HI Rolf,

Am also using NLB in my network. two servers are connected in once switch and NLB is working fine in unicast mode. 

If that switch1 goes down servers will be disconnected. So i have activated second ports on each server and enabled  NLB in second switch2 with same virtual mac address and same VIP which i configured in switch1 but failover is not working.

Could you please help me on this please. how we need to do automatic failover NLB between two switches.

Thanks

Satya

[Rolf Fischer]

Hi Satya,

NLB unicast mode works differently. From a network perspective, it's the worst option. By using bogus MAC addresses (treated as unknown unicast), it forces a switch to act like a hub. The only "good" thing is, that no extra configuration is needed on network devices.

Unfortunately I don't know how to configure more than one NIC per cluster member for NLB without bundeling the respective switchports to an etherchannel. If your concern is that switch1 is a potentilal single point of failure for the NLB cluster, couldn't you simply connect server1 to switch1 and server2 to switch 2?

HTH

Rolf

[Walter Astori]

In my environment i have some Microsoft NLB. On the Windows Operating System i configured the NLB with Multicast address with IGMP Multicast. On the switch i configured the static mac address with the instruction :

mac-address-table static 0100.5e7f.5afe vlan 90 interface GigabitEthernet2/0/1 GigabitEthernet2/0/2

My environment work fine

[srinivas1683]

HI Rolf,

Please find the attached diagram of our NLB, and in this two switches we are using HSRP.

Please suggest.

Thanks

Satya