cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
15964
Views
26
Helpful
21
Replies

multicast mac-address Nexus 7k

dinoantonucci
Level 1
Level 1

Hi,

i'm going to use Nexus 7000 in Data Center.

During analysis configuration, I need define mac-address-static configuration for multicast mac address for Firewall Checkpoint cluster.

In "Layer 2 Switching Configuration Guide, Release 4.1.pdf" documentation speak about

"Configuring a Static MAC Address

[..]You cannot configure broadcast or multicast addresses as static MAC addresses[..]"

Have you a suggestion to manage this problem and why is it not possible configure mac address static multicast?

Regards

Dino

21 Replies 21

Joseph did you ever get this to work with the latest NX-OS  ?

Yes there is a fix or a workaround for this.  What we did was to have the Checkpoint Firewall host the multicast MAC and respond using proxy-arp for the VIP.  We also turned off igmp snooping on the Nexus 7k.  For a layer 3 interface on the switch the 'ip arp' command worked to allow the switch to communicate to the VIP of the firewall.

The other options would be to turn on broadcast HA on the checkpoints but that would be a less desireable setup in our situation - or - in later releases of the checkpoint software you can actually setup the firewall to join an igmp group.

I hope this helps.

what we are seeing now that the checkpoint guy has explained it is the CPP traffic is never making it to each of the cluster members when in the 7k.  If you move the interface to a 3750 the cpp packets work. It doesn't matter if its in mulitcast or broadcast mode with IGMP snooping disabled.  Any ideas ?

Cisco says upgrade the the 7k to 5.2 code so we can do a static mac entry with the multicast address of the CCP packet source.

We are only running 5.1(1a).

You must disable IGMP snooping.  It sounds like you did.

If you don't have a layer 3 interface on the 7K for the subnet then you must enable the checkpoint firewall to advertise the arp entry for that subnet.

If you are routing through the 7K for the end host then all you should need is the static MAC entry for the VIP of the checkpoint cluster to allow the traffic to be routed correctly.

This all stems from how Checkpoint accomplishes its active/active load balancing.  In multicast mode (which in our lab when we first tested checkpoint worked the best), the checkpoint cluster IP is a unicast IP address and is associated with a multicast MAC address.  You must disable igmp snooping for that subnet to allow the mulicast packets to be treated as a broadcast by the switch (forwarded out all interfaces in that VLAN except for the interface that it was recieved on).  When the end host sends a packet destine for another host on a different network and the checkpoint firewall is next hop/Defualt gateway, an arp request has to be generated by the end host asking the MAC of the DG (and in checkponts case is a multicast MAC).  The fix for this if you have the 7K routing to the firewall is to statically set the MAC in the layer 3 interface configuration, like so:

interface Vlan10

no shutdown

ip address 192.168.100.1/24

ip arp 192.168.100.10 0100.5e11.1111

As for the CCP packets between the checkpoints, we never had any issues with those packets traversing the 7K.  Make sure that the checkpoint firewalls are running proxy-arp as stated in my previous post.

I hope this helps!

Thanks for the reply. So what actually happened.  We didn't have an active/active Checkpoint cluster we had an active/passive.  This checkpoint was the eggress gateway for the Nexus but also plugged into the nexus core VDC via 1 Gig.  After working with TAC and doing packet captures we saw the CCP packets both in Broadcast mode and Mulitcast mode packets. The checkpoint vendor basically show us where the cluster was flipping back and forth on the nexus because it was not recieve the CCP heartbeats on that interface.   TAC found us two bugs and we able to fix one of them.

This one:

CSCtl67036  basically pryer to NX-OS 5.1(3) the nexus will discard packets that have a source of 0.0.0.0.  Which in broadcast mode is exactly what the CCP heartbeat is. 

We bypassed this one.

CSCsx47620 is the bug for the for static multicast MAC address feature but it requires 5.2 code on the 7k and that then requires.

Joseph - The ClusterXL A/A configuration is a variation of the  StoneSoft or Rainfinity clustering technologies that have been used to  cluster Solaris and other *NIX flavored servers and firewalls for  years.  (In fact, StoneSoft filed suit against Check Point in Europe 8  or 9 years ago for patent violations, and lost.)  These configurations  were very common on Check Point clusters running on Solaris from the  late 90's forward - and, as you describe, have unicast IP's with a  multicast MAC for the VIP.  Even from the days of installing these on  the brand new (at the time) 2900 series switches you had to do exactly  as you state above - static MAC entries (or in some cases port mirrors)  so traffic was directed to both active switch ports.  In Active/Passive  mode Check Point ClusterXL clusters are almost always "plug and play"  today - rarely do the switches need anything beyond speed/duplex  settings.  The VIP assumes the MAC of the physical NIC it is currently  bound to, and therefore there are no issues as far as switch config or  proxy ARP entries on the gateways.  All of these issues have to do with  traffic flowing to the VIP and through the firewall, and the ability of  the switch to correctly identify which physical switch port(s) the VIP  is currently patched in to.  This is one of three types of traffic  associated with ClusterXL itself.  The second is state synchronization,  which is accomplished through a crossover cable and therefore not  relevant.  Even when using a switch state sync is a typical TCP 18181  connection from a unicast IP/unicast MAC on one gateway to the other  through a dedicated interface pair.

The challenge described by CJ is not with the traffic  flowing to the VIP, however.  It is an entirely separate process - Check  Point Clustering Protocol (aka CPHA if filtering in WireShark) is  essentially the heart beat traffic.  Every interface pair within a Check  Point cluster continually communicates with its "partner" interface on  the other cluster members.  If any packet takes over 100ms or shows more  than a 5% loss the gateway is forced in to "probing" mode where it  falls back to ICMP to determine the state of the other cluster member.   Depending on the CPHA timing settings an active gateway will failover to  the passive in as quickly as 500ms or so.  ClusterXL will fail over the  entire gateway to the standby to avoid complications with asynchronous  routing.

Out of the box, CCP is configured to use  multicast, but it supports broadcast as well. To change this in real  time (no restart required) simply issue the command:

cphaconf set_ccp {broadcast/multicast}

At  the Ethernet level, CCP traffic will always have a source MAC of the  Magic MAC of 00:00:00:00:xx:yy where XX is the “Cluster ID” – something  identical on each cluster member but unique from one cluster to another,  and YY is the cluster priority (00, 01, etc.) based on the priority  levels set on cluster members within Dashboard on the cluster object.  The destination MAC will always be the Ethernet broadcast of  ff:ff:ff:ff:ff:ff.

At the IP level the source of CCP  will always appear as 0.0.0.0. The destination will always be the  network address (ie, x.x.x.0).

Similarly in multicast mode you will see the same traffic  at the IP level but at the Ethernet level the destination will now be a  IPv4 multicast MAC (ie, 01:00:5e:4e:c2:1e).

In a tcpdump  with the –w flag opened in WireShark and a filter applied of just “cpha”  (without the quotes) you should see a continual stream of traffic with  the same source and destination IPs on all packets (0.0.0.0 and network  IP), the destination of either a bcast or mcast MAC and the source MAC  alternating between 00:00:00:00:xx:00 and 00:00:00:00:xx:01.

Long story short, the problem CJ is describing is a  behavior on the 7K where a packet capture taken on the Check Point  interface itself (ie, tcpdump –i eth0 –w capture.cap) ONLY shows CPHA  traffic from it’s own source MAC and no packets from it’s partner. A  tcpdump on the 7K itself will show traffic from both.

As CJ mentioned, a simple NxOS upgrade will fix the issue per:

This one:

CSCtl67036  basically pryer to NX-OS 5.1(3) the nexus will discard packets that have a source of 0.0.0.0.  Which in broadcast mode is exactly what the CCP heartbeat is. 

We bypassed this one.

CSCsx47620 is the bug for the for static multicast MAC address feature but it requires 5.2 code on the 7k

(NOTE:Additional RAM may be required for the 5.2 update)

Also note that Check Point gateways do support IGMP  multicast groups, given that you have the correct license. It is a  feature of SecurePlatform Professional on the higher end gateways or as a  relatively inexpensive upgrade on the lower end boxes or open  platforms. For lab purposes you can simply type “pro enable” at the CLI  (without the quotes). As of the latest build there is no technical  limitation (no license check) so you can enable advanced routing  features as needed for testing in a lab. For step by step details on  configuring IGMP on SPLAT Pro go to the Check Point support site and  search for sk32702.

This can be a frustrating issue to troubleshoot, so hopefully this helps someone avoid the headaches I ran in to.

Good Night All,

 

I am presently having this issue with a Checkpoint cluster operating in load sharing multicast mode connected to a nexus 5500 series cluster core.

When in this mode all switches including the nexus are not able to arp to the VIP of the respective firewall interfaces. So what is occurring is that the DG of the nexus as well as 3com switches in the customer's environment is having an incomplete ARP entry for the Firewall VIP.

I have seen in stated below that RFC 1812 does not allow this. Can you advise if this is the reason why all switches with a layer 3 interface are not able to arp to the respective DG on the Firewall cluster?

Are there any workaround for this beside entering manually arp entries on the switches to point to the multicast mac?

We are trying to explain to the customer that this is the way forward however they are of the belief that the core switch (nexus) is the issue and once resolved there will allow the other switches 3com to communicate without the need of the multicast mac static entry.

 

Please help!!! I have seen much discussions here and we presently have a cisco tac case opened but no real root cause determined by cisco.

Review Cisco Networking for a $25 gift card