Odd MAC addresses on ports that connect Cisco IP phones

[James Hawkins]

Hello,

I have a CUCM 10.5 cluster with 7960, 7970, 7937 and 8941 IP phones connected to Catalyst 3850 switches.

I have enabled DHCP Snooping and Dynamic ARP inspection on the switch ports that connect the phones and am seeing odd messages logged to the console as shown below:

021496: Dec 15 23:09:48 UTC: %SW_DAI-4-INVALID_ARP: 1 Invalid ARPs (Res) on Gi2/0/38, vlan 20.([0004.f2f2.f871/10.84.8.14/f09e.63a4.ab31/0.0.0.0/23:09:47 UTC Mon Dec 15 2014])

021497: Dec 15 23:09:48 UTC: %SW_DAI-4-INVALID_ARP: 1 Invalid ARPs (Res) on Gi2/0/38, vlan 20.([0004.f2f2.f871/10.84.8.14/f09e.63dc.8bb1/0.0.0.0/23:09:48 UTC Mon Dec 15 2014])

021498: Dec 15 23:09:48 UTC: %SW_DAI-4-INVALID_ARP: 1 Invalid ARPs (Res) on Gi1/0/43, vlan 20.([0004.f2f2.fbc3/10.84.8.40/f09e.63dc.8bb1/0.0.0.0/23:09:48 UTC Mon Dec 15 2014])

The MAC/IP addresses that I have colored blue are the addresses that expected - i.e. the phone MAC address and the corresponding IP address that it received from DHCP.

The MAC/IP addresses shown in red are what I believe are causing the error messages to be logged.

Note that f09e.63dc.8bb1 is seen on both Gi2/0/38 and Gi1/0/43. The f09e63 vendor part of the address belongs to Cisco.

I only see these messages for phones that connect 7962, 7937 and 8941 phones - not for 7960 phones.

I do not see this MAC address displayed if I run the show mac address-table command on the switch.

Does anyone know what these odd MAC addresses are caused by?

T thought it could be peer firmware sharing but the messages are still appearing when that is disabled.

[Dennis Mink]

could it be that you that dhcp snooping "database" is empty for a certain MAC?

I.e can you force the phone to request a new IP through DHCP to populate it?

see:

https://supportforums.cisco.com/document/25151/swdai-4-dhcpsnoopingdeny-error-message-received-when-configuring-dynamic-arp

[James Hawkins]

Hi Dennis,

Thanks for replying. The DHCP Snooping database has the bindings I would expect.

I am not at the site at the moment but I have a theory that this could be caused by gratuitous ARP being enabled on the phones.

I am going to try disabling this feature and seeing if it resolves the issue.

[James Hawkins]

Hello,

I have got a wireshark trace showing the traffic which I have attached.

It is ARP traffic but it still appears even if I turn Gratuitous ARP off on the IP phones.

Any suggestions as to how I can get rid of this unwanted traffic?

Thanks

James

[Aaron Harrison]

Hi James

This doesn't look like GARP to me (i.e. a device announcing itself) - it's the phone responding to an ARP request from the ab31 MAC address, I assume this phone is 10.84.8.59?

You're sure that ab31 MAC address isn't in your MAC tables? That would be odd as it's obviously traversing the switch? 

Aaron

[James Hawkins]

Hi Aaron,

Thanks for your response.

The ab31 address is in the address table for the switch with the phone connected and is shown as being on an uplink to the core switch.

On that core switch the ab31 address is shown as being on an uplink to another access switch with phones attached.

When I look for the ab31 address on that access switch it is not shown as being in the CAM table.

It is odd the the IP address associated with the ab31 address is 0.0.0.0 - the place I have seen 0.0.0.0 used is in DHCP requests.

What I can say is that the ab31 address is not in the CAM table for the port to which the phone is attached.

The interfaces are running port security for just one address and CDP is turned off. The switches are only used to support phones and the customer wants them as secure as possible hence DHCP snooping, ARP inspection, IP source guard, port security and port based ACLs to filter traffic. I have disabled these features with the exception of snooping and DAI on some test ports but as still seeing the problem.

James

[Aaron Harrison]

Hi James

Agreed - 0.0.0.0 implies the source doesn't have an IP. It seems odd that the ARP packet is targeted at the phone MAC as well, if you know the IP and the MAC address why ARP for either?

Did you filter the capture on ARP? Or was there other traffic between this mystery-mac and the phone?

I'm fairly sure the phone is just responding as it thinks it should. I'd be trying to trace where the ARP request came from as you have done, but it must be sourced from somewhere on the other switch. 

Aaron

[James Hawkins]

Thanks Aaron,

The vendor part of the MAC belongs to Cisco. The only devices other than phones in this VLAN are the VLAN interfaces on the two core switches.

The MAC addresses for these are:

f09e.63a4.2ad6 and f09e.6364.d756 which look similar to the culprits.

[James Hawkins]

I think I have worked out what the issue was.

I had entered the command:

ip arp inspection validate src-mac dst-mac ip

When I removed the ip option the log messages disappeared.

Apologies for the wild goose chase and thanks for your replies.

[WebOps eDreams]

hi james.

you need to write it like this in order to avoid the dai messages you are getting:

"ip arp inspection validate src-mac dst-mac ip allow zeros"

regards

[Dennis Mink]

I know you fixed this issue, 

but to answer your question, GARP can be disabled on the phone configuration pages in CUCM.  I believe it is disabled by default though.