Problem with DHCP Snooping on Cisco 6506

[svjatoslav]

Hello!

I have a problem with high CPU load by DHCP Snooping process on Catalyst 6506 (WS-SUP720-3B, soft:

s72033-ipservices_wan-mz.122-18.SXF11.bin). I have it enabled on 15 VLANS, in which there are subscriber devices residing, and sending DHCP requests through Cisco to DHCP server (Cisco acts as DHCP relay, and it's collecting the snooping database, I also use DAI).

Snooping database contains 6962 bindings now.

CPU load goes high only sometimes, and I don't have a clue, why it's going so high. It can load as high as 45-47% of CPU, like this:

 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
 116    81471244 322596368        252 42.95% 43.48% 36.06%   0 DHCP Snooping  

When the load is high, the command: show ip dhcp snooping statistics is showing, that the overall quantity of

Packets Processed by DHCP Snooping is increasing rapidly. In normal situations, it's like 10-20 packets per second, but when the load is high, it's 1000-10000 pps.

But when I look at SPAN from my subscriber's VLANS, I don't really see any flood of DHCP requests, or something like that - everything looks as usual. Maybe, some of subscriber's devices are sending incorrect DHCP requests, that are causing packets to loop inside RP, or something like that? How can I detect that thing?

Also I thought, that if I enable the ip dhcp snooping trust mode on all of the Catalyst interfaces, the DHCP snooping will not process the subscribers DHCP packets, and I can, by exclusion of interfaces from one to one, detect, from which interface the problem is originating. But this seems to be incorrect, I turned the ip dhcsp snooping trust on all interfaces, and I still get spikes of CPU load by DHCP snooping process. Why it's still examining packets, even on trusted interfaces, is it ok?

And one more question - if I disable the ip dhcp snooping globally, will it clear all my existing bindings in snooping database?

[m.glosson]

This will be a partial reply only. I'm guessing that when you enable trust on all ports you are still seeing the traffic spike because it is always building the database, even though it trusts everything. You haven't mentioned where you are storing the database, but Cisco recommends that you keep it off-device somewhere, such as a tftp server, especially with a database that big. I'm not sure if you can help it, but DHCP snooping is meant to be deployed on access switches more so than a switch in the core or distribution later.

I know this next thing is a crappy answer, but I'm 90% sure that disabling dhcp snooping will not delete the database. But you can make a backup of it first by simply copying the file from wherever it resides (show ip dhcp database) to somewhere else.

Good luck,

Matt

[svjatoslav]

Thanks for your reply. Yes, I have my database stored on TFTP server, and I'll try to disable the DHCP Snooping tomorrow, to see if the binding database will be deleted, or not. I know that it's best to keep snooping on the access, but I don't have that function on all of my equpment, so for now it's more likely to stay on 6506.

Strange that the problem is appearing, and disappearing periodically, I think it's not related to database size (if it was, the CPU load, maybe, would increase linearly as the database grows). Maybe it's some incorrect DHCP packets, but I wonder, how can I debug that?

DHCP Snooping process certainly doesn't build database if you enable trust on interfaces (when I enabled trust on all interfaces today, Cisco stopped adding new entries to database, until I enabled it again). But why it's still examining packets on trusted interfaces? I thought, that if you enable trust on some interfaces, Cisco just skips DHCP packets on these interfaces, and they don't go to CPU.

[svjatoslav]

I've executed the command no ip dhcp snooping today's morning, and it cleared all bindings from database. Bindings are saved on TFTP server, so it's not a problem, but still, it would be very good to know some way to debug the problem with snooping, because I want it to work (and DAI too).