ANNOUNCEMENT - The community will be down for maintenace this Thursday August 13 from 12:00 AM PT to 02:00 AM PT. As a precaution save your work.
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
512
Views
0
Helpful
1
Replies
Highlighted
Beginner

DAI disabled - High CPU RT-Q

Hi all,

 

I assume this is not an easy at least if you are not a real expert.

Background information: We have real high cpu now for about 3 weeks on our 3750X-Stack with 6 switches. The high CPU came from out of nowhere. It is about 70% most of the time and 25-30% are from the Hulc DAI process. However we do not have DAI activated. After a little deep diving I found a way to find the cause of it.

 

"show controllers cpu-interface" shows a strong growing routing-protocol Queue. That is strange as the stack isn't a router anymore for about one year now. Before it was routing between different VLANs but that role was migrated to a 4500X VSS.

 

"debug platform cpu-queues routing-protocol-q"brings really a lot of - if not a flood - of messages like these:

 

021559: Oct 20 16:31:48.017: RT-Q:DAI Consume: Remote Port Blocked L3If:Vlan1 L2If:GigabitEthernet2/0/37 DI:0xAC, LT:1, Vlan:1   SrcGPN:93, SrcGID:93, ACLLogIdx:0x0, MacDA:ffff.ffff.ffff, MacSA: 0cc4.7aa9.f2ea   ARP: 00010800_06040001_0CC47AA9_F2EA0A02_96050000_00000000_0A029701

   TPFFD:CDC0005D_00010001_00800040-000000AC_1C1C0000_00000000

 

021560: Oct 20 16:31:48.017: RT-Q:DAI Consume: Local Port Fwding L3If: L2If:TenGigabitEthernet5/1/2 DI:0xFC, LT:1, Vlan:405   SrcGPN:584, SrcGID:584, ACLLogIdx:0x0, MacDA:ffff.ffff.ffff, MacSA: 901b.0ec0.848a   ARP: 00010800_06040001_901B0EC0_848A0A0A_05C80000_00000000_0A0A0525

   TPFFD:CD800248_01950195_00880040-000000FC_17170000_00000000

 

Now we see here the "RT-Q:DAI" which I assume is the root cause of the problem. The messages mention different VLANs and different uplinks while most of them show "Local Port Fwding", some of the "Remote Port Blocked".

 

I have no idea where the packets come from. I mean I followed one up to it's source port (according to the MAC address in the log message) but could not see why such a packet should be generated there.

 

Any help would be greatly appreciated.

 

Thanks

Daniel

Everyone's tags (4)
1 REPLY 1
Highlighted
Cisco Employee

Re: DAI disabled - High CPU RT-Q

Hi, 

 

I just came across with the same logs when I was trying to do the lab repro for one of my customer. These logs are seen for the DAI process (DAI would enable automatically in case you have 802.1x configured or you have IPDT enabled) which is receiving the ARP broadcast packets from Gig 2/0/37 and Ten 5/1/2. These packets would have been seen in the stack while the stack master would have been the switches other than switch 2 and switch 5. Packets received from the other stack member to the master would be using the routing protocol queue. 

Content for Community-Ad