WS-C3750X-24S hl3mm process issue.

[jedson328]

We are running a stack of 2 WS-C3750X-24S-E's with IOS version 12.2(58)SE2 and we are seeing continuous high processor utilization involving the hl3mm process.  The stack is configured for PIM Sparse mode, with 2 upstream Nexus 5596'es acting as both RP candidates and as BSRs.  The hl3mm process generally runs at about 19% but can spike up into the 60% range.

Symptoms on the 3750X are spiking latency and sluggish CLI perfformance.  I thought at first that we might be running into SDM limits, but the "show platform tcam utilization" show that we are far under the limits of the default SDM template.  Performing the "show ip pim (vrf name) bsr-router" command freezes the system and takes 20-30 seconds to return results.  Similiarly configured WS-C3750G-12S running 12.2(55)SE3 does not exhibit these same issues.

So I guess my question is this - is anyone else seeing issues like this on the 3750X line of switches?

Jeremy

[Alan Dunne]

Yes we are seeing the same issue.

High processor utilization where the process is the hog:

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

124   503369974   2707797     185902 29.14% 52.70% 56.08%   0 hl3mm 

We have a TAC case open.  They want us to turn off PIM, but can only do so for a brief period.

Switch Ports Model              SW Version            SW Image                

------ ----- -----              ----------            ----------              

     1 54    WS-C3750X-48P      12.2(55)SE4           C3750E-UNIVERSALK9-M    

*    2 54    WS-C3750X-48P      12.2(55)SE4           C3750E-UNIVERSALK9-M    

Alan

[cdemont]

Hello,

As I'm having the same issue of HL3MM process in multicast environnement taking about 10 (agv) upto 30% of my CPU. Could you please tell me if your case was solved by any action ? (config or IOS upgrade) ?

Many thanks for your feedback,

Regards,

Christophe

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

136   722748178    42573904      16976 11.35%  5.70%  7.83%   0 hl3mm

     443494334444435544434444455444544545444544445444443454455445

      459394489246470346444005613366276145561185361355649125553773

  100     *

   90     *

   80     *

   70     *

   60     *                              *                   *

   50  *  *   *  *  ** *     **** ****** *** *** ** ***   ******

   40 ****#* ************ *****#********************************

   30 ##########################################################

   20 ##########################################################

   10 ##########################################################

     0....5....1....1....2....2....3....3....4....4....5....5....6

               0    5    0    5    0    5    0    5    0    5    0

               CPU% per minute (last 60 minutes)

              * = maximum CPU%   # = average CPU%

Switch Ports Model              SW Version            SW Image

------ ----- -----              ----------            ----------

     1 30    WS-C3750X-24       12.2(58)SE2           C3750E-UNIVERSALK9-M

     2 30    WS-C3750X-24       12.2(58)SE2           C3750E-UNIVERSALK9-M

*    3 30    WS-C3750X-24       12.2(58)SE2           C3750E-UNIVERSALK9-M

     4 30    WS-C3750X-24       12.2(58)SE2           C3750E-UNIVERSALK9-M

     5 30    WS-C3750X-24       12.2(58)SE2           C3750E-UNIVERSALK9-M

[Leo Laohoo]

Hi Christophe,

Unless you have critical reasons to run 12.2(58)SE, 15.0(1)SE, and 15.0(2)SE, avoid these IOS as they have various CPU bugs.

Investigate 12.2(55)SE6 instead.

[cdemont]

Thank you, that's helpful !

Sent from Cisco Technical Support iPhone App

[superego]

I have the same problem running on version 15.2(4)E6 in 3750X.

I applied "ip pim spt-threshold infinity" and it solved the problem.

Reference for using Shared tree vs Source Tree:

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipmulti_optim/configuration/12-4t/imc-optim-12-4t-book/imc_pim_sparse.html#GUID-53401AB5-DA27-4CBB-886C-40D577875B45

[joseph.bernard]

This instantly fixed our high CPU issue on 3750E's running 15.0(2)SE11.  We are large environment, so this page will be definitely be good to reference.  Thank you!