cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
63
Views
2
Helpful
7
Replies
Highlighted
Cisco Employee

CPU usage

 

Experts,

 

  I noticed when NSO is calculating southbound diffs, it is only using 2 of the available 8 cores.  Is there a way to speed things up and use all 8?

top - 04:45:09 up 39 min,  4 users,  load average: 0.65, 0.76, 0.53

Tasks: 148 total,   2 running, 146 sleeping,   0 stopped,   0 zombie

%Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

%Cpu1  : 36.3 us,  0.7 sy,  0.0 ni, 63.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

%Cpu2  : 63.7 us,  0.3 sy,  0.0 ni, 36.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

%Cpu4  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

%Cpu5  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

%Cpu6  :  0.0 us,  0.3 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

%Cpu7  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

KiB Mem : 65978316 total, 59363276 free,  6413252 used,   201788 buff/cache

KiB Swap:        0 total,        0 free,        0 used. 59360844 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND   

847 root      20   0 1595764 406080   3256 S 101.0  0.6  15:39.88 ncs.smp   

  980 root      20   0 20.818g 5.295g  11504 S   0.3  8.4   0:19.44 java      

    1 root      20   0   54140   3764   2412 S   0.0  0.0   0:01.28 systemd   

    2 root      20   0       0      0      0 S   0.0  0.0   0:00.00 kthreadd  

    3 root      20   0       0      0      0 S   0.0  0.0   0:00.00 ksoftirqd/0

    5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:+

    6 root      20   0       0      0      0 S   0.0  0.0   0:00.00 kworker/u1+

    7 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 migration/0

    8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh    

    9 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/0  

Everyone's tags (4)
7 REPLIES 7
Highlighted
Cisco Employee

Re: CPU usage

 

Not really no.

 

 

Once the diffs are calculated, we start to use more cores though. I.e the actual talk-to-device-and-send-data is parallel, whereas the diff calculation is not.

 

 

/klacke

 

Highlighted
Cisco Employee

Re: CPU usage

 

Reality check: Is the diff something that contributes significantly to the duration of the operation? Does it need sped up?

 

Cheers,

 

KJ.

 

Highlighted
Cisco Employee

Re: CPU usage

 

From our observation, with a large config the diff calculation is the significant contributor to the time.  The total process is taking almost 15 minutes, and about 12 minutes of that is calculating southbound diffs.

 

Highlighted
Cisco Employee

Re: CPU usage

 

KJ, All,

 

 

Yes, with NCS 3.4.1 we have seen that with large device cisco-ios configs (50K lines) that the 'ncs: calculating southbound diffs¹ is taking a considerable amount of time to complete.

 

There have been several performance enhancements added in NCS 3.4.5 which effectively cuts the time expended 'calculating southbound diffs¹ significantly (I¹ve seen near 50% improvement in some tests compared to 3.4.1).

 

 

-Larry

 

Highlighted
Cisco Employee

Re: CPU usage

 

We have a customer who were unable to delete service instances since southbound diff took too long. NSO took nearly an hour calculating southbound diff and timed out. NSO 3.4.5 made a fair improvement on certain scenarios like extensive usage of leafrefs, however we still have issues and are waiting for 3.4.5.1 fix. Tail-f tac says they reduced to 30 mins with the 3.4.5.1 fix. We are hoping this solves the issue.

 

 

Jiwon Hu

 

Highlighted
Cisco Employee

Re: CPU usage

 

Hi Jiwon,

 

 

Do we know the timeline on when this fix 3.4.5.1 could be made available?

 

 

Thanks,

Krishna

 

Highlighted
Cisco Employee

Re: CPU usage

 

Hi Krishna,

 

 

The NED team said they will deliver us the 3.4.5.1 fix this week. I am not sure about the general availability.

 

 

Regards,

 

Jiwon

 

Content for Community-Ad
Cisco Community July 2020 Spotlight Award Winners