High IO on Prime after upgrade from 3.0 to 3.1

[Kanes Ramasamy]

Hi All,

We are seeing high IO on Cisco Prime after upgrading from 3.0 to 3.1.

Would this be its usual behaviour or could it be affected by a bug?

Please share your thoughts/experience. 

Thanks in advance. 

Regards,

Kanes.R

[marce1000]

 - How are you checking these high IO values ?

                     (which command is being used) 

M.

[Kanes Ramasamy]

Hi M,

We are seeing the IO rate has increased from V-Center dashboard.

Regards,

Kanes.R

[marce1000]

 - You may want to check virtual memory usage too; Prime 3.1 uses more memory; if needed increase the virt mem of the VM (high IO rate may be a result of increased swapping).

M.

[gary.gatten]

I do not have before and after, but PI 3.1 seems to use a lot of RAM and does a fair amount of disk I/O for no apparent reason.  During "off hours" it continually reads 30MB/s - but writes almost nothing. Makes no sense to me...

I am checking using *nix tools: top, iotop, iostat, ps, etc.

I posted just a min ago regarding "Performance" issues.  I think some are related to disk IO, and perhaps large RAM footprint.  Maybe we will find some common problem and get both our issues addressed?

Thanks!

[Kanes Ramasamy]

Hi Gary,

Thanks for the sharing the details. For now we will be rebooting the prime, just to see if it clears off some issues or hung services after the upgrade. However if the issue persist, I will be raising a TAC case with Cisco, lets see how it goes.

Thanks and Regards,

Kanes.R

[Kanes Ramasamy]

Hi Guys,

I rebooted the prime and the IO is back to normal as per before upgrade.

Regards,

Kanes.R

[Joakim A]

We're currently experienceing a similar issue with version 3.2.

We're seeing continuous disk reads at between 100 and 200 MB/s. and IOPS at around 2k.

Is this normal behaviour?

[m-niemi]

Having same kind of problem with PI 3.3.

It seems that there is some internal scheduled job, which is running oracle_wcs prosesses with high load.

Disk read is around 350MB/s. This happens every 10 minutes.

[dan.letkeman]

Did you ever find a solution to this?

I am seeing reads of 300MB/sec to 800MB/sec on 3.3

Thanks,

Dan.

[cwdykstra]

We're seeing this same thing - did anyone ever find a solution?

[RDavidov]

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvm53104/?reffering_site=dumpcr

[RDavidov]

I have the same problem in PI 3.2, 300-800 MB/s read, reboot is not helps.

I assume that the problem may be due to a lack of RAM

Mem:  16331496k total, 15581572k used,   749924k free,    12016k buffers

Does anyone have a solution ?

[Luke2]

Hey, any luck on this, i'm having the same issue.

[m-niemi]

Hi.

Still no clue what caused hi IO. It seemed like there were ”too many” upgrades Ih the databse and the DB got somehow corrupted.

Solved the problem by installing fresh copy of PI.

Br

Mika