06-03-2020 04:06 AM
Hi,
We had a issue with cisco prime 2.2 which we have been using for long time.
Now we had a issue where the prime is crashed and the only available backup which we have is 2 years old.
We had two to three issues:-
1. OPT Volume was full --> NCS Clean up fixed this issue
2. VAR volume is full --> Not able to fix this issue? --> Any suggestions how to fix this issue?
3. In the logs it was oracle memory issue --> i think is released to space issue which could be fixed if i fix point 2.
Post doing NCS clean up --> now its staying the DB not start.
I understand we can do ncs db reinit --> but will i lose all the data? if i am trying to do a backup as well its not working as it didnt get finished only.
Also is it possible to get the wireless map data from the CLI some place as we are using that mosly.
Any suggestions??
06-03-2020 05:34 AM
ncs start verbose
Starting Prime Infrastructure...
Reporting Server Heap size = 4096m
XMP Server Heap size = 6144m
Starting Health Monitor
Starting Health Monitor as a primary
Checking for Port 8082 availability... OK
CERT MATCHED :
Updating web server configuration file ...
Starting Health Montior Web Server...
Health Monitor Web Server Started.
Setting UID to 441:110
UID set to 441:110
Starting Health Monitor Server...
Health Monitor Server Started.
Failure during Prime Infrastructure startup. Check launchout.log for details.
Starting strongSwan 5.0.1 IPsec [starter]...
When seeing the status is showing db error
------------
/dev/mapper/smosvg-optvol 647G 454G 160G 74% /opt
/dev/mapper/smosvg-varvol 3.8G 3.8G 0 100% /var --> Concern on this
06-03-2020 10:24 AM
>VAR volume is full --> Not able to fix this issue? --> Any suggestions how to fix this issue?
- You should probably cleanup the directory under a root shell, as far as I remember in Prime 2.x you become root with the shell-command root. Below is a link to finding biggest directories and files. Of course not everything can be deleted but look for directory related to logging (e.g.). Same applies for other partitions that are full :
https://www.tecmint.com/find-top-large-directories-and-files-sizes-in-linux/
M,
06-03-2020 12:24 PM
Will doing a ncs db reinit --> as a solution can that help in re-initializing the database or will it delete all my database?
06-04-2020 12:03 AM
- Yes, it will delete the content of the database and can be considered as being a 'factory defaults reset' for Cisco Prime.
M.
06-08-2020 01:04 AM
hi..
i checked --> /dev/mapper/smosvg-varvol 3.8G 3.8G 0 100% /var
i should be deleting these logs but not sure can you please let me know if that okay to delete?
So i checked under /var/logs -->
ade # cd log/
ade # ls -l
total 163604
86556 cron
275570 cron.1
241104 cron.2
275361 cron.3
275352 cron.4
147168 lastlog
221835 maillog
696894 maillog.1
543885 maillog.2
701135 maillog.3
703399 maillog.4
15508693 secure
39034880 secure.1
22847488 secure.2
41873408 secure.3
43315200 secure.4
06-08-2020 07:16 AM
- You can delete all the red-colored-files , except for the first maillog (the one without extension). The other ones are result for log-rotating and will appear again after some time.
M.
06-08-2020 07:35 AM
but these are in MB i cant find anything which i think can remove like 100 or 200 mb files from the same.
Is there any place else in /var other than logs.
06-08-2020 08:10 AM
- Following root-shell command examples from the link below, you may look for big files on the whole of the /var partition, if their names relate to logging and or temporary usage, you can delete them (too) :
https://www.cyberciti.biz/faq/linux-find-largest-file-in-directory-recursively-using-find-du/
M.
06-08-2020 10:19 AM
06-09-2020 03:57 AM - edited 06-09-2020 04:25 AM
hi,
tried deleting those under secure.1 to .4 didnt help and its showing /var 100%
ran the command for top 20 in /var folder
ade # sudo find /var -type f -printf "%s\t%p\n" | sort -n | tail -20
1041141 /var/log/sa/sar27
1041141 /var/log/sa/sar28
1041141 /var/log/sa/sar29
1060080 /var/log/sa/sa24
1060080 /var/log/sa/sa25
1060080 /var/log/sa/sa26
1060080 /var/log/sa/sa27
1060080 /var/log/sa/sa28
1060080 /var/log/sa/sa29
1060080 /var/log/sa/sa31
1318912 /var/lib/rpm/__db.002
1388779 /var/log/audit/audit.log
2641920 /var/lib/rpm/Filemd5s
2945024 /var/lib/rpm/Basenames
5243040 /var/log/audit/audit.log.3
5243060 /var/log/audit/audit.log.1
5243067 /var/log/audit/audit.log.2
15514164 /var/log/secure
20729856 /var/lib/rpm/Packages
23988767 /var/lib/mlocate/mlocate.db
--------------
backup ciscprime repository defaultRepo application NCS
DO NOT press ^C while the backup is in progress
Aborting backup with a ^C may terminate the backup operation or the backup file may be corrupted
Checking for arhivelog cleanup job
Backup Started at : 06/09/20 15:19:42
DB size is 318 GB --> this even possible?
Available size is 172 GB
you might run out of disk space in repository, Do you want to proceed with backup
do you want to proceed [y/n]?
How am i suppose to take a backup if there is not enough space
06-09-2020 10:24 AM
- You may want to cleanup the database with ncs stop ; ncs cleanup ; ncs start. Subsequently if there is still not enough space you could backup to an external repository,
M.
06-09-2020 10:22 PM
hi
i did the ncs cleanup but no luck that was my first thing to do.
that /var vloume i am not able to cleanup space and it keeps stuck on db error when trying to start the services.
06-12-2020 02:22 AM
- You may then need to re-install Cisco prime and restore from a valid backup. Note that as far as partition sizing being provided , choose the installation type according to the size of your network and or the number of devices which will be entered into Prime, check the following info-resources :
M.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide