Cisco Prime Infra 3.1 does not start

[Sergey Prishchepa]

Hello!

I have a Cisco Prime Infra 3.1.0.0.132 with all updates installed. After restarting the ncs has not started.

If I try to start manually, I see:

setup of pnp failed please see the /var/KickStart/install/setup.log

FATAL ERROR: Could not start display service

setup.log

/bin/cp: writing `/var/KickStart/install/ks_env_tmp.in': No space left on device
"/bin/cp -f /var/KickStart/install/ks_env.sh /var/KickStart/install/ks_env_tmp.in" failed
cleanning up.......
execute (rm -f /var/KickStart/install/ks_env_tmp.sh)
execute (rm -f /var/KickStart/install/ks_env_tmp.in)
execute (rm -f /var/KickStart/install/ncsenv_tmp.sh)
execute (rm -f /var/KickStart/install/ncsenv_tmp.in)
execute (rm -f /tmp/ncsenv.tmp)
cleanning up done.......
/bin/cp: writing `/var/KickStart/install/ks_env_tmp.in': No space left on device
"/bin/cp -f /var/KickStart/install/ks_env.sh /var/KickStart/install/ks_enPlug and Play Gateway is success
fully disabled
Setup failure /ks_env.sh or /ncsenv.sh file not available
/opt/CSCOlumos/KickStart/bin/ncsconfigure.sh: line 216: $PNP_SETUP_LOG: ambiguous redirect

I found this

/dev/mapper/smosvg-varvol
                       4031680   4031680         0 100% /var

ncs cleanup did not help

What to do?

[ReichmuthM]

Hello

I get the same error message after installing “PI 3.1.4 Update 01” via GUI.

A restore attempt of the application did not solve the problem.

INFO: Restore completed successfully.
setup of pnp failed please see the /var/KickStart/install/setup.log

The problem is below the application on the level of the unix os. But without a root account there is nothing to fix.

The admin account has not enough permissions to delete files from the folder “/dev/mapper/smosvg-varvol”

Does anyone has a solution?

thx

Cisco Prime Infrastructure
********************************************************
Version : 3.1.0
Build : 3.1.0.0.132
Critical Fixes:
       PI 3.1.4 Maintenance Release ( 4.0.0 )


/dev/mapper/smosvg-varvol
            3.9G  3.9G     0 100% /var

[marce1000]

 - You should be able to get to a root-based shell, after being logged in  with 'admin'; following

# shell

# sudo -s 

M.

[Sergey Prishchepa]

And what to do?

[ReichmuthM · Accepted solution]

SSH Login as admin

shell

sudo su -

cd /var/cache/logwatch/

rm -rf logwatch.*

vi /usr/share/logwatch/default.conf/logwatch.conf

remove "# " from line "DailyReport = No"

:wq

check with:

df -h

and voila:

/dev/mapper/smosvg-varvol
                      3.9G  242M  3.5G   7% /var

thx for the help
Regards

Other interresting links to this topic: 
http://www.cisco.com/c/en/us/support/docs/cloud-systems-management/prime-infrastructure/200760-CLI-Root-User-Password-Recovery-Process.html

http://unix.stackexchange.com/questions/42857/how-to-extend-centos-5-partition-on-vmware#42910

[Sergey Prishchepa]

Reichmuth I did not try your method.

I have found that is not installed PI_3_1_4-1.0.38.ubf and reinstall the server.

It installed properly PI_3_1_4_update_01-1.0.11.ubf

Thank you all!

[dbergman13]

I wish I found this thread sooner. I ended up having to delete the /var/cache/logwatch files as well to make room for ncs to start after a power failure. Thanks for the information.

[Robert Singer]

Thank you for your threat. It helped at least to get the filesystem done in terms of working.

best regards

Robert

[afabrizio1]

Thanks for the post.  This helped us get our server back up and running after a reboot.  

[patoberli]

The logwatch.conf file nor directory exists in PI 3.5 :(
Searching now for an alternative solution for a full /var.

Issue is the /var/log/wtmp file, which uses nearly 80% of the partition.

[edit]

And I've already found a (temporary?) solution:

#enter shell:
shell
#get root
sudo su -
#write an empty new wtmp file (this can take a minute)
cat /dev/null > /var/log/wtmp

#In my case the messages file was also already > 500 MB, I'm not sure yet if I should also clean it. wtmp file was 2.7 GB though.

[patoberli]

Ok, fixed it (probably until the next patch/major release) by editing the /etc/logrotate.d/syslog file (plus all the others for good measure, where I also received an error message). I added the line ' su prime gadmin '

[root@cpi1 logrotate.d]# vi syslog 
/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler
{
    su prime gadmin    
    missingok
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

# To safe enter :wq
# It will logrotate all the files mentioned in this file within the next 24 hours automatically
# To manually rotate enter 'run-parts /etc/cron.daily/'

Please note, this will NOT cleanup the used space, as the files will not be compressed nor removed (they will be kept for 1-4 weeks). So if you need space now, remove the files with a .1, .2, .3 or .4 ending after logrotate has run. 

Please also note, logrotate can only successfully run if there is at least some space left. If the partition is completely full, clean the wtmp file first (or any other large file). 

[patoberli]

One more little addition, to rotate wtmp and btmp, edit the /etc/logrotate.conf to:

…
…
…
# other lines removed
/var/log/wtmp {
    su root guser    
    monthly
    create 0640 root guser
    rotate 1
}
/var/log/btmp {
    su root guser    
    weekly
    create 0600 root guser
    rotate 4
}
# other lines removed
…
…
.