cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1731
Views
8
Helpful
17
Replies
Highlighted
Beginner

CMX backup automation

Dear community:

We would like to create a job to automatically create a backup every night in the CMX.

We have created this script:

Inline image 4

When we execute the script when cmxadmin user, we have this error:

Inline image 2

With root user, we have another error:

Inline image 1

Any suggestion on how to proceed?

Is anybody doing unattended backup in CMX?

Thanks!

Carolina

17 REPLIES 17
Highlighted
Frequent Contributor

Re: CMX backup automation

Hi Carolina,

The images of the script and errors are not visible.  Please edit or resend message with images or text.

Thank you,

Matt

Highlighted
Beginner

Re: CMX backup automation

Hi Matt,

This is the script content:

cmxos backup --path /tmp --offline

When I try to execute it, I got this:

[cmxadmin@RKAVLMSETEST01 ~]$ ./backup.sh

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/runpy.py", line 151, in _run_module_as_main

    mod_name, loader, code, fname = _get_module_details(mod_name)

  File "/usr/local/lib/python2.7/runpy.py", line 109, in _get_module_details

    return _get_module_details(pkg_main_name)

  File "/usr/local/lib/python2.7/runpy.py", line 101, in _get_module_details

    loader = get_loader(mod_name)

  File "/usr/local/lib/python2.7/pkgutil.py", line 464, in get_loader

    return find_loader(fullname)

  File "/usr/local/lib/python2.7/pkgutil.py", line 474, in find_loader

    for importer in iter_importers(fullname):

  File "/usr/local/lib/python2.7/pkgutil.py", line 430, in iter_importers

    __import__(pkg)

  File "/opt/cmx/sw/lib/pymodules/cmxos/__init__.py", line 5, in <module>

    from __main__ import cmxos

  File "/opt/cmx/sw/lib/pymodules/cmxos/__main__.py", line 23, in <module>

    from cmxctl.util import Backup

  File "/opt/cmx/sw/lib/pymodules/cmxctl/util/__init__.py", line 7, in <module>

    from cmxconfig import CmxConfig

  File "/opt/cmx/sw/lib/pymodules/cmxctl/util/cmxconfig.py", line 22, in <module>

    from cmxagent.util import ParseConf

  File "/opt/cmx/sw/lib/pymodules/cmxagent/util/__init__.py", line 10, in <module>

    from qlessqueue import QlessQueue

  File "/opt/cmx/sw/lib/pymodules/cmxagent/util/qlessqueue.py", line 10, in <module>

    import cmxjobs

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/__init__.py", line 5, in <module>

    from redisCleanupTask import RedisCleanupTask

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/redisCleanupTask.py", line 5, in <module>

    from abctask import ABCTask

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 6, in <module>

    class ABCTask(object):

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 9, in ABCTask

    handler = logging.handlers.RotatingFileHandler(LOG_FILE, 'a', 1000000, 5)

  File "/usr/local/lib/python2.7/logging/handlers.py", line 117, in __init__

    BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)

  File "/usr/local/lib/python2.7/logging/handlers.py", line 64, in __init__

    logging.FileHandler.__init__(self, filename, mode, encoding, delay)

  File "/usr/local/lib/python2.7/logging/__init__.py", line 911, in __init__

    StreamHandler.__init__(self, self._open())

  File "/usr/local/lib/python2.7/logging/__init__.py", line 936, in _open

    stream = open(self.baseFilename, self.mode)

IOError: [Errno 13] Permission denied: '/opt/cmx/var/log/cmxjobs.log'

[cmxadmin@RKAVLMSETEST01 ~]$

With root I don't get any error now.

Is there any other way of doing this unattended backup?

Thanks.

Best regards,

Carolina

Highlighted
Frequent Contributor

Re: CMX backup automation

Hi Carolina,

Let me check with the CMX Developers for advise.  I will follow up soon.

Thanks,

Matt

Highlighted
Frequent Contributor

Re: CMX backup automation

Hi Carolina,

I found this for CMX 10.3 support case.  Hopefully this helps.

To retrieve the data for 1 day at a time.

Thus this script will loop through the days of June and produce and output file for each day of the month.  It looks like some of the days of June are already empty as we only keep 30 days of data.

for day in $(seq 1 30);do

  start=`date --date="$day-JUN-17" +%s`

  end=$((${start} + 86400))

  output_file="Jun$day-${start}000to${end}000.txt"

  echo "Processing $output_file"

  /opt/apache-cassandra-2.1.13/bin/cqlsh --request-timeout=300 cassandra.service.consul -e "select * from loc.hist_mob where changedon >${start}000 and changedon <${end}000 allow filtering" > /tmp/${output_file}

done

I create /home/cmxadmin/june.sh to wrap the logic above.  Since it would run quite a while, it would be good to run like this:

nohup /home/cmxadmin/june.sh &

The output files could be found in the /tmp directory.

I hope this helps.

Thanks

Highlighted
Beginner

Re: CMX backup automation

Is it me or does this reply have nothing to do with the question of how to automate the call to "cmxos backup --all --path /tmp --offline" ?
Highlighted
Frequent Contributor

Re: CMX backup automation

Hi ccoqutnoc,

Please see https://www.cisco.com/c/en/us/td/docs/wireless/mse/10-5/cmx_config/b_cg_cmx105/performing_administrative_tasks.html

for the following procedure to be added as a cron entry.

Thanks.

Highlighted
Beginner

Re: CMX backup automation

This is extremely unhelpful. Did you read the doco ? It doesn't outline using ftp, it says to allocate a NAS mount.



Running the "cmxos backup ..." works when you are logged in, but scripting that ? Forget it. You end up with the error the original poster listed on v10.4 CMX.




Using FTP Commands for Cisco CMX
You can use File Transfer Protocol (FTP) commands for backing up and restoring data on Cisco CMX 10.x. We recommend you to follow the below best practice for data backup automation:
Procedure
________________________________
Step 1

Setup a NAS Storage and mount it to your Cisco CMX box, for example, mount the storage to a local directory such as /mnt/nas.







Highlighted
Beginner

Re: CMX backup automation

If you are able to advise how to get the cmxos inside a #!/bin/sh call that’d be great.

This is the script I would be using but for the issue that cmxos fails when not run on an interactive ssh connection.

Autocreation script for data: ~/FTPbackup.sh
#!/bin/sh

SERVER=server
USER=user
PASS=password
WORKDIR=~/backups

#Move to working directory
mkdir -p $WORKDIR 2>/dev/null
cd $WORKDIR

if [ -r ~/FTPbackup.lock ]
then
echo "Lock file present, go see what didn't terminate in `host` cmxadmin ~/FTPbackup.lastRunLog"
exit
else
touch ~/FTPbackup.lock
echo "" > ~/FTPbackup.lastRunLog
fi

echo "lcd /tmp
cd cmx
binary
put `ls -1t cmx_backup* | head -n 1`

bye

" > ~/ftp-upload

echo "machine $SERVER
login $USER
password $PASS

" > ~/.netrc

echo "yes
" >~/yes

chmod 600 ~/.netrc

ls -1tr cmx_backup* | head -n -3 | xargs -d '\n' rm -f --
#/opt/cmx/bin/cmxos backup --all --path $WORKDIR --offline < ~/yes >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog
/opt/cmx/bin/cmxos backup -i conf --path $WORKDIR --online >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog

/usr/bin/ftp -i $SERVER < ~/ftp-upload >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog

#Clear run flag
/bin/rm -f ~/FTPbackup.lock

$ cat FTPbackup.lastRunLog

Traceback (most recent call last):
File "/usr/local/lib/python2.7/runpy.py", line 151, in _run_module_as_main
mod_name, loader, code, fname = _get_module_details(mod_name)
File "/usr/local/lib/python2.7/runpy.py", line 109, in _get_module_details
return _get_module_details(pkg_main_name)
File "/usr/local/lib/python2.7/runpy.py", line 101, in _get_module_details
loader = get_loader(mod_name)
File "/usr/local/lib/python2.7/pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "/usr/local/lib/python2.7/pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "/usr/local/lib/python2.7/pkgutil.py", line 430, in iter_importers
__import__(pkg)
File "/opt/cmx/sw/lib/pymodules/cmxos/__init__.py", line 7, in <module>
from cmxos import CMXOS
File "/opt/cmx/sw/lib/pymodules/cmxos/cmxos.py", line 36, in <module>
from cmxagent.util.prettyfmt import print_pretty_table
File "/opt/cmx/sw/lib/pymodules/cmxagent/util/__init__.py", line 10, in <module>
from qlessqueue import QlessQueue
File "/opt/cmx/sw/lib/pymodules/cmxagent/util/qlessqueue.py", line 10, in <module>
import cmxjobs
File "/opt/cmx/sw/lib/pymodules/cmxjobs/__init__.py", line 5, in <module>
from redisCleanupTask import RedisCleanupTask
File "/opt/cmx/sw/lib/pymodules/cmxjobs/redisCleanupTask.py", line 5, in <module>
from abctask import ABCTask
File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 6, in <module>
class ABCTask(object):
File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 9, in ABCTask
handler = logging.handlers.RotatingFileHandler(LOG_FILE, 'a', 1000000, 5)
File "/usr/local/lib/python2.7/logging/handlers.py", line 117, in __init__
BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
File "/usr/local/lib/python2.7/logging/handlers.py", line 64, in __init__
logging.FileHandler.__init__(self, filename, mode, encoding, delay)
File "/usr/local/lib/python2.7/logging/__init__.py", line 911, in __init__
StreamHandler.__init__(self, self._open())
File "/usr/local/lib/python2.7/logging/__init__.py", line 936, in _open
stream = open(self.baseFilename, self.mode)
IOError: [Errno 13] Permission denied: '/opt/cmx/var/log/cmxjobs.log'
Local directory now /tmp
(local-file) usage: put local-file remote-file

Highlighted
Beginner

Re: CMX backup automation

Also, as originally mentioned the command is not able to execute inside the shell, there are missing environment possibly. Running a 'set' command give variables but also a heap of functions that are not present in a /bin/sh environment.



I am able to automate the FTP upload myself, but getting the backup script to function inside a shell script is not working.



My test case:

[cmxadmin@itn-cmx02 ~]$ cat /home/cmxadmin/backup-cmx.sh

#!/bin/sh



cd /tmp



cmxos backup -i conf --path /tmp --online



[cmxadmin@itn-cmx02 ~]$ /home/cmxadmin/backup-cmx.sh

Traceback (most recent call last):

File "/usr/local/lib/python2.7/runpy.py", line 151, in _run_module_as_main

mod_name, loader, code, fname = _get_module_details(mod_name)

File "/usr/local/lib/python2.7/runpy.py", line 109, in _get_module_details

return _get_module_details(pkg_main_name)

File "/usr/local/lib/python2.7/runpy.py", line 101, in _get_module_details

loader = get_loader(mod_name)

File "/usr/local/lib/python2.7/pkgutil.py", line 464, in get_loader

return find_loader(fullname)

File "/usr/local/lib/python2.7/pkgutil.py", line 474, in find_loader

for importer in iter_importers(fullname):

File "/usr/local/lib/python2.7/pkgutil.py", line 430, in iter_importers

__import__(pkg)

File "/opt/cmx/sw/lib/pymodules/cmxos/__init__.py", line 7, in

from cmxos import CMXOS

File "/opt/cmx/sw/lib/pymodules/cmxos/cmxos.py", line 36, in

from cmxagent.util.prettyfmt import print_pretty_table

File "/opt/cmx/sw/lib/pymodules/cmxagent/util/__init__.py", line 10, in

from qlessqueue import QlessQueue

File "/opt/cmx/sw/lib/pymodules/cmxagent/util/qlessqueue.py", line 10, in

import cmxjobs

File "/opt/cmx/sw/lib/pymodules/cmxjobs/__init__.py", line 5, in

from redisCleanupTask import RedisCleanupTask

File "/opt/cmx/sw/lib/pymodules/cmxjobs/redisCleanupTask.py", line 5, in

from abctask import ABCTask

File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 6, in

class ABCTask(object):

File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 9, in ABCTask

handler = logging.handlers.RotatingFileHandler(LOG_FILE, 'a', 1000000, 5)

File "/usr/local/lib/python2.7/logging/handlers.py", line 117, in __init__

BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)

File "/usr/local/lib/python2.7/logging/handlers.py", line 64, in __init__

logging.FileHandler.__init__(self, filename, mode, encoding, delay)

File "/usr/local/lib/python2.7/logging/__init__.py", line 911, in __init__

StreamHandler.__init__(self, self._open())

File "/usr/local/lib/python2.7/logging/__init__.py", line 936, in _open

stream = open(self.baseFilename, self.mode)

IOError: [Errno 13] Permission denied: '/opt/cmx/var/log/cmxjobs.log'

[cmxadmin@itn-cmx02 ~]$


Highlighted
Beginner

Re: CMX backup automation

If you are able to advise how to get the cmxos inside a #!/bin/sh call that’d be great.

 

This is the script I would be using but for the issue that cmxos fails when not run on an interactive ssh connection.

 

Autocreation script for data: ~/FTPbackup.sh

#!/bin/sh

 

SERVER=server

USER=user

PASS=password

WORKDIR=~/backups

 

#Move to working directory

mkdir -p $WORKDIR 2>/dev/null

cd $WORKDIR

 

if [ -r ~/FTPbackup.lock ]

then

        echo "Lock file present, go see what didn't terminate in `host` cmxadmin ~/FTPbackup.lastRunLog"

        exit

else

        touch ~/FTPbackup.lock

        echo "" > ~/FTPbackup.lastRunLog

fi

 

echo "lcd /tmp

cd cmx

binary

put `ls -1t cmx_backup* | head -n 1`

 

bye

 

" > ~/ftp-upload

 

echo "machine $SERVER

        login $USER

        password $PASS

 

" > ~/.netrc

 

echo "yes

" >~/yes

 

chmod 600 ~/.netrc

 

ls -1tr cmx_backup* | head -n -3 | xargs -d '\n' rm -f --

#/opt/cmx/bin/cmxos backup --all --path $WORKDIR --offline < ~/yes >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog

/opt/cmx/bin/cmxos backup -i conf --path $WORKDIR --online         >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog

 

/usr/bin/ftp -i $SERVER < ~/ftp-upload >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog

 

#Clear run flag

/bin/rm -f ~/FTPbackup.lock

 

$ cat FTPbackup.lastRunLog

 

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/runpy.py", line 151, in _run_module_as_main

    mod_name, loader, code, fname = _get_module_details(mod_name)

  File "/usr/local/lib/python2.7/runpy.py", line 109, in _get_module_details

    return _get_module_details(pkg_main_name)

  File "/usr/local/lib/python2.7/runpy.py", line 101, in _get_module_details

    loader = get_loader(mod_name)

  File "/usr/local/lib/python2.7/pkgutil.py", line 464, in get_loader

    return find_loader(fullname)

  File "/usr/local/lib/python2.7/pkgutil.py", line 474, in find_loader

    for importer in iter_importers(fullname):

  File "/usr/local/lib/python2.7/pkgutil.py", line 430, in iter_importers

    __import__(pkg)

  File "/opt/cmx/sw/lib/pymodules/cmxos/__init__.py", line 7, in <module>

    from cmxos import CMXOS

  File "/opt/cmx/sw/lib/pymodules/cmxos/cmxos.py", line 36, in <module>

    from cmxagent.util.prettyfmt import print_pretty_table

  File "/opt/cmx/sw/lib/pymodules/cmxagent/util/__init__.py", line 10, in <module>

    from qlessqueue import QlessQueue

  File "/opt/cmx/sw/lib/pymodules/cmxagent/util/qlessqueue.py", line 10, in <module>

    import cmxjobs

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/__init__.py", line 5, in <module>

    from redisCleanupTask import RedisCleanupTask

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/redisCleanupTask.py", line 5, in <module>

    from abctask import ABCTask

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 6, in <module>

    class ABCTask(object):

  File "/opt/cmx/sw/lib/pymodules/cmxjobs/abctask.py", line 9, in ABCTask

    handler = logging.handlers.RotatingFileHandler(LOG_FILE, 'a', 1000000, 5)

  File "/usr/local/lib/python2.7/logging/handlers.py", line 117, in __init__

    BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)

  File "/usr/local/lib/python2.7/logging/handlers.py", line 64, in __init__

    logging.FileHandler.__init__(self, filename, mode, encoding, delay)

  File "/usr/local/lib/python2.7/logging/__init__.py", line 911, in __init__

    StreamHandler.__init__(self, self._open())

  File "/usr/local/lib/python2.7/logging/__init__.py", line 936, in _open

    stream = open(self.baseFilename, self.mode)

IOError: [Errno 13] Permission denied: '/opt/cmx/var/log/cmxjobs.log'

Local directory now /tmp

(local-file) usage: put local-file remote-file

 

 

Highlighted
Frequent Contributor

Re: CMX backup automation

Hi,

CMX documentation indicate that it's possible to use FTP to extract a backup from the appliance. CMX does not have support for FTP as of 10.2.  There is a bug report CSCuz66206 to remove using ftp for backup in the CMX config guide.  The Workaround: use SCP copy

Thanks.

 
 

 

Highlighted
Beginner

Re: CMX backup automation - FTP solution

Here is how I have it working. The cmxos will not run inside a script due to other environment variables missing. So here is how I have it working, with FTP and installed to crontab:

 

#!/bin/sh

# Cisco CMX crontab backup to FTP server
# Antony Somerville 7/August/2018
# install into /home/cmxadmin and crontab execute for 'cmxadmin' user:
# 0 1 * * * 0 /home/cmxadmin/cmx_backup.sh

SERVER=servername
USER=username
PASS=userpasswd

# use /tmp or the DB dump fails
WORKDIR=/tmp

 

#<start .bashrc> pull this from the ~/.bashrc
# Auto complete files created for CMX commands, creates
# the environment to allow cmxos to run
. /etc/bash_completion.d/cmxctl.bash
. /etc/bash_completion.d/cmxha.bash
. /etc/bash_completion.d/cmxos.bash
. /etc/bash_completion.d/cmxloc.bash

alias cmxctl='sudo -u cmx /opt/cmx/bin/cmxctl --prompt'
alias cmxloc='sudo -u cmx /opt/cmx/bin/cmxloc'
alias cmxha='sudo -u cmxadmin /opt/cmx/bin/cmxha'
alias proxyctl='sudo -u cmx /opt/cmx/bin/proxyctl'
alias nodesetup='sudo -u cmx /opt/cmx/bin/nodesetup'
alias cmxos='sudo -u root /opt/cmx/bin/cmxos'
alias cmxverify='sudo -u root /opt/cmx/bin/cmxverify'
#
#<end .bashrc>

#Move to working directory
mkdir -p $WORKDIR 2>/dev/null
cd $WORKDIR

if [ -r ~/FTPbackup.lock ]
then
echo "Lock file present, go see what didn't terminate in `host` cmxadmin ~/FTPbackup.lastRunLog"
exit
else
touch ~/FTPbackup.lock
echo "" > ~/FTPbackup.lastRunLog
fi

echo "machine $SERVER
login $USER
password $PASS

" > ~/.netrc

echo "yes
" >~/yes

chmod 600 ~/.netrc

ls -1tr cmx_backup* 2>/dev/null | head -n -3 | xargs -d '\n' rm -f --
cmxos backup --all --path $WORKDIR --offline < ~/yes >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog
#mini backup for testing purposes
#cmxos backup -i conf --path $WORKDIR --online >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog

echo "lcd $WORKDIR
cd cmx
binary
put `ls -1t cmx_backup* | head -n 1`

bye

" > ~/ftp-upload

/usr/bin/ftp -i $SERVER < ~/ftp-upload >>~/FTPbackup.lastRunLog 2>>~/FTPbackup.lastRunLog

#Clear run flag
/bin/rm -f ~/FTPbackup.lock

 

Highlighted
Beginner

Re: CMX backup automation - FTP solution

Hi ccoqutnoc,

 

Thanks for the script.

Manually executing the script works perfectly but when executed via crontab, nothing happens.

It seems a permission issue with crontab.

 

Did you modify something to have your script working on your appliance?

 

Thanks,

Sylvain.

Highlighted
Beginner

Re: CMX backup automation - FTP solution

Finally found my issue.

Backup are working fine now.

 

Sylvain.

Content for Community-Ad
Cisco Community July 2020 Spotlight Award Winners
This widget could not be displayed.