cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

Python 2 and 3 packages at the same time and together with virtualenv

161
Views
10
Helpful
0
Comments
Cisco Employee

Intro

Want to run both python2 and python3 packages at the same time? Maybe you don't want to upgrade all your python2 services at the same time? With some small changes to the "ncs-python-start-vm" script you can. While you are at it maybe you would like to start using virtual environments in python? With a few more lines in "ncs-python-start-vm" you can have that too. 

After the script changes all you need to do to run some packages with python3 is to put the package name in to the python3packages.txt file. If the package name isn't in the file it will just use the default python version. 

If "ncs-python-start-vm" finds a virtual environment in your NSO package (in reality looking for the venv activate command) it will activate that environment first before starting the python vm. 

Disclaimer, you will not get any Cisco support for any of these changes. 

Installation

If you are running a local installation (in a system install you have to change the paths below) you just have to create a bin folder in you NSO runtime folder and create your own copy of ncs-start-python-vm.

cd your_nso_runtime/
mkdir bin
cp $NCS_DIR/bin/ncs-start-python-vm bin/
touch bin/python3packages.txt

Modify your copy of ncs-python-start-vm script to the below or even better get it from the git repo (you might have to add the execute bit to the script, chmod +x ncs-python-start-vm)

#!/bin/sh

pypath="${NCS_DIR}/src/ncs/pyapi"

# Make sure everyone finds the NCS Python libraries at startup
if [ "x$PYTHONPATH" != "x" ]; then
    PYTHONPATH=${pypath}:$PYTHONPATH
else
    PYTHONPATH=${pypath}
fi
export PYTHONPATH

main="${pypath}/ncs_pyvm/startup.py"


echo "Starting ${main} $*"

# input argument 6 is the package name # activating the virtual environment for the package for path in ${PYTHONPATH//:/ }; do basename=$(basename $path) package=$(basename $(dirname $path)) packagepath=$(dirname $path) #locate the python package path in $PYTHONPATH and check if it has a virtualenv if [ "$basename" == "python" ] && [ "$package" == "$6" ] && [ -f "$packagepath/env/bin/activate" ]; then echo "Found virtualenv for package $6, activating it" source $packagepath/env/bin/activate fi done # bin/python3packages.txt should be in a bin folder in the NSO runtime folder if [ ! -z $(grep $6 "bin/python3packages.txt") ]; then echo "Starting python v3 vm for package $6" exec python3 -u ${main} $* else echo "Starting the system default python vm for package $6" exec python -u ${main} $* fi

Then change your ncs.conf so that it executes your new ncs-start-python-vm script instead of the default one.

vi ncs.conf
....
   <python-vm>
       <start-command>bin/ncs-start-python-vm</start-command>
   </python-vm>
....

After that just add the packages you want to be running with python3 to the python3packages.txt file

cat python3packages.txt
external-id-allocator
selftest
l3vpn

Restart NSO, the restart is only needed for NSO to read in the new python vm startup script, might even be enough with a ncs --reload. Adding new packages to python3packages.txt will not require a reload.

ncs --stop
ncs

 

Using virtualenv

Installation

virtualenv is used to manage Python packages for different projects. Using virtualenv allows you to avoid installing Python packages globally which could break system tools or other projects. You can install virtualenv using pip.

python3 -m pip install --user virtualenv

 

Creating a virtual environment 

venv (for Python 3) and virtualenv (for Python 2) allow you to manage separate package installations for different projects. They essentially allow you to create a “virtual” isolated Python installation and install packages into that virtual installation. When you switch projects, you can simply create a new virtual environment and not have to worry about breaking the packages installed in the other environments. It is always recommended to use a virtual environment while developing Python applications.

 

To create a virtual environment for you NSO package, go to your package directory and run venv. If you are using Python 2, replace venv with virtualenv in the below commands.

cd packages/l3vpn
python3 -m venv env

 

Activating a virtual environment

Before you can start installing or using packages in your virtual environment you’ll need to activate it. Activating a virtual environment will put the virtual environment-specific python and pip executables into your shell’s PATH.

source env/bin/activate

if you want to switch projects or otherwise leave your virtual environment, simply run:

deactivate

 

Installing packages in your virtualenv

Now that you’re in your virtual environment you can install packages. Let’s install the excellent Requests library from the Python Package Index (PyPI):

pip install requests

 

Using requirements files

Instead of installing packages individually, pip allows you to declare all dependencies in a Requirements File. For example you could create a requirements.txt file containing:

requests==2.18.4
google-auth==1.1.0

And tell pip to install all of the packages in this file using the -r flag:

pip install -r requirements.txt

Pip can export a list of all installed packages and their versions using the freeze command:

pip freeze

Which will output a list of package specifiers such as:

cachetools==2.0.1
certifi==2017.7.27.1
chardet==3.0.4
google-auth==1.1.1
idna==2.6
pyasn1==0.3.6
pyasn1-modules==0.1.4
requests==2.18.4
rsa==3.4.2
six==1.11.0
urllib3==1.22

This is useful for creating Requirements Files that can re-create the exact versions of all packages installed in an environment.