Solved: Re: Question on YDK, pip, and Virtual Environments

brfoster · ‎10-20-2016

Team,

I'm not quite understanding something with YDK.

I created a fresh setup of YDK as shown below...

[gibson@rickenbacker PythonProjects]$ git clone https://github.com/CiscoDevNet/ydk-py.git

Initialized empty Git repository in /home/gibson/PythonProjects/ydk-py/.git/

remote: Counting objects: 4461, done.

remote: Compressing objects: 100% (12/12), done.

remote: Total 4461 (delta 5), reused 0 (delta 0), pack-reused 4449

Receiving objects: 100% (4461/4461), 67.25 MiB | 5.84 MiB/s, done.

Resolving deltas: 100% (3103/3103), done.

[gibson@rickenbacker PythonProjects]$

[gibson@rickenbacker PythonProjects]$ cd virtualenvs/

[gibson@rickenbacker virtualenvs]$ virtualenv -p python2.7 ydk-py-venv

Already using interpreter /usr/bin/python2.7

New python executable in /home/gibson/PythonProjects/virtualenvs/ydk-py-venv/bin/python2.7

Also creating executable in /home/gibson/PythonProjects/virtualenvs/ydk-py-venv/bin/python

Installing setuptools, pip, wheel...done.

[gibson@rickenbacker virtualenvs]$

[gibson@rickenbacker virtualenvs]$ source ydk-py-venv/bin/activate

(ydk-py-venv) [gibson@rickenbacker virtualenvs]$

(ydk-py-venv) [gibson@rickenbacker virtualenvs]$ which python

~/PythonProjects/virtualenvs/ydk-py-venv/bin/python

(ydk-py-venv) [gibson@rickenbacker virtualenvs]$

(ydk-py-venv) [gibson@rickenbacker virtualenvs]$ which pip

~/PythonProjects/virtualenvs/ydk-py-venv/bin/pip

(ydk-py-venv) [gibson@rickenbacker virtualenvs]$ pip list

pip (8.1.2)

setuptools (28.6.1)

wheel (0.30.0a0)

(ydk-py-venv) [gibson@rickenbacker virtualenvs]$

Then I installed the YDK core module as shown below...

(ydk-py-venv) [gibson@rickenbacker virtualenvs]$ cd ../ydk-py

(ydk-py-venv) [gibson@rickenbacker ydk-py]$ ls

CHANGES.md cisco-ios-xr Copyright core ietf LICENSE openconfig README.rst

(ydk-py-venv) [gibson@rickenbacker ydk-py]$ cd core

(ydk-py-venv) [gibson@rickenbacker core]$

(ydk-py-venv) [gibson@rickenbacker core]$ python setup.py sdist

running sdist

running egg_info

creating ydk.egg-info

writing requirements to ydk.egg-info/requires.txt

writing ydk.egg-info/PKG-INFO

writing top-level names to ydk.egg-info/top_level.txt

writing dependency_links to ydk.egg-info/dependency_links.txt

writing manifest file 'ydk.egg-info/SOURCES.txt'

reading manifest file 'ydk.egg-info/SOURCES.txt'

reading manifest template 'MANIFEST.in'

writing manifest file 'ydk.egg-info/SOURCES.txt'

warning: sdist: standard file not found: should have one of README, README.rst, README.txt

running check

creating ydk-0.5.1

creating ydk-0.5.1/ydk

creating ydk-0.5.1/ydk.egg-info

creating ydk-0.5.1/ydk/_core

creating ydk-0.5.1/ydk/mdt

creating ydk-0.5.1/ydk/models

creating ydk-0.5.1/ydk/providers

creating ydk-0.5.1/ydk/services

copying files to ydk-0.5.1...

copying MANIFEST.in -> ydk-0.5.1

copying README.md -> ydk-0.5.1

copying setup.py -> ydk-0.5.1

copying ydk/__init__.py -> ydk-0.5.1/ydk

copying ydk/errors.py -> ydk-0.5.1/ydk

copying ydk/types.py -> ydk-0.5.1/ydk

copying ydk.egg-info/PKG-INFO -> ydk-0.5.1/ydk.egg-info

copying ydk.egg-info/SOURCES.txt -> ydk-0.5.1/ydk.egg-info

copying ydk.egg-info/dependency_links.txt -> ydk-0.5.1/ydk.egg-info

copying ydk.egg-info/requires.txt -> ydk-0.5.1/ydk.egg-info

copying ydk.egg-info/top_level.txt -> ydk-0.5.1/ydk.egg-info

copying ydk/_core/__init__.py -> ydk-0.5.1/ydk/_core

copying ydk/_core/_dm_meta_info.py -> ydk-0.5.1/ydk/_core

copying ydk/mdt/__init__.py -> ydk-0.5.1/ydk/mdt

copying ydk/mdt/event_tcp.py -> ydk-0.5.1/ydk/mdt

copying ydk/mdt/proto_to_dict.py -> ydk-0.5.1/ydk/mdt

copying ydk/mdt/proto_to_ydk.py -> ydk-0.5.1/ydk/mdt

copying ydk/mdt/receiver.py -> ydk-0.5.1/ydk/mdt

copying ydk/mdt/simple_tcp.py -> ydk-0.5.1/ydk/mdt

copying ydk/mdt/telemetry_kv_pb2.py -> ydk-0.5.1/ydk/mdt

copying ydk/models/__init__.py -> ydk-0.5.1/ydk/models

copying ydk/providers/__init__.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_decoder.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_encoder.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_importer.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_provider_plugin.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_session_config.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_validator.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_value_encoder.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/_ydk_types.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/codec_provider.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/native_netconf_provider.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/netconf_provider.py -> ydk-0.5.1/ydk/providers

copying ydk/providers/provider.py -> ydk-0.5.1/ydk/providers

copying ydk/services/__init__.py -> ydk-0.5.1/ydk/services

copying ydk/services/codec_service.py -> ydk-0.5.1/ydk/services

copying ydk/services/crud_service.py -> ydk-0.5.1/ydk/services

copying ydk/services/executor_service.py -> ydk-0.5.1/ydk/services

copying ydk/services/meta_service.py -> ydk-0.5.1/ydk/services

copying ydk/services/netconf_service.py -> ydk-0.5.1/ydk/services

copying ydk/services/service.py -> ydk-0.5.1/ydk/services

Writing ydk-0.5.1/setup.cfg

creating dist

Creating tar archive

removing 'ydk-0.5.1' (and everything under it)

(ydk-py-venv) [gibson@rickenbacker core]$

(ydk-py-venv) [gibson@rickenbacker core]$ pip install dist/ydk-0.5.1.tar.gz

Processing ./dist/ydk-0.5.1.tar.gz

Requirement already satisfied (use --upgrade to upgrade): ydk==0.5.1 from file:///home/gibson/PythonProjects/ydk-py/core/dist/ydk-0.5.1.tar.gz in /home/gibson/PythonProjects/ydk-py/core

Collecting ecdsa==0.13 (from ydk==0.5.1)

Using cached ecdsa-0.13-py2.py3-none-any.whl

Collecting enum34==1.1.3 (from ydk==0.5.1)

Using cached enum34-1.1.3-py2.py3-none-any.whl

Collecting lxml==3.4.4 (from ydk==0.5.1)

Collecting paramiko==1.15.2 (from ydk==0.5.1)

Using cached paramiko-1.15.2-py2.py3-none-any.whl

Collecting pyang==1.6 (from ydk==0.5.1)

Using cached pyang-1.6-py2.py3-none-any.whl

Collecting pycrypto==2.6.1 (from ydk==0.5.1)

Collecting Twisted>=16.0.0 (from ydk==0.5.1)

Collecting protobuf==3.0.0b2.post2 (from ydk==0.5.1)

Using cached protobuf-3.0.0b2.post2-py2-none-any.whl

Collecting ncclient>=0.4.7 (from ydk==0.5.1)

Collecting zope.interface>=3.6.0 (from Twisted>=16.0.0->ydk==0.5.1)

Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/gibson/PythonProjects/virtualenvs/ydk-py-venv/lib/python2.7/site-packages (from protobuf==3.0.0b2.post2->ydk==0.5.1)

Collecting six>=1.9 (from protobuf==3.0.0b2.post2->ydk==0.5.1)

Using cached six-1.10.0-py2.py3-none-any.whl

Building wheels for collected packages: ydk

Running setup.py bdist_wheel for ydk ... done

Stored in directory: /home/gibson/.cache/pip/wheels/73/68/7b/ad30d2c8d6b51e430590eeca4b2ac13d502f705e34dfb29f87

Successfully built ydk

Installing collected packages: ecdsa, enum34, lxml, pycrypto, paramiko, pyang, zope.interface, Twisted, six, protobuf, ncclient

Successfully installed Twisted-16.4.1 ecdsa-0.13 enum34-1.1.3 lxml-3.4.4 ncclient-0.5.2 paramiko-1.15.2 protobuf-3.0.0b2.post2 pyang-1.6 pycrypto-2.6.1 six-1.10.0 zope.interface-4.3.2

(ydk-py-venv) [gibson@rickenbacker core]$

When I list the installed packages with "pip list" I can see ydk(0.5.1) is there...

(ydk-py-venv) [gibson@rickenbacker core]$ pip list

ecdsa (0.13)

enum34 (1.1.3)

lxml (3.4.4)

ncclient (0.5.2)

paramiko (1.15.2)

pip (8.1.2)

protobuf (3.0.0b2.post2)

pyang (1.6)

pycrypto (2.6.1)

setuptools (28.6.1)

six (1.10.0)

Twisted (16.4.1)

wheel (0.30.0a0)

ydk (0.5.1)

zope.interface (4.3.2)

(ydk-py-venv) [gibson@rickenbacker core]$

However, when I change directories I no longer see it...

(ydk-py-venv) [gibson@rickenbacker core]$ cd

(ydk-py-venv) [gibson@rickenbacker ~]$

(ydk-py-venv) [gibson@rickenbacker ~]$ pwd

/home/gibson

(ydk-py-venv) [gibson@rickenbacker ~]$ pip list

ecdsa (0.13)

enum34 (1.1.3)

lxml (3.4.4)

ncclient (0.5.2)

paramiko (1.15.2)

pip (8.1.2)

protobuf (3.0.0b2.post2)

pyang (1.6)

pycrypto (2.6.1)

setuptools (28.6.1)

six (1.10.0)

Twisted (16.4.1)

wheel (0.30.0a0)

zope.interface (4.3.2)

(ydk-py-venv) [gibson@rickenbacker ~]$

Also, I expected to see ydk in the "site-packages" directory of the virtual environment Python instance...

(ydk-py-venv) [gibson@rickenbacker ~]$

(ydk-py-venv) [gibson@rickenbacker ~]$ cd PythonProjects/virtualenvs/ydk-py-venv/lib/python2.7/site-packages/

(ydk-py-venv) [gibson@rickenbacker site-packages]$ ls -l

total 192

drwxrwxr-x. 10 gibson gibson 4096 Oct 20 12:10 Crypto

-rw-rw-r--. 1 gibson gibson 126 Oct 20 12:08 easy_install.py

-rw-rw-r--. 1 gibson gibson 315 Oct 20 12:08 easy_install.pyc

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 ecdsa

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 ecdsa-0.13.dist-info

drwxrwxr-x. 3 gibson gibson 4096 Oct 20 12:10 enum

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 enum34-1.1.3.dist-info

drwxrwxr-x. 3 gibson gibson 4096 Oct 20 12:10 google

drwxrwxr-x. 5 gibson gibson 4096 Oct 20 12:10 lxml

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 lxml-3.4.4.dist-info

drwxrwxr-x. 5 gibson gibson 4096 Oct 20 12:10 ncclient

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 ncclient-0.5.2.dist-info

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 paramiko

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 paramiko-1.15.2.dist-info

drwxrwxr-x. 10 gibson gibson 4096 Oct 20 12:08 pip

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:08 pip-8.1.2.dist-info

drwxrwxr-x. 4 gibson gibson 4096 Oct 20 12:08 pkg_resources

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 protobuf-3.0.0b2.post2.dist-info

-rw-rw-r--. 1 gibson gibson 305 Oct 20 12:10 protobuf-3.0.0b2.post2-py2.7-nspkg.pth

drwxrwxr-x. 4 gibson gibson 4096 Oct 20 12:10 pyang

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 pyang-1.6.dist-info

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 pycrypto-2.6.1.dist-info

drwxrwxr-x. 4 gibson gibson 4096 Oct 20 12:08 setuptools

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:08 setuptools-28.6.1.dist-info

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 six-1.10.0.dist-info

-rw-rw-r--. 1 gibson gibson 30098 Oct 20 12:10 six.py

-rw-rw-r--. 1 gibson gibson 29545 Oct 20 12:10 six.pyc

drwxrwxr-x. 26 gibson gibson 4096 Oct 20 12:10 twisted

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 Twisted-16.4.1.dist-info

drwxrwxr-x. 5 gibson gibson 4096 Oct 20 12:08 wheel

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:08 wheel-0.30.0a0.dist-info

drwxrwxr-x. 3 gibson gibson 4096 Oct 20 12:10 zope

drwxrwxr-x. 2 gibson gibson 4096 Oct 20 12:10 zope.interface-4.3.2.dist-info

-rw-rw-r--. 1 gibson gibson 349 Oct 20 12:10 zope.interface-4.3.2-py2.7-nspkg.pth

(ydk-py-venv) [gibson@rickenbacker site-packages]$

But it isn't there! What am I missing here?

abhirame · ‎10-21-2016

We figured out that 'pip list' was showing ydk if we were in the ydk-py/core directory. Not sure why. Could be because somehow the PYTHONPATH was getting modified, but 'echo $PYTHONPATH' was not showing anything? After installing the tar file with 'pip install dist/*.tar.gz', pip was correctly showing ydk no matter which directory we were in.

BTW I couldn't reproduce this behavior on my centos VM either.

View solution in original post

abhirame · ‎10-20-2016

Hi Jonathon,

I'm not able to reproduce this. When I install ydk using 'pip install', ydk shows up in the 'pip list' and in the virtualenv python site-packages, no matter which directory I'm in. Can you try to delete and create the virtualenv and try to install ydk again?

brfoster · ‎10-20-2016

Actually, I did those steps twice already! In the log I posted earlier I had completely deleted the venv and ydk installation directories and started over.

Could we do a WebEx?

abhirame · ‎10-20-2016

Sure. This is my link Cisco - Internal Sites WebEx Enterprise Site

Thanks

saalvare · ‎10-20-2016

Can you import ydk in the Python interpreter? I experience a somewhat related issue where pip doesn't list ydk even though it is installed and operation. I only see the issue in my system install. Not in virtual environments.

$ pip list | grep ydk

$ python

Python 2.7.12 (default, Jul 1 2016, 15:12:24)

[GCC 5.4.0 20160609] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import ydk

>>>

In my experience, pip isn't always very reliable and, every once in a while, it get into inconsistent states.

abhirame · ‎10-21-2016

We figured out that 'pip list' was showing ydk if we were in the ydk-py/core directory. Not sure why. Could be because somehow the PYTHONPATH was getting modified, but 'echo $PYTHONPATH' was not showing anything? After installing the tar file with 'pip install dist/*.tar.gz', pip was correctly showing ydk no matter which directory we were in.

BTW I couldn't reproduce this behavior on my centos VM either.