Fedora 27 and libydk installation issues

[jbritt]

I cannot install the ydk without installing libydk. However I cannot install the libydk rpm as it asks for the epel-release repository. In Fedora the epel-release repository doesn't exist because everything within that repository is in the default repository for Fedora. Can I compile libydk from source or get an RPM that doesn't have the epel-release requirement?

[jbritt · Accepted solution]

My apologies I just realized I could remove the epel-release requirement myself. I used rpmrebuild to modify the requirements list and removed that entry. I was able to successfully install the libydk afterwards and then install ydk-py with pip3 install ydk.

So for Fedora the install process looks like this:

# Install all required prereqs for ydk

sudo dnf install libssh-devel libxml2-devel libxslt-devel pcre-devel cmake gcc-c++ python3-devel libcurl-devel clang

# download libydk

wget https://devhub.cisco.com/artifactory/rpm-ydk/0.7.0/libydk-0.7.0-1.x86_64.rpm

# Install rpm requirement editing tool

sudo dnf install rpmrebuild

# Edit RPM and remove EPEL-RELEASE

rpmrebuild -e -p libydk-0.7.0-1.x86_64.rpm

- delete "Requires:      epel-release" and save

- 'y' to continue

cd /home/[user]/rpmbuild/RPMS/x86_64

# run modified RPM

sudo dnf install libydk-0.7.0-1.x86_64.rpm

# Install ydk-py

[sudo] pip3 install ydk

I wasn't able to find this documented anywhere, please correct me if I'm wrong.

[abhirame]

Thanks for the update! We could update the installation document with this. Let me know in case of any further issues.

[jbritt]

Yes, updating the documentation would be great. I have had no issues since then.

[Johan Marcusson]

I just installed the other depedencies for the rpm by hand and then installed the libydk rpm with --nodeps, but I have another issue after the installation. I've created a virtualenv and successfully installed ydk with pip, pip list output:

pip (9.0.2)

pybind11 (2.2.2)

setuptools (39.0.1)

wheel (0.30.0)

ydk (0.7.0)

ydk-models-cisco-ios-xr (6.3.1)

ydk-models-ietf (0.1.4)

ydk-models-openconfig (0.1.4)

But when I try to import anything or run the provided sample bgp.py I get errors from a .so file:

$ python bgp.py

Traceback (most recent call last):

  File "bgp.py", line 29, in <module>

    from ydk.types import Empty

  File "/home/joma03/python/ydk-py/lib/python2.7/site-packages/ydk/types/__init__.py", line 17, in <module>

    from .py_types import Entity, YList, YLeafList

  File "/home/joma03/python/ydk-py/lib/python2.7/site-packages/ydk/types/py_types.py", line 24, in <module>

    from ydk_ import is_set

ImportError: /home/joma03/python/ydk-py/lib/python2.7/site-packages/ydk_.so: undefined symbol: _ZNK3ydk6Entity31get_bundle_yang_models_locationB5cxx11Ev

Has anyone else seen this? I'm also on Fedora 27

[abhirame]

Hi Johan,

The ydk python core is using C extensions which are compiled into this .so file.

For this issue, I remember seeing this if gcc is older version. Just checking if you have upgraded the gcc as specified in the README?

# Upgrade compiler to gcc 5.*
$ yum install centos-release-scl -y > /dev/null
$ yum install devtoolset-4-gcc* -y > /dev/null
$ ln -sf /opt/rh/devtoolset-4/root/usr/bin/gcc /usr/bin/cc
$ ln -sf /opt/rh/devtoolset-4/root/usr/bin/g++ /usr/bin/c++

What is the output of 'c++ --version' for you?

Thanks

[Johan Marcusson]

$ c++ --version

c++ (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)

I don't think I can downgrade to gcc 5.*

[Johan Marcusson]

I'm not very good at this stuff but I've tried to troubelshoot a bit on my own and I think the ydk_.so that is generated when doing pip install on the core part is not linked to my /usr/lib/libydk.a.

I've installed from source by doing:

$ cd core

core$ python setup.py sdist

core$ pip install dist/ydk*.gz

When running sample/bgp.py I get:

ImportError: /home/joma03/python/ydk-py/lib/python2.7/site-packages/ydk_.so: undefined symbol: _ZNK3ydk6Entity31get_bundle_yang_models_locationB5cxx11Ev

$ nm -g /home/joma03/python/ydk-py/lib/python2.7/site-packages/ydk_.so | grep _ZNK3ydk6Entity31get_bundle_yang_models_locationB5cxx11Ev

                 U _ZNK3ydk6Entity31get_bundle_yang_models_locationB5cxx11Ev

So it's undefined here, but I guess that's ok since it should be defined in the shared library distributed in the RPM?

And I can find a definition there:

nm -g /usr/lib/libydk.a | grep get_bundle_yang_models_location

00000000000000c0 T _ZNK3ydk6Entity31get_bundle_yang_models_locationEv

                 U _ZNK3ydk6Entity31get_bundle_yang_models_locationEv

However if I check what the ydk_.so from core is linked against I don't see anything pointing to the library from the RPM:

$ ldd /home/joma03/python/ydk-py/lib/python2.7/site-packages/ydk_.so | grep ydk

$

(no matches)

Is this something that should be handled in the Cmake file? I don't think I know how to fix this myself

[abhirame]

Can you please try the steps to build everything from source? Please remove the installed libydk & ydk python core:

 rpm -q libydk
pip uninstall ydk -y

Then run the below steps:

$ ./generate.py --libydk
$ cd gen-api/cpp/ydk/build
$ make

# To create the libydk binary package to use for later installation, run the below command
$ make package

# To install the compiled libydk code, run the below command after the above
$ [sudo] make install


$ ./generate.py --python --core
$ pip install gen-api/python/ydk/dist/ydk*.tar.gz

[Johan Marcusson]

So the reason I didn't see libydk in the ldd output is that it's statically linked. I found out that the symbols doesn't match because of the cxx11 part, and this is apparently some behavior that changed in GCC5.1 to be compatible with C++ 2011 standard? https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

So the reason it doesn't work is the RPM was compiled with GCC older than 5.1, right? And I'm using GCC7 when compiling ydk-py. So I tried compiling the rpm from source on my machine to get the correct ABI compatability

Now I get a new error when trying to do pip install on the ydk-py core:

  In file included from /home/joma03/python/ydk-py/include/site/python2.7/pybind11/operators.h:12:0,

                   from /tmp/pip-LW_cHq-build/python.cpp:16:

  /home/joma03/python/ydk-py/include/site/python2.7/pybind11/pybind11.h:1132:13: note: candidate: template<class ... Args, class ... Extra> pybind11::class_<type_, options>& pybind11::class_<type_, options>::def(pybind11::detail::initimpl::pickle_factory<Args ...>&&, const Extra& ...) [with Args = {Args ...}; Extra = {Extra ...}; type_ = ydk::NetconfService; options = {}]

       class_ &def(detail::initimpl::pickle_factory<Args...> &&pf, const Extra &...extra) {

               ^~~

  /home/joma03/python/ydk-py/include/site/python2.7/pybind11/pybind11.h:1132:13: note:   template argument deduction/substitution failed:

  /tmp/pip-LW_cHq-build/python.cpp:955:82: note:   mismatched types ‘pybind11::detail::initimpl::pickle_factory<Args ...>’ and ‘const char [12]’

               arg("error_option") = std::string{""}, return_value_policy::reference)

                                                                                    ^

  gmake[2]: *** [CMakeFiles/ydk_.dir/build.make:63: CMakeFiles/ydk_.dir/python.cpp.o] Error 1

  gmake[2]: Leaving directory '/tmp/pip-LW_cHq-build/build/temp.linux-x86_64-2.7'

  gmake[1]: *** [CMakeFiles/Makefile2:68: CMakeFiles/ydk_.dir/all] Error 2

  gmake[1]: Leaving directory '/tmp/pip-LW_cHq-build/build/temp.linux-x86_64-2.7'

  gmake: *** [Makefile:84: all] Error 2

[abhirame]

I believe the libydk RPM was generated using 5.3.1

cc --version
cc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

About the new error, it looks like there is some issue with pybind11 and gcc7. Can you please check the version of pybind11 you have?

pip list|grep pybind

[Johan Marcusson]

Ok sure:

$ pip list|grep pybind

pybind11 (2.2.2)

[Johan Marcusson]

Any ideas about pybind?

[abhirame]

Sorry for the delay. I was able to install and run ydk 0.7.0 on Fedora27 with the below steps on a fresh Fedora27 image:

# Install system dependencies
yum install git which libxml2-devel libxslt-devel libssh-devel libtool gcc-c++ pcre-devel cmake wget curl-devel unzip python-devel python-pip make go sudo rpm-build redhat-lsb l -y

# Install libydk from source
git clone https://github.com/CiscoDevNet/ydk-gen.git
cd ydk-gen
git checkout 0.7.0
pip install -r requirements.txt
./generate.py --core --cpp
make -C gen-api/cpp/ydk/build install

# Install ydk-py packages
pip install ydk
pip install --install-option="--install-purelib=/usr/lib64/python2.7/site-packages" --no-deps ydk-models-ietf
pip install --install-option="--install-purelib=/usr/lib64/python2.7/site-packages" --no-deps ydk-models-openconfig
pip install --install-option="--install-purelib=/usr/lib64/python2.7/site-packages" --no-deps ydk-models-cisco-ios-xr
pip install --install-option="--install-purelib=/usr/lib64/python2.7/site-packages" --no-deps ydk-models-cisco-ios-xe