Pioneer's Internal compiler error

[Iman1]

I'm trying to build netconf NED for Nokia (ALU) device. Following the Pioneer approach I encounter an Internal compiler error (please see below). I'm going to guess it's due to problems in Nokia's yang but my question is how can I go about troubleshooting this and find which yang is problematic? There is a line below that says See log for complete details, but I couldn't locate any log related to this. Where does Pioneer store logs related to error below? Is there a way to troubleshoot further and figure out which of the Nokia's yang modules are the problem? Thanks.

admin@ncs% request pioneer yang build-netconf-ned yang-directory /win7/Nokia/15.0.R4_ALL_YANG/YANG/ ned-directory /root/ncs/4.4.2/ncs-4.4.2/packages/neds name 15r4

Cleaning up existing ned-directory

Starting build of 151 YANG modules, this may take some time

PATH=/root/ncs/4.4.2/ncs-4.4.2//bin:/root/ncs/4.4.2/ncs-4.4.2/lib/ncs/erts/bin:/root/ncs/4.4.2/ncs-4.4.2/lib/ncs/bin:/root/ncs/4.4.2/ncs-4.4.2//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin:/root/go/bin ncs-make-package --verbose --netconf-ned /win7/Nokia/15.0.R4_ALL_YANG/YANG/ 15r4 --dest /root/ncs/4.4.2/ncs-4.4.2/packages/neds --no-java

Wrote package to /root/ncs/4.4.2/ncs-4.4.2/packages/neds

make clean fxs -C /root/ncs/4.4.2/ncs-4.4.2/packages/neds/src NCSC=/root/ncs/4.4.2/ncs-4.4.2//bin/ncsc --verbose

make: Entering directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/src'

rm -rf ncsc-out ../load-dir

cd ../netsim && make clean || true

make[1]: Entering directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/netsim'

make[1]: Leaving directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/netsim'

mkdir -p ncsc-out

mkdir -p ../load-dir

/root/ncs/4.4.2/ncs-4.4.2//bin/ncsc --verbose --ncs-compile-bundle yang                        \

                 --ncs-device-dir ncsc-out   \

                 --ncs-device-type netconf            \

                  &&                          \

        cp ncsc-out/modules/fxs/*.fxs ../load-dir;

Internal compiler error

Makefile:25: recipe for target 'ncsc-out/.done' failed

make: *** [ncsc-out/.done] Error 1

make: Leaving directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/src'

Build failed. Error and warning messages below. See log for complete details

Internal compiler error

make: *** [ncsc-out/.done] Error 1

failure Build failed

[ok][2017-11-11 22:45:14]

[Jan Lindblad]

You'll see the details if you enable the pioneer logging

admin@ncs(config)# python-vm logging level level-debug

admin@ncs(config)# commit

The log file to have a look at is logs/ncs-python-vm-pioneer.log . If the failure is not during compilation (as here), but during loading, it may also be interesting to have a look at logs/ncs.log and logs/devel.log.

There are cases where it's not so easy to figure out which YANG is causing an issue. When that happens, I sometimes try to randomly exclude half of the YANGs and see if that helps. Then successively try to narrow down where the problem is by adding and removing YANGs to the set.

If you can't figure out what the problem is, don't hesitate to reach out to me.

[Iman1]

Thanks Jan. I enabled the Pioneer log as you said, but unfortunately my eyes don't see any extra information there. Nothing interesting in logs/ncs.log and logs/devel.log either. Am I missing something? Is there no further step to figure out why Pioneer complains about an internal compilation error?

==> logs/ncs-python-vm-pioneer.log <==

2017-11-13 16:56:43 - pioneer - DEBUG - ========== pioneer cb_action() ==========

2017-11-13 16:56:43 - pioneer - DEBUG - thandle=173 usid=54

2017-11-13 16:56:43 - pioneer - DEBUG - yang_build_netconf_ned() with device mr30

2017-11-13 16:56:43 - pioneer - DEBUG - Cleaning up existing ned-directory

2017-11-13 16:56:43 - pioneer - DEBUG - Starting build of 151 YANG modules, this may take some time

2017-11-13 16:56:43 - pioneer - DEBUG - run '/bin/bash /root/ncs/4.4.2/ncs-run-4.4.2/state/packages-in-use/1/pioneer/python/pioneer/ncs-make-package-verbose /win7/Nokia/15.0.R4_ALL_YANG/YANG/ 15r4 /root/ncs/4.4.2/ncs-4.4.2/packages/neds', input len=0

2017-11-13 16:56:43 - pioneer - DEBUG - run_outputfun '/bin/bash /root/ncs/4.4.2/ncs-run-4.4.2/state/packages-in-use/1/pioneer/python/pioneer/ncs-make-package-verbose /win7/Nokia/15.0.R4_ALL_YANG/YANG/ 15r4 /root/ncs/4.4.2/ncs-4.4.2/packages/neds

2017-11-13 16:56:43 - pioneer - DEBUG - run_outputfun, going in

2017-11-13 16:56:43 - pioneer - DEBUG - run_outputfun, output len=378

2017-11-13 16:56:43 - pioneer - DEBUG - PATH=/root/ncs/4.4.2/ncs-4.4.2//bin:/root/ncs/4.4.2/ncs-4.4.2/lib/ncs/erts/bin:/root/ncs/4.4.2/ncs-4.4.2/lib/ncs/bin:/root/ncs/4.4.2/ncs-4.4.2//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin:/root/go/bin ncs-make-package --verbose --netconf-ned /win7/Nokia/15.0.R4_ALL_YANG/YANG/ 15r4 --dest /root/ncs/4.4.2/ncs-4.4.2/packages/neds --no-java

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=57

2017-11-13 16:56:44 - pioneer - DEBUG - Wrote package to /root/ncs/4.4.2/ncs-4.4.2/packages/neds

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=113

2017-11-13 16:56:44 - pioneer - DEBUG - make clean fxs -C /root/ncs/4.4.2/ncs-4.4.2/packages/neds/src NCSC=/root/ncs/4.4.2/ncs-4.4.2//bin/ncsc --verbose

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=71

2017-11-13 16:56:44 - pioneer - DEBUG - make: Entering directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/src'

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=28

2017-11-13 16:56:44 - pioneer - DEBUG - rm -rf ncsc-out ../load-dir

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=35

2017-11-13 16:56:44 - pioneer - DEBUG - cd ../netsim && make clean || true

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=77

2017-11-13 16:56:44 - pioneer - DEBUG - make[1]: Entering directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/netsim'

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=76

2017-11-13 16:56:44 - pioneer - DEBUG - make[1]: Leaving directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/netsim'

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=18

2017-11-13 16:56:44 - pioneer - DEBUG - mkdir -p ncsc-out

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=21

2017-11-13 16:56:44 - pioneer - DEBUG - mkdir -p ../load-dir

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=299

2017-11-13 16:56:44 - pioneer - DEBUG - /root/ncs/4.4.2/ncs-4.4.2//bin/ncsc --verbose --ncs-compile-bundle yang                        \

                --ncs-device-dir ncsc-out  \

                --ncs-device-type netconf            \

                  &&                          \

        cp ncsc-out/modules/fxs/*.fxs ../load-dir;

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=24

2017-11-13 16:56:44 - pioneer - DEBUG - Internal compiler error

2017-11-13 16:56:44 - pioneer - DEBUG - run_outputfun, output len=160

2017-11-13 16:56:44 - pioneer - DEBUG - Makefile:25: recipe for target 'ncsc-out/.done' failed

make: *** [ncsc-out/.done] Error 1

make: Leaving directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/src'

2017-11-13 16:56:44 - pioneer - DEBUG - run_finished, output len=1357

2017-11-13 16:56:44 - pioneer - DEBUG - Output from build

PATH=/root/ncs/4.4.2/ncs-4.4.2//bin:/root/ncs/4.4.2/ncs-4.4.2/lib/ncs/erts/bin:/root/ncs/4.4.2/ncs-4.4.2/lib/ncs/bin:/root/ncs/4.4.2/ncs-4.4.2//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin:/root/go/bin ncs-make-package --verbose --netconf-ned /win7/Nokia/15.0.R4_ALL_YANG/YANG/ 15r4 --dest /root/ncs/4.4.2/ncs-4.4.2/packages/neds --no-java

Wrote package to /root/ncs/4.4.2/ncs-4.4.2/packages/neds

make clean fxs -C /root/ncs/4.4.2/ncs-4.4.2/packages/neds/src NCSC=/root/ncs/4.4.2/ncs-4.4.2//bin/ncsc --verbose

make: Entering directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/src'

rm -rf ncsc-out ../load-dir

cd ../netsim && make clean || true

make[1]: Entering directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/netsim'

make[1]: Leaving directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/netsim'

mkdir -p ncsc-out

mkdir -p ../load-dir

/root/ncs/4.4.2/ncs-4.4.2//bin/ncsc --verbose --ncs-compile-bundle yang                        \

                --ncs-device-dir ncsc-out  \

                --ncs-device-type netconf            \

                  &&                          \

        cp ncsc-out/modules/fxs/*.fxs ../load-dir;

Internal compiler error

Makefile:25: recipe for target 'ncsc-out/.done' failed

make: *** [ncsc-out/.done] Error 1

make: Leaving directory '/root/ncs/4.4.2/ncs-4.4.2/packages/neds/src'

2017-11-13 16:56:44 - pioneer - DEBUG - Build failed. Error and warning messages below. See log for complete details

Internal compiler error

make: *** [ncsc-out/.done] Error 1

[Jan Lindblad · Accepted solution]

As I wrote, sometimes there isn't much to go by. If you could send me the YANGs, I'd have a closer look.

[rogaglia]

Iman,

In our previous tests against Nokia NETCONF inteface, they are having a basic non-compliance with the standard that is preventing them to work with NSO and other systems. Basically, Nokia allows a single lock and not one per datastore. They promised to solve this issue (I believe maybe later this year) but just be sure that you are testing their latest release in case they did.

Please see some ODL mailing list discussions: https://lists.opendaylight.org/pipermail/netconf-dev/2016-August/000560.html

Regards,

Roque

[Iman1]

Thanks Roque.

Yes, I'm aware of that particular issue with Nokia's netconf implementation. The way I have worked around that issue is by disabling running DS capability on Nokia device, and that way NSO only tries to hold a lock on candidate DS. I think we'll have to wait for SROS 16 at minimum for a permanent fix to this issue.

The problem I have in this thread though seems to be YANG related, and I'm wondering how to go about troubleshooting it.

Iman

[Iman1]

Jan, sorry for delay... I was away and then busy for a while. Attached are the YANGs as you requested.

[Jan Lindblad]

I had a look and found the reason for the internal compiler error: a couple of modules imported themselves(!). Once that was fixed, there were other errors cropping up. All in all I fixed two dozen issues, look for JLINDBLA in the YANGs if you want to see what I did. 60 of the modules in the package have zero size; I guess the download failed somehow. After removing them and with the fixes above, the module set compiles and loads on NSO 4.5.1.