11-24-2023 12:40 AM - edited 11-24-2023 03:42 AM
Hello all,
I am working on self-made NSO NETCONF notifications in LSA environment (RFS node - CFS node).
There are several difficulties I faced so far. For the context I am using Python, because my company wants to use python.
Python sender function:
def send(self):
notification_socket = socket.socket()
try:
# init daemon
ctx = _ncs.dp.init_daemon(f"netconf-notification-{self.qual_name}")
# making a socket
_ncs.dp.connect(
dx=ctx,
sock=notification_socket,
type=_ncs.dp.CONTROL_SOCKET,
ip="127.0.0.1",
port=_ncs.PORT,
)
# register the stream
livectx = _ncs.dp.register_notification_stream(
ctx,
None,
notification_socket,
self._stream,
)
# time
now = datetime.now(timezone.utc) # timezone must be set correctly!!!
time = _ncs.DateTime(
now.year,
now.month,
now.day,
now.hour,
now.minute,
now.second,
now.microsecond,
int(now.strftime("%z")[:3]),
int(now.strftime("%z")[3:]),
)
self.log.debug(f"Generating notification message...")
message = self._create_xml()
self.log.debug(f"Notification message is set to={message}")
self.log.debug(f"Sending notification with={livectx=}, {time=}, {message=}...")
# sending the notification
_ncs.dp.notification_send(
livectx,
time,
message,
)
self.log.debug(f"Message sent...")
except Exception as e:
self.log.info(e)
finally:
notification_socket.close()
Example of underlying YANG on RFS node:
module device-status-notifications {
yang-version 1.1;
namespace "http://example.com/device-status-notifications";
prefix status-notifi;
import tailf-common {
prefix tailf;
}
notification reachability-status {
leaf customer-name {
type string;
}
leaf site-id {
type string;
}
leaf device-id {
type string;
}
leaf device-uuid {
type string;
}
leaf host-name {
type string;
}
leaf status {
type string;
}
}
}
As "dirty-fix" I tought about listening on stream with an kicker on CFS node and re-sending the same notification from CFS (on the same stream). I don't really like this solution because I have to develop 2 YANG models (for RFC and CFS) with notification in it and also implement 2 senders in python (on RFS and CFS).
Do you have different ideas / approaches to solve this?
P.S. I also realised that python has the worst API support of the main 3 APIs (C, Java, Python)
This post should be seen as feature request / discussion thread.
Solved! Go to Solution.
11-30-2023 06:18 AM - edited 11-30-2023 06:38 AM
Hi,
There is no built-in support for forwarding notifications other than the device-notifications stream. So I would suggest something like attached image:
Note: Only the RFS will have the "reachability-stream" defined in its ncs.conf file
If you need the CFS node to forward and expose a reachability stream (not the device-notifications stream) to the Management-System, you need an application in a package on the CFS that does the forwarding.
Also, note that the Management-System can create a RESTCONF subscription for the CFS device-notifications stream if REST is preferred over NETCONF.
11-26-2023 09:21 AM
I believe that you can set /cluster/device-notifications/enabled to forward device notifications.
From the $NCS_DIR/src/ncs/yang//tailf-ncs-cluster.yang notifications container description:
"NETCONF notification streams used by NCS nodes in a clustered environment. A top NCS node will automatically subscribe to these streams for all remote NCS nodes.The 'ncs-events' stream is used by the commit queue. The 'device-notifications' stream is used to forward NETCONF notifications originating from devices managed by remote NCS nodes.";
Note that you can use the high-level API to handle sockets for notification sending. Example:
import ncs
import _ncs
d = ncs.dp.Daemon("change-notifier", port=port)
nctx = _ncs.dp.register_notification_stream(d.ctx(), None,
ncs.dp.take_worker_socket(d,
'notif',
'notif-key'),
'some_state')
d.start()
...
_tm.dp.notification_send(nctx, time, message)
11-28-2023 12:49 AM
Hello,
thank you for your answer. Device-Notifications Stream can be successfully forwarded (Notifi -> RFS -> CFS -> Management Systems) but custom streams like "my-stream" are not forwarded this way.
If RFS sends an notification on "my-stream" CFS-Node gets it successfully but management systems that subscribe to CFS "my-stream" do not recive the notification (RFS --Notifi--> CFS --X--> Management Systems).
Is there any way to forward "my-stream" like "device-notifications" stream?
11-28-2023 11:34 AM - edited 11-28-2023 11:46 AM
Hello,
It works, but what you probably missed, since it is undocumented and no examples show it, is that you need to load the device NED package(s) with the CFS NSO node, too (not just the RFS nodes). The NED should not be configured on the CFS node. It should just be loaded so the CFS NSO has the FXS from the YANG model with the notification loaded. Otherwise, you will get a "connecting" status for the device-notifications stream:
admin@upper-nso# show cluster
RECEIVED
NAME NAME STATUS LAST EVENT NOTIFICATIONS
------------------------------------------------------------------------------------------------
lower-nso-1 device-notifications connecting 0000-01-01T00:00:00-00:00 0
ncs-events up 2023-11-28T18:52:50.312666+00:00 9
lower-nso-2 device-notifications connecting 0000-01-01T00:00:00-00:00 0
ncs-events up 2023-11-28T18:52:50.310534+00:00 9
... and a CFS node error log similar to this:
$ ncs --printlog upper-nso/logs/ncserr.log.1
28-Nov-2023::19:52:50.271 <0.433.0> <0.1790.0> ncs_cluster_nc_notifs:1246: cluster: add error code for: {ncs_cluster_nc_notifs,
{unknown_notification,
"http://example.com/device-status-notifications",
"reachability-status"}} (<<"lower-nso-1">> <<"admin">>)
[{ncs_cluster_nc_notifs,mk_session_error,2,
[{file,"ncs_cluster_nc_notifs.erl"},{line,1247}]},
{ncs_cluster_nc_notifs,subscriber_loop,2,
[{file,"ncs_cluster_nc_notifs.erl"},{line,711}]},
{ncs_cluster_nc_notifs,subscriber,3,
[{file,"ncs_cluster_nc_notifs.erl"},{line,658}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]
28-Nov-2023::19:52:50.274 <0.433.0> <0.1783.0> ncs_cluster_nc_notifs:1246: cluster: add error code for: {ncs_cluster_nc_notifs,
{unknown_notification,
"http://example.com/device-status-notifications",
"reachability-status"}} (<<"lower-nso-2">> <<"admin">>)
[{ncs_cluster_nc_notifs,mk_session_error,2,
[{file,"ncs_cluster_nc_notifs.erl"},{line,1247}]},
{ncs_cluster_nc_notifs,subscriber_loop,2,
[{file,"ncs_cluster_nc_notifs.erl"},{line,711}]},
{ncs_cluster_nc_notifs,subscriber,3,
[{file,"ncs_cluster_nc_notifs.erl"},{line,658}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]
I will add a ticket to create some documentation and an example that shows how to set it up to avoid wasting more time for users. Also, the error should be handled better so the user is notified of the issue somehow.
11-29-2023 07:25 AM - edited 11-29-2023 07:26 AM
Hello,
thank you for your answer. I have created new environment and loaded NED on CFS as you described in your post.
Now I have the issue that everything is up but CFS gets no notifications (without configuring subscriber under "devices/device/netconf-notifications/subscription"). With an exclusive subscription CFS recives the notifications but doesn't re-send them on its norhboud -> the tool "netconf-console2" didn't recive any notification from CFS on "vedge-reachability-status" / "my-stream" stream.
cloud@ncs> show cluster
RECEIVED
NAME NAME STATUS LAST EVENT NOTIFICATIONS
-----------------------------------------------------------------------------------
rfs_sdwan device-notifications up 0000-01-01T00:00:00-00:00 0
ncs-events up 0000-01-01T00:00:00-00:00 0
I don't see the custom stream
There are also no errors in ncserr.log.1
cloud@ncs> show devices device rfs_sdwan notifications
REPLAY
LOG
REPLAY AGED
NAME DESCRIPTION SUPPORT REPLAY LOG CREATION TIME TIME
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
NETCONF default NETCONF event stream false - -
ncs-alarms NCS alarms according to tailf-ncs-alarms.yang false - -
ncs-events NCS event according to tailf-ncs-devices.yang true 2023-11-29T12:47:35.900918+00:00 -
kicker-events NCS event according to tailf-kicker.yang true 2023-11-29T12:47:35.911359+00:00 -
device-notifications NCS events forwarded from devices true 2023-11-29T12:47:35.915956+00:00 -
service-state-changes Service state changes according to
tailf-ncs-plan.yang and tailf-ncs-services.yang
true 2023-11-29T12:47:35.920803+00:00 -
vedge-reachability-status NETCONF notifications about vedge reachability status generated by rfs-node false - -
NAME URI
---------------------------------------------------------
test-notification http://example.com/rfs-notifications
cloud@ncs> show packages package oper-status
PACKAGE
PROGRAM META FILE
CODE JAVA PYTHON BAD NCS PACKAGE PACKAGE CIRCULAR DATA LOAD ERROR
NAME UP ERROR UNINITIALIZED UNINITIALIZED VERSION NAME VERSION DEPENDENCY ERROR ERROR INFO WARNINGS
-------------------------------------------------------------------------------------------------------------------------------------------
bgworker X - - - - - - - - - - -
observability-exporter X - - - - - - - - - - -
rfs-notifications-ned X - - - - - - - - - - -
tailf-hcc X - - - - - - - - - - -
Is there anything I am missing at this point?
11-29-2023 02:18 PM - edited 11-29-2023 10:39 PM
<notifications>
<event-streams>
...
<stream>
<name>device-notifications</name>
<description>NSO events forwarded from devices</description>
<replay-support>true</replay-support>
<builtin-replay-store>
<enabled>true</enabled>
<dir>./state</dir>
<max-size>S10M</max-size>
<max-files>50</max-files>
</builtin-replay-store>
</stream>
...
</event-streams>
</notifications>
ncs_cli --port $MY_RFS_NC_PORT -u admin -C << EOF
config
devices device my-dev0 netconf-notifications subscription some-name stream my-stream-name local-user my-user
devices device my-dev1 netconf-notifications subscription some-name stream my-stream-name local-user my-user
commit
EOF
ncs_cli --port $MY_CFS_IPC_PORT -n -u admin -C << EOF
config
cluster commit-queue enabled
cluster device-notifications enabled
commit
EOF
netconf-console --port=$MY_RFS_NC_PORT --create-subscription=device-notifications
netconf-console --port=$MY_CFS_NC_PORT --create-subscription=device-notifications
11-29-2023 03:51 PM - edited 11-29-2023 03:53 PM
Hi,
so I get it, ok that works.
The final Question: Is it possible to have multiple streams in cluster configuration like below?
The main point of this is: I / we don't want to use "device-notifications" stream, instead we would like to listen on an exclusive stream for that specific notification.
cloud@ncs> show cluster
RECEIVED
NAME NAME STATUS LAST EVENT NOTIFICATIONS
-----------------------------------------------------------------------------------
rfs_sdwan device-notifications up 0000-01-01T00:00:00-00:00 0
ncs-events up 0000-01-01T00:00:00-00:00 0
vedge-reachability up 0000-01-01T00:00:00-00:00 0
vmanage-status up 0000-01-01T00:00:00-00:00 0
management-info up 0000-01-01T00:00:00-00:00 0
I attach an image with final architecture I / we want to have. Maybe this would help to understand my case better
Btw: both CFS and RFS will have "reachability-stream" defined in their ncs.conf file.What I want to do
11-30-2023 06:18 AM - edited 11-30-2023 06:38 AM
Hi,
There is no built-in support for forwarding notifications other than the device-notifications stream. So I would suggest something like attached image:
Note: Only the RFS will have the "reachability-stream" defined in its ncs.conf file
If you need the CFS node to forward and expose a reachability stream (not the device-notifications stream) to the Management-System, you need an application in a package on the CFS that does the forwarding.
Also, note that the Management-System can create a RESTCONF subscription for the CFS device-notifications stream if REST is preferred over NETCONF.
11-30-2023 02:43 PM
Hi,
thank you for all your help and explanations.
Would be nice if NSO would support custom stream forwarding in future
Again thank you very much.
Jakub H.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide