07-30-2021 01:13 AM
Hi,
I need some help from some cisco folk around about a possible issue im seeing on NCS5508
RP0 is apparently the active RP as can be seen in the below and things mostly work.
Except that sometimes the box claims RP0 is standby (but its lying)
(eg when i try to attach to a fabric card - granted you dont need this every day, but if needed its not possible to execute)
outputs below show better the situation im facing
box is running 7.2.2, issue was also present on 7.1
other 5508 box with identical hardware and software does not have this issue, so something wrong is happening here,
would appreciate if anyone can give any insight on this
tac case open but no joy yet
thanks in advance for you help
Mark
RP/0/RP0/CPU0:NCS5508-1#show redundancy Thu Jul 22 08:05:15.241 CEST Redundancy information for node 0/RP0/CPU0: ========================================== Node 0/RP0/CPU0 is in ACTIVE role Partner node (0/RP1/CPU0) is in STANDBY role Standby node in 0/RP1/CPU0 is ready Standby node in 0/RP1/CPU0 is NSR-ready RP/0/RP0/CPU0:NCS5508-1#show platform Thu Jul 22 08:05:31.766 CEST Node Type State Config state -------------------------------------------------------------------------------- 0/RP0/CPU0 NC55-RP-E(Active) IOS XR RUN NSHUT 0/RP1/CPU0 NC55-RP-E(Standby) IOS XR RUN NSHUT RP/0/RP0/CPU0:NCS5508-1#admin show controller card-mgr inventory summary Thu Jul 22 08:05:57.344 CEST Card Manager Inventory Summary : BP HW Location Card Type ID Serial Number Ver Card State ------------------------------------------------------------------------------ 0/RP0 NC55-RP-E (Slave) 27 JAE245000A0 1.1 CARD_READY <<<<<======== RP0 is the real Active ! 0/RP1 NC55-RP-E (Master) 28 JAE245000CF 1.1 CARD_READY [sysadmin-vm:0_RP0:~]$.attach_console 25 SLAVE-RP: Please run Remote console connection from Master RP <<<<<====== Same here. I am connected to the active RP, otherwise I wouldnt even have CLI access !!
07-30-2021 05:52 AM
Hi Mark,
there are two containers, the admin and XR vm.
the admin container can run on either of the RP's
i have attached output from our lab.
admin container is active on RP1, xr container active on RP0
you can type "admin" hit enter to go to the container and check if the prompt changes to 0_RP1
sysadmin-vm:0_RP1# show plat | inc RP Fri Jul 30 12:48:58.358 UTC+00:00 0/RP0 NC55-RP OPERATIONAL OPERATIONAL NSHUT 0/RP1 NC55-RP-E OPERATIONAL OPERATIONAL NSHUT sysadmin-vm:0_RP1# exit Fri Jul 30 12:49:02.330 UTC+00:00 show pRP/0/RP0/CPU0:NCS-5508-B#show plat | inc RP Fri Jul 30 04:49:07.353 GMT+8 0/RP0/CPU0 NC55-RP(Active) IOS XR RUN NSHUT 0/RP1/CPU0 NC55-RP-E(Standby) IOS XR RUN NSHUT RP/0/RP0/CPU0:NCS-5508-B#
07-30-2021 06:02 AM
Hi,
thanks for your reply
actually, if I enter admin, the prompt remains as RP0
RP/0/RP0/CPU0:NCS5508-1#admin Fri Jul 30 14:55:17.242 CEST Last login: Thu Jul 29 14:07:50 2021 from 192.0.108.4 admin connected from 192.0.108.4 using ssh on sysadmin-vm:0_RP0 sysadmin-vm:0_RP0#
even if then it shows up as RP1 in
admin show controller card-mgr inventory summary
and attempting to attach to the fabric is not possible as it 'complains' im on the slave RP (which is not the case)
[sysadmin-vm:0_RP0:~]$.attach_console 25 SLAVE-RP: Please run Remote console connection from Master RP
So the questions are:
- I understand from you it appears to be normal that the sysadmin plane may be running on RP1 and XR plane on RP0. Thats good to know I wasnt aware.
- However, I dont think this is what we are seeing here right ? Both are apparently running on RP0, and I cant seem to attach. I dont think this is expected behaviour ?
thanks
Mark
08-09-2021 10:30 AM
There are three separate, unrelated things:
For the XR VM on each RP, which is active.
For the admin VM on each RP which has the active confd_helper role and which is the slave. This is the RP that shows up in the command prompt when you type "admin" and you can see more information in the output of "show process confd_helper location all":
sysadmin-vm:0_RP0# show processes confd_helper location all
Mon Aug 9 18:08:27.100 UTC+01:00
----------------------------------------------------------------------
node: 0/RP0
----------------------------------------------------------------------
PID: 3368
Executable path: /opt/cisco/calvados/packages/ncs5500-sysadmin-mgbl-7.1.2-r712/rp/bin/confd_helper
Instance #: 0
Args: -t token -d -w 400 -b 30 -p 600 -r 10 -f 10
Respawn: ON
Respawn count: 1
Max. spawns per 4 mins: 4
Last started: 07/14/2021 00:29:10.000
Process state: Run
startup_path: /opt/cisco/calvados/packages/ncs5500-sysadmin-mgbl-7.1.2-r712/rp/etc/startup/confd_helper.startup
Ready: 0s
Table of services hosted on this process:
----------------------------------------------------------------------
LAST STARTED SCOPE SELE- ROLE STATE HA- SERVICE NAME
CTED RDY
----------------------------------------------------------------------
07/14/2021 00:30:02.000 SYS Y ACT Run confd_helper
----------------------------------------------------------------------
----------------------------------------------------------------------
node: 0/RP1
----------------------------------------------------------------------
PID: 3479
Executable path: /opt/cisco/calvados/packages/ncs5500-sysadmin-mgbl-7.1.2-r712/rp/bin/confd_helper
Instance #: 0
Args: -t token -d -w 400 -b 30 -p 600 -r 10 -f 10
Respawn: ON
Respawn count: 1
Max. spawns per 4 mins: 4
Last started: 07/13/2021 23:29:08.000
Process state: Run
startup_path: /opt/cisco/calvados/packages/ncs5500-sysadmin-mgbl-7.1.2-r712/rp/etc/startup/confd_helper.startup
Ready: 0s
Table of services hosted on this process:
----------------------------------------------------------------------
LAST STARTED SCOPE SELE- ROLE STATE HA- SERVICE NAME
CTED RDY
----------------------------------------------------------------------
07/13/2021 23:30:02.000 SYS Y STBY Run Y confd_helper
----------------------------------------------------------------------
If from admin you kill the active confd_helper process you will get kicked out back to XR and if you issue the admin command again you will find you are on the other RP and its confd_helper process is now active.
Lastly there is which RP has mastership for the card_mgr process and that's in the output of "show controller card-mgr inventory" as you pointed out earlier. If you want to attach to a console, you need to do that from the master, as the output from the .attach_console or rconsole commands tells you.
E.g.:
sysadmin-vm:0_RP0# show controller card-mgr inventory summary | include RP
Mon Aug 9 18:19:24.460 UTC+01:00
0/RP0 NC55-RP-E (Slave) 27 JAExxxxxxxxxD 1.1 CARD_READY
0/RP1 NC55-RP-E (Master) 28 JAExxxxxxxxF 1.1 CARD_READY
sysadmin-vm:0_RP0# run /opt/cisco/calvados/sbin/rconsole -l 0/4
Mon Aug 9 18:20:01.120 UTC+01:00
Connecting to Card Manager Node Scope service
Getting Mastership Status from Card Manager
Slot 27 HW arbitration result = Slave
Execute Remote console utlity from peer RP. This RP does not does not have MASTERSHIP
sysadmin-vm:0_RP0# attach location 0/RP1
Mon Aug 9 18:20:12.380 UTC+01:00
Last login: Mon Aug 9 16:52:52 2021 from 192.0.108.1
[sysadmin-vm:0_RP1:~]$ /opt/cisco/calvados/sbin/rconsole -l 0/4
Connecting to Card Manager Node Scope service
Getting Mastership Status from Card Manager
Slot 28 HW arbitration result = Master
Writing to console register
=> Address 0xfc set to 0x80050000
Connecting .. Escape char is ^x [control+x]
host:0_LC4 login:
Which is master/slave active/standby is a matter of timing and not something I suggest you worry about.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide