Introduction
This document shows you how to reset a lost password or when you have locked yourself out due to a problematic AAA configuration.
Core Issue
Because IOS-XR is substantially different in the way config files are managed, the standard trick of conf-reg 0x2142 will not work for IOS-XR.
You can lock yourself out if you are configuring aaa authentication to tacacs with no local fall back, if the tacacs server is unavailable there
is no way for you to get in.
eg:
aaa authentication login default groupt tacacs
Also this procedure is good when you have forgotten the password to your super user in IOS-XR to manage your machine.
Resolution
The following step through guide can be tried, the details of each step are listed below with more explanation:
- •1) Fixing AAA configuration errors
- •a. On the standby RP/RSP from the CONSOLE port hit the ESC key and type ‘ksh’ without quotes and hit ENTER
- i. Login with a local username and password
- ii. If this fails get the standby RP/RSP into ROMMON
- iii. Bypass KSH authentication with AUX_AUTHEN_LEVEL=0 and boot
- iv. Try step 1a again or use the AUX port and go to step 1b
- •b. View and edit the configuration from KSH
- i. Save the configuration to harddisk with ‘nvgen -c -l 1 -t 1 -o 1 > harddisk:/backupconfig.txt’
- ii. Edit out the bad AAA statements with ‘nano –e /harddisk:/backupconfig.txt’
- •c. Try to roll back the configuration with ‘config_rollback –n 0x1’
- •d. Bypass AAA and enter exec mode with ‘/pkg/bin/exec –a’
- •e. Attempt to use show commands or change the configuration
- i. If this fails reload all RP/RSP ROMMON
- ii. On the standby card set IOX_CONFIG_FILE=/harddisk:/backupconfig.txt or use ‘boot <image> -a <bogus_config>’ and boot
- iii. Also follows step 2g if you saw issues in 1a
- iv. If nothing above worked then this is the only option
- •2) Fixing a lost local username/password
- •a. Get the standby RP/RSP into ROMMON
- i. Bypass KSH authentication with AUX_AUTHEN_LEVEL=0 and boot
- •b. View the admin configuration with ‘nvgen –b /admin/cfg’
- •c. Save the admin configuration to the harddisk and edit out any and all users if you need other portions of this file
- •d. Bypass AAA and enter exec mode with ‘/pkg/bin/exec –a’
- •e. Attempt to use show commands or change the configuration
- •f. If this fails reload all RP/RSP to ROMMON
- •g. Set confreg 0x142 or IOX_ADMIN_CONFIG_FILE=/harddisk:/backupconfig.txt on the standby card or ‘boot <image> -o <bogus_config>’ and boot
- i. Note that this does not ignore the exec configuration and will not help if the issue is AAA related
- •h. Enter a new username and password when prompted
- •3) Fixing both issues
- •a. If you do not know a local login or cannot use the KSH method to recover the configuration then both the IOX_CONFIG_FILE and IOX_ADMIN_CONFIG_FILE will need to be pointed towards non-existent files. Both the admin and exec configurations will be cleared by this method
- •4) Make sure to remove any ROMMON variables which were change
- 5) XR-VM Username/Password reset procesdure using Sysadmin VM
There are 2 steps to this process.
1) Override the BASE running configuration
When you configure the problematic AAA statement sample as above.
2) Override the admin configuration that stores local usernames and passwords
When you don't remember any of the local usernames/passwords you have defined locally.
Overriding the Base configuration in XR:
Step 1
In rommon set the following variable:
rommon> IOX_CONFIG_FILE=/harddisk:/no-config
the file no-config is just a non existent file, you can give any name here really.
Note: This ROMMON variable will persist and needs to be removed after password recovery. Check the 'clean up' section.
Step 2
And issue 'sync', this will make the change persistent in the rommon config vars.
rommon> sync
Step 3
Issue 'i' or 'reset' and when the rsp is booting up, it should ignore the config file, since there's no config file found on /harddisk: called no-config
rommon> reset
or
rommon> i
Overriding the ADMIN configuration in XR:
In Admin configuration we store all the local usernames and passwords.
Step 1
Similarly you can do the same thing for admin config:
IOX_ADMIN_CONFIG_FILE=/disk0:/none
You should get prompted for root user/pass and will have a blank config on the box.
You need to load your config and do your modification.
Note: This ROMMON variable will persist and needs to be removed after password recovery. Check the 'clean up' section.
Step 2 and 3
are the same as for the base xr config file.
Second Option
Another way of recoveryof the password is to enable the following again in rommon:
rommon> AUX_AUTHEN_LEVEL=0
Which will allow the aux port to drop to ksh upon the RSP bootup with no prompt for login.
At the prompt you can either type:
/pkg/bin/exec -a
Which will give you a router prompt: Or simply
# Config
Which drops you into EXEC config mode.
Example
# uname -a
QNX node0_RSP0_CPU0 6.4.0 2009/12/10-13:43:22PST asr9k ppcbe
# config
RP/0/RSP0/CPU0:RO-A(config)#exit
#
# /pkg/bin/exec -a
RP/0/RSP0/CPU0:RO-A#
RP/0/RSP0/CPU0:RO-A#
RP/0/RSP0/CPU0:RO-A#exit
#
Clean up
Make sure that after you're done with your changes, in case you made the rommon vars persistent, you may want to unset
the variables to get back to the normal files that are used.
rommon> unset IOX_ADMIN_CONFIG_FILE
rommon> unset IOX_CONFIG_FILE
rommon> sync
All set!
If you forget the cleanup, you might see these lines:
RP/0/RSP1/CPU0:Oct 28 07:18:37.141 : locald_DSC[301]: %SECURITY-LOCALD-3-LWA_ADD_FAIL : Failed to add the username admin to lightweight authentication password database: No such file or directory
Another way to clear the variable:
more nvram:/classic-rommon-var location 0/RSP1/CPU0
run iox_on 0/RSP1/CPU0 nvram_rommonvar IOX_CONFIG_FILE ""
Step 5
XR-VM Username/Password reset procedure using Sysadmin VM
Note: This is not a process to hack router but user need sysadmin username / password for the accessibility of box by bypass XR credentials.
Steps to perform this activity:
1. login to router : I was having console access to the box.
bgl-xdm-009:112> telnet 10.67.30.20 2037
Trying 10.67.30.20...
Connected to 10.67.30.20.
Escape character is '^]'.
User Access Verification
Password:
Password OK
2. Pass interrupts "ctrl + o" to toggle to sysadmin
sysadmin-vm:0_RP0#
*** IDLE TIMEOUT ***
System Admin Username:
3. enter sysadmin username and password
System Admin Username: xxxxx
Password:
xxxxx connected from 127.0.0.1 using console on sysadmin-vm:0_RP0
sysadmin-vm:0_RP0#
4. with this login you can access "sysadmin- VM prompt"
sysadmin-vm:0_RP0#
5. From Sysadmin VM to access XR - VM perform following action
i. list sdr ips
sysadmin-vm:0_RP0# show sdr
Tue Apr 3 05:12:47.110 UTC
SDR: default-sdr
Location IP Address Status Boot Count Time Started
-----------------------------------------------------------------------------
0/RP0/VM1 192.0.0.4 RUNNING 1 03/01/2019 02:10:28
0/RP0/VM2 192.0.0.6 RUNNING 1 03/01/2019 02:10:56
ii. ssh SDR VM1 address
[sysadmin-vm:0_RP0:~]$ssh 192.0.0.4
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
<>
Last login: Fri Mar 1 02:14:39 2019 from 192.0.0.4
iii. Now you are at XR-VM and to disable credential pass "ctrl + a" interrupt
[xr-vm_node0_RP0_CPU0:~]$exec -a
6. After disabling credential you can access XR-VM
RP/0/RP0/CPU0:customer2#
at this stage you can create/ delete / modify user credentials to access router at XR-VM directly.
P/0/RP0/CPU0:customer2#show running-config
Tue Apr 3 05:13:02.445 UTC
Building configuration...
!! IOS XR Configuration version = 6.3.3
!! Last configuration change at Sun Feb 17 23:31:51 2019 by ZTP
!
hostname customer2 >>>
username asd
group root-lr
group cisco-support
secret 5 $1$rdaY$qJt7aNcc8uFKqhP/rK11V1
!
Related Information
It has been seen that sometimes a system autonomously enters password recovery mode. This is identified with:
“enter root-system username”
This is due to a ddts known as CSCth03923
You end up providing what you think is a known username and password combination and it failes to get you in.
The solution is simple, just enter a fake username/password that you know for sure has not been configured yet and you're in!
Xander Thuijs - CCIE #6775
Sr Tech Lead ASR9000
