Cisco Prime infrastructure 3.7 HA issue || unable to remove HA settings in Secondary Server through GUI & CLI

[BaVir160195]

Hello everyone, We have a cisco prime infrastructure 3.7 with HA, virtually hosted in our environment.

Last week our Primary-Prime-server got crashed and currently Secondary-prime-server is up and running as active one. Yesterday we try to rebuild the Prime-prime-server including recommended patches and successfully restored with Secondary-prime-server backup (since we don't have Primary server backup). 

But while we try to reconfigure HA, In Primary-Prime-server we got the error "secondary Server should be in same patch level, kindly upgrade it first". On the other hand, in Secondary-prime-server if we try patch, it is throwing "remove the Ha configuration first and retry". We try to remove HA in GUI, but is it in grade-out(unable to click). In cli, 'NCS HA remove' command is not working.

We try to uninstall the patch in Primary-Prime-server, which we are not able to perform in cli. Please suggest either *how we can uninstall the patch in Primary-Prime-server or **how we can remove HA settings in Secondary-prime-server.

[Sergiu.Daniluk]

Hi Bavir,

It looks like you are in a chicken-and-egg kind of situation, where your secondary is missing the HA peer, while primary is not seeing the same patch image level on the secondary.

In the config guide, it mentions the following:

"""

Note that, to use this method, you must ensure that the primary Prime Infrastructure server is currently in the “Primary Active” state. If for any reason the secondary server is currently active, perform a failback and then try to remove the HA configuration after the failback is complete and the secondary’s automatic restart has finished.

"""

I think you will need to do a fresh install of the secondary.

Hope it helps,

Sergiu

[BaVir160195]

Hello Sergiu(@msdaniluk), Appreciate your quick response.

Is there is any possibilities to edit the HA config in secondary-Prime-Server and so that we can remove the entries manually.?  below is the path in CLI.

haSettings.xml >> file have the HA config

/opt/CSCOlumos/conf/rfm/classes/com/cisco/common/ha/config/

i gone through few of the post in Cisco forum and found the path for the HA-config in CLI. above CLI path is yet verify!.https://community.cisco.com/t5/network-management/cisco-prime-ha-secondary/td-p/2474214

Looking for your further suggestion and thanks in advance!.

[marce1000]

  = Why don't you rebuild the primary-prime at exact the same version as the HA=partner, then try to join the HA  again, and consider the patching as 'another project' afterwards ?

 M.

[BaVir160195]

Hello M(marce1000), Thanks for your suggestion.

we have a very limited downtime. so we can't rebuild the server again. Even, i though of rebuilding the Primary-Prime-Server, but it is not feasible right now. we are looking for the alternate solutions.

Appreciate, if you can advice on the either uninstalling the patch on Primary server or removing the HA config entries through CLI in Secondary Server.

[fobaker · Accepted solution]

1. Stop Services on the Secondary
2. From the shell, execute /opt/CSCOlumos/bin/cleanSecondary.sh
   1. This will delete the database on the Seconday
   2. But it will remain a Secondary HA server
   3. Services will start back up as a part of the script ~20mins
3. Once services are backup up on the Secondary:
   1. Navigate to https://<primeSecondary>:8082
   2. This will display the Health Monitor Page
4. Click on Software Update on the upper right side of the Health Monitor Page
5. Update the Software on the Secondary server to match the Primary
   1. EACH update will cause services to stop and start again ~20min per update
6. Now go to the Primary server:
   1. Administration > Settings > High Availability > HA Configuration
   2. Enter the information for the Secondary server
   3. Click on "Check Readiness"
   4. Resolve any issues that are reported
   5. Click "Register"
   6. Wait while HA is setup and the DB is copied from Primary to Secondary (hours)
7. Celebrate!

[BaVir160195]

Excellent!!!. this works. thanks a lot Forest@fobaker

[Kevin Hartle]

Thank you for posting.  I had to run the /opt/CSCOlumos/bin/cleanSecondary.sh script with the sudo command for it to execute properly.

" sudo /opt/CSCOlumos/bin/cleanSecondary.sh "