cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
Walkthrough Wednesdays

Regenerating Call Manager, TVS, IPSEC and Other certificates

5096
Views
45
Helpful
0
Comments

 Hi Experts,

 

 

I am writing this blog for the work around we need to follow while regenerating the Call Manager certificates especially when you have to regenerate TVS and Call manager certificates at the same time.

 

This action plan is suggested based on the observation that the Call Manager certificates

are expired on all nodes and cluster is in non-secure node.

 

Before Performing the regeneration activity do the following pre-requisites.

 

  1. Take a backup of the whole cluster
  2. Make sure Phones are having updated ITL.

You can verify the above by running show itl command on TFTP server and then matching the ITL hash with one present in the Phone.

If they are not matching then Phone reboot might be required.

 

There is no such order to be followed to regenerate the Certificates. You can regenerate any of the certificates first namely Call manager, TVS, CAPF, IPSEC. However, the only thing to make sure during the certificate regeneration is to avoid regenerating TVS and Call Manager Certificates together. Please read the document below to understand the process.

 

 

Steps for re-generating certificates.

 

 

  1. Set pre8.0 rollback parameter in Enterprise Parameters  to True  and restart TFTP service on all TFTP nodes.

This will create an blank ITL file.

 

  1. Reset all the Phones to make sure they get blank ITL . Ideally in step 1 only when you will change the Enterprise parameters , Phones will reset but resetting them once will make sure they get blank itl.

Ensure all phones register back so they have the empty ITL.

    

 

  1. Regenerate TVS across all nodes

                Restart Trust Verification Service on all nodes(In some cases it restarts automatically)

 

Here is a catch "Please restart all Devices" either from enterprise parameters or select all devices and then restart. This is very helpful because when you restart Call Manager service phones will use TVS to register.

 

  1. Regenerate Call Manager certificate on all nodes

Restart Call Manager and TFTP service on all nodes

 

"Please restart all Devices again"

 

 

  1. Verify updated ITL on all TFTP nodes by using 'show ITL' via CLI ensuring the new certificates are listed (Call Manager & TVS)

 

  1. Remove rollback parameter and ensure all phones register. (set to false)

                  (Phones will obtain a valid ITL)

-----------------------------------------------------------------

 

Please note if the step 1 and 2 didn't work as mentioned don't worry at all. We have one more option left. 

Please follow the below steps.

 

1 Regenerate the TVS certificate

2 Restart all the phones (This restart is very crucial, don't think we will regenerate both certificates then restart the devices only one time. It can cause a huge problem and you may have to reset all the devices manually).

3 Regenerate the call manager certificate

4 Restart the phones again

 

-----------------------------------------------------------------

 

 

 

Remaining certificates won’t have any impact  on ITL and you can regenerate them in any order. Perform the below steps to regenerate them.

 

  1. Regenerate Tomcat across all nodes

Restart Tomcat service on all nodes

(Impacts corporate directory access and extension mobility)

 

  1. Regenerate CAPF on all nodes

(No impact as CAPF service is not running. Recommended so alerts aren't giving via RTMT)

 

  1. Regenerate IPSEC ( Applicable for CUCM/CUC/IMP...)

Note: CUCM/Instant Messaging and Presence (IM&P) before version10.X the DRF Master Agent runs on both CUCM Publisher and IM&P Publisher.  DRF Local service runs on the subscribers respectively.  Versions 10.X and higher, DRF Master Agent runs on the CUCM Publisher only and DRF Local service will be on CUCM Subscribers and IM&P Publisher and Subscribers.

Note: The Disaster Recovery System uses an Secure Socket Layer (SSL) based communication between the Master Agent and the Local Agent for authentication and encryption of data between the CUCM cluster nodes. DRS makes use of the IPSec certificates for its Public/Private Key encryption. Be aware that if you delete the IPSEC truststore (hostname.pem) file from the Certificate Management page, then DRS will not work as expected. If you delete the IPSEC-trust file manually, then you must ensure that you upload the IPSEC certificate to the IPSEC truststore. For more details, refer to the certificate management help page in the Cisco Unified Communications Manager Security Guides.

 

  1. Navigate to each server in your cluster (in separate tabs of your web browser) begin with the publisher, followed by each subscriber.  Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Select the IPSEC pem Certificate.
    • Once open select Regenerate and wait until you see the Success pop-up then close pop-up or go back and select Find/List
  2. Continue with subsequent Subscribers; follow the same procedure in step 1 and complete on all subscribers in your cluster
  3. After all Nodes have regenerated the IPSEC certificate then restart services.
    • Navigate to the Publisher's Cisco Unified Serviceability
      1. Cisco Unified Serviceability > Tools > Control Center - Network Services
      2. Select Restart on Cisco DRF Master service
      3. Once the service restart completes, select Restart on the Cisco DRF Local Service on the publisher then continue with the subscribers and select Restart on the Cisco DRF Local Service

The IPSEC.pem certificate in the publisher must be valid and must be present in all subscribers as IPSEC truststores. The subscribers IPSEC.pem certificate will not be present in the publisher as IPSEC truststore in a standard deployment. In order to verify the validity compare the serial numbers in the IPSEC.pem certificate  from the PUB with the IPSEC -trust in the SUBs. They MUST match.

 

What to do incase the Serial Number is not matching?

 1. Go to Publisher download ipsec.pem and rename it with ipsec-trust.

 2. Go back to Certificate Management Page on all the nodes(Publisher and all Subcribers) and find the existing ipsec-trust file and Delete it.

 3. Upload the file you downloaded(ipsec.pem renamed with ipsec-trust) in step 1. in Publisher and all subscribers.

 4. Restart the services in the following order:

    a. Restart DRS Master on Publisher

    b. Restart DRS Local on Publisher

    c. Restart DRS Local on all Subscribers.

 

(Impacts backups and ipsec tunnels)

 

There will be some certificates ending with –trust  and you won't get any option to regenerate them: you can go ahead and Delete them without any issues. 

 

If you delete -trust (expired certificates) normally we don’t need to stop any services but with 10.5 CUCM, we need to stop Certificate Change Notification service due to a known bug that deleted certificates keep coming back even after deleting them. Please start Certificate Change Notification service again after you delete the unnecessary certificates. 

 

Summary of the document:

1. Take backup of whole cluster.

2. Set pre8.0 rollback parameter in Enterprise Parameters  to True

3. Restart TFTP services on all the TFTP nodes.

4. Reset all the Phones.

5. Regenerate the certificates one by one in any order.

6. Don't Regenerate Call Manager and TVS Certificates without restarting all the devices between them. 

    Example: Let's assume you are regenerating Call manager certificate first and then TVS.

    So you will regenerate Call Manager Certificate first.

    restart all the devices.

    Then Regenerate TVS and restart all the devices again.

 

Please rate "Helpful", if this helps you.

Content for Community-Ad