Automatic ISE Portal Certificate Renew with Letsencrypt - how to guide

[Arne Bier]

Free Certs Anyone?

Have you ever wanted to get a public certificate for your ISE Portal, without the hassle of involving a commercial CA?  If so, read on ..

This lengthy guide (maybe better as a Youtube video ... in future) will show you how I setup a certbot on a Linux host to manage the lifecycle of an ISE Guest Portal system certificate. The goal was to move away from hassling with manual and costly annual public certificate renewals. In addition, if (or when) the industry moves to even shorter certificate lifespans (90 or 45 days), you won't care, because the robot is doing the work for you.  Bring on the robots!!!

Disclaimer: it worked in my lab and there is a chance that if you follow this you might have different outcomes (using a different CA, or Linux or certbot version).  The main point of this guide is to show what's involved, and I am confident that you can also roll your own solution out of this recipe.

Before we get started - this process contains concepts that you can explore in more details (e.g. Letsencrypt, certbot, ACME protocol etc) - I won't explain it here. And there is also an assumption that you can read/hack a Linux script to make it work for your purposes.  The other pre-requisite is that you own or can manage a public DNS domain - either for your lab or your organisation. I registered a domain on Cloudflare for roughly $6(USD) per year which was perfect for my experimentation.

Letsencrypt

The goal is to automate the certificate renewal for an ISE Guest Portal using Letsencrypt's certbot. Certbot is an application that , implements the ACME protocol to automate certificate renewals. Other CAs (commercial) also support this. But Letsencrypt is free and by now the most popular CA in the world.

Feature request: ISE does not have any features that allow native integration with Letsencrypt - if it did, I would not be writing this how-to guide. But perhaps Cisco will consider this in future.

The Linux command is 'certbot' and once everything is installed/configured, it runs on its own and will renew the ISE Portal cert(s) every 60 days (as of now - Letsencrypt plans to introduce much shorter-lived certificates from April 2025 - watch this space). Let's get going!

Get access to a Linux host

I used a popular Linux distribution - Ubuntu Server 24.01 LTS but anything will do. This host must have internet access since certbot talks ACME protocol to Letsencrypt servers on the internet! Certbot can co-exist on an existing Linux server or just create another light-weight install that is dedicated to this one task. 1 vCPU, 1GB RAM, 20GB disk  is more than enough. If you are super smart, you might even containerize this.

Get a public DNS Domain

We need admin access to a public DNS domain provider to prove ownership of the DNS domain. In my example, I chose Cloudflare, and to keep my lab costs down, I chose the cheapest domain I could find at the time - 128bits.win (less than $10 a year to own the domain)

Admin portal at https://dash.cloudflare.com/

In the DNS Admin GUI, I created an API Token to allow editing of the 128bits.win domain - since certbot will use the API to create TXT records.

https://dash.cloudflare.com/profile/api-tokens

This API token will be stored on the Linux host that runs the certbot.

Certbot Installation

Once Ubuntu is installed, install certbot. I wanted the certbot application to run under the user 'letsencrypt' - therefore we create a separate user account called 'letsencrypt'

NB: I was unable to get certbot to execute/run automatically as this user - when it runs automatically, it always runs as user root. Certbot uses its own type of 'cronjob' to execute. Ideally would like to have the certbot automatically execute with a non-root user. I was unable to find a solution to this. In Ubuntu, I installed certbot as a 'snap' to get the latest version of certbot.  In this case, certbot is executed as part of the systemd timer. You can view this if you are curious:

systemctl edit snap.certbot.renew.timer

Anyway, I created the user called 'letsencrypt' to store things like the ISE script, and also to store the curl secret credentials.

I created this user as part of the Ubuntu Server Installation process - but it can also be done subsequently if needed

sudo apt update
sudo apt upgrade
sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot

I am using DNS Plugin for Cloudflare integrations

sudo snap set certbot trust-plugin-with-root=ok
sudo snap install certbot-dns-cloudflare

Login as the non-root user, and then create the directories for certbot, and the secrets:

mkdir ~/certbotconf
mkdir ~/certbotwork
mkdir ~/certbotlogs
mkdir ~/.secrets
nano ~/.secrets/.cloudflare.ini

In the .cloudflare.ini file, add the API key for cloudflare - e.g.:

dns_cloudflare_api_token = xyz12345

and save the file.

Create the CURL .netrc file that contain machine auth credentials with your favourite editor (like nano):

nano ~/.secrets/.netrc

..add the ISE FQDN and login creds to the .netrc - e.g.:

machine rnolabise01.rnlab.local login admin password $KeepMeSafe$
machine rnolabise02.rnlab.local login admin password $KeepMeSafe$

Make the files readable/writeable by the user only (of course root can also read these but …)

chmod 600  ~/.secrets/.netrc
chmod 600  ~/.secrets/.cloudflare.ini

Create the initial certbot skeleton files by running the following command:

certbot certonly --config-dir ~/certbotconf  --work-dir ~/certbotwork --logs-dir  ~/certbotlogs -v

Select option 1 (obtain cert via dns-cloudlfare)

Enter a valid email address

Pressy Y to agree to Terms of Service

Press N to not receive emails

Enter the domain for the cert - e.g. 128bits.win

Enter path to cloudflare .ini  -  /home/letsencrypt/.secrets/.cloudflare.ini

That should create the first cert! Congratulations.

Edit the certificate renewal file to change a default.

nano ~/certbotconf/renewal/128bits.win.conf

Add the lines shown below to the end of the file

#Added manually to make the solution a bit more tolerant
dns_cloudflare_propagation_seconds = 30

ISE Cert Import Script using OpenAPI

We now have a solution that updates the cert every 60 days and provides you with a private key and the certificate on that host. The next step is to get that private key and certificate installed in ISE. For this we need a regular shell script.

Create the directory and the script:

mkdir ~/ise_portal_auto_cert
nano ~/ise_portal_auto_cert/import_ise_port_cert.sh

The contents of the import_ise_port_cert.sh script. You should read through every line and edit any of the variables to suit your environment:

#!/bin/sh
#
# This script can be called to perform the ISE System Cert Import
# The recommended certbot hook is the deploy hook, but for testing you can also run it for the post hook.
# This script should be placed in the appropriate certbot directory - e.g. certbotconf/renewal-hooks/deploy
#
#########################################################
# User defined variables below                          #
# NB: Script is executed as root user - do not use the  #
# ~ (tilde) to reference relative path names !          #
# Rather refer to the absolute pathnames below          #
#########################################################
# Portal Cert domain name
CertDomainName=128bits.win
# Base Directory of this script
thisscriptbase=/home/letsencrypt/ise_portal_auto_cert
# The ISE FQDN used for OpenAPI calles
ISEGatewayFQDN=rnolabise01.rnlab.local
# The ISE Hostname as defined in ISE
ISEHostname=rnolabise01
# Filename containing the Root CA used by ISE Admin System Certificate
ISECAFile=RNLAB-ROOTCA.pem
# Letsencrypt base directories. Currently no environment variables exist. There is a feature request in github
certbotconf=/etc/letsencrypt
certbotlogs=/var/log/letsencrypt
curlsecrets=/home/letsencrypt/.secrets

# This script should run after a new Letsencrypt cert has been successfully renewed
# Letsencrypt container currently does not support bash scripts - hence why we are using /bin/sh
# Append an entry to the log file
echo Script ran at `date` >> $thisscriptbase/import_ise_port_cert.log

#Strip the special characters from the certificate file
certdata=$(awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' $certbotconf/live/$CertDomainName/cert.pem)

# ISE expects the private key to be password protected - Letsencrypt priv keys are not password protected and don't pose a risk.
# Keeping this password secure is not an issue - it's just temporary and used during the REST call
privkeypass=SomeSecr3t
#Strip the special characters from the private key file after performing password protection on it - required by ISE API
certkey=$(openssl pkcs8 -topk8 -in $certbotconf/live/$CertDomainName/privkey.pem -passout pass:$privkeypass | awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}')

#Generate a unique Friendly Name since ISE expects unique Friendly Names. Importing with same Friendly Name will cause an error
friendlyname="Portal Cert from $(date)"

# Take note of the double quotes in the curl request. This is used instead of single quotes, since single quotes
# preserve the string as a literal - this would prevent variable substitution. As a result, all literal double quotes (as required
# by the JSON syntax) must be explicitly escaped. This is tricky and looks a bit messy, but it's 100% correct and unavoidable
curl -s --cacert $thisscriptbase/$ISECAFile --netrc-file $curlsecrets/.netrc -X 'POST' \
  "https://$ISEGatewayFQDN:443/api/v1/certs/system-certificate/import" \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d "{
  \"admin\": false,
  \"allowExtendedValidity\": true,
  \"allowOutOfDateCert\": true,
  \"allowPortalTagTransferForSameSubject\": true,
  \"allowReplacementOfCertificates\": true,
  \"allowReplacementOfPortalGroupTag\": true,
  \"allowRoleTransferForSameSubject\": true,
  \"allowSHA1Certificates\": true,
  \"allowWildCardCertificates\": false,
  \"data\": \"$certdata\",
  \"eap\": false,
  \"ims\": false,
  \"name\": \"$friendlyname\",
  \"password\": \"$privkeypass\",
  \"portal\": true,
  \"portalGroupTag\": \"My Default Portal Certificate Group\",
  \"privateKeyData\": \"$certkey\",
  \"pxgrid\": false,
  \"radius\": false,
  \"saml\": false,
  \"validateCertificateExtensions\": false
}"

#####################
# Cleanup task.     #
#####################

# Attempt to delete any Portal Certificate remaining in "Not in use" status.
# In ISE 3.4 the API call does not support filtering by "usedBy" id - I had to  devise my own method.
# Fetch all the cert IDs that where the "Issued To" field contains your Portal domain and use the 'jq' tool to extract the IDs
# It will return a list, if there are multiple certs
currentcerts=`curl -s --cacert $thisscriptbase/$ISECAFile --netrc-file $curlsecrets/.netrc \
                -X 'GET' "https://$ISEGatewayFQDN:443/api/v1/certs/system-certificate/$ISEHostname?filter=issuedTo.CONTAINS.$CertDomainName" \
                -H 'accept: application/json' \
                -H 'Content-Type: application/json' | jq ".response.[] | select(.usedBy==\"Not in use\" and .issuedTo==\"$CertDomainName\")" | jq .id`

# Iterate over the list and delete the certs from ISE
# Strip the double quotes before executing the for loop
cleancurrentcerts=`echo $currentcerts | tr -d \"`
for id in ${cleancurrentcerts};
   do
       curl -s --cacert $thisscriptbase/$ISECAFile --netrc-file $curlsecrets/.netrc \
            -X 'DELETE' "https://$ISEGatewayFQDN:443/api/v1/certs/system-certificate/$ISEHostname/$id" \
                -H 'accept: application/json' \
                -H 'Content-Type: application/json' \
                -d "{\"allowWildcardDelete\": false}"
done

Make executable and create a link in the letsencrypt renew-hooks directory

chmod +x ~/ise_portal_auto_cert/import_ise_port_cert.sh
ln -s ~/ise_portal_auto_cert/import_ise_port_cert.sh  ~/certbotconf/renewal-hooks/deploy/import_ise_port_cert.sh

Easy peasy ...

PKI CA Cert Chain

Root CA import

Create the file below and paste in the Chain in PEM format - it's a once off process to allow the curl command to trust the ISE cert.

nano  ~/ise_portal_auto_cert/RNLAB-ROOTCA.pem

Letsencrypt CA Cert Chain

We need the Letsencrypt CA cert chain installed in ISE prior to the cert renewal process

Ensure that you have the Root and Issuing public CA certs installed (they differ based on whether you use RSA or ECC certs)

Check ISE Reports

You can keep an eye on certificate operations in the ISE Audit Reports.

Operations > Reports > Audit >OpenAPI Operations

Check certs and Simulate Dry Run

As root user, the first command will list your certificate status, and the second command will simulate a renewal:

certbot certificates
certbot renew --dry-run -v

Forced Renewal for Demo Purposes

During testing is was useful to force a certificate renewal process. This will make a new certificate before it is due. Beware though, Letsencrypt has limits on how often you can do that.

certbot renew --force-renewal  -v

[rezaalikhani]

Thank you for your great article. Very helpful and indeed needed. Why not published it in the Knowledge Base?

[Arne Bier]

I agree that since I was not asking a question, my article should have been posted in a knowledge base somewhere. But first of all, I cannot find this knowledgebase you're referring to, and I have a suspicion that I am not the only one - which means that the article will go unnoticed because some people (like myself) might have RSS feeds for NAC and other forums, and I don't look at most of the other forums.  I wanted maximum exposure - therefore I posted it here. And maybe there is a better way to post these kinds of articles in a KB of some kind, whilst letting people in THIS forum know that it's there.  

I'm open to any advice.  I'm also not a big fan of people posting non-questions in this forums - and I am guilty of this myself.

[rezaalikhani]

@Arne Bier https://community.cisco.com/t5/security-knowledge-base/tkb-p/4561-docs-security