Automatic renewal of Let’s Encrypt SSL certificates with Cloudflare using Ansible

• Author
• Recent Posts

Riccardo Bicelli

Riccardo is a system administrator and software developer for small and mid-size businesses in Italy.

Latest posts by Riccardo Bicelli (see all)

• Tactical RMM: Open Source remote monitoring and management for Windows - Thu, Jan 27 2022
• Use OpenSSL-based software XCA as offline root certificate authority for AD Certificate Services - Wed, Jan 27 2021
• Deploy software with WPKG and Active Directory - Tue, Dec 8 2020

Prerequisites

Consider a scenario such as this:

The Ansible host will contact Cloudflare servers via the Cloudflare API for the DNS101 challenge. Once the certificate is obtained or renewed, it will deploy the certificate on IIS Servers (via Ansible) and on NetScaler (via ns-letsencrypt script).

A typical Citrix delivery controller—NetScaler scenario

Let's make some assumptions:

• An Ansible installation with managed Windows hosts is in place
• You have a domain with DNS managed by Cloudflare
• You have basic knowledge of the Linux, Git, and Ansible command lines.
• The Active Directory domain name is lab.example.com.
• The public domain name is example.com.
• The internal FQDN of NetScaler and delivery controllers is appsinternal.example.com.
• The external FQDN of NetScaler is apps.example.com.
• The Ansible username on the Ansible host is ansible.
• The Ansible directory is /etc/ansible.

Please note that the Ansible host also acts as the Let's Encrypt certificate store. This is not a pure Ansible approach; some scripting is involved, especially for Let's Encrypt renewal hooks and uploading the NetScaler certificate.

Install the required tools

On your Ansible host, you need to install my modified version of the ns-letsencrypt:

cd /opt
git clone https://github.com/rbicelli/ns-letsencrypt.git
cp /opt/ns-letsencrypt/mynsconfig.py.example /opt/ns-letsencrypt/mynsconfig.py

Then edit the following in mynsconfig.py:

nitroNSIP = "NETSCALER IP ADDRESS"
nitroUser = "NetScaler Username"
nitroPass = "NetScaler Password"

Check NetScaler connectivity with:

/opt/ns-letsencrypt/ns-copytons.py test

Cloudflare configuration

Go to your Cloudflare DNS control panel and create:

• An A record: vpx.example.com that points to the NetScaler public IP
• A CNAME record named apps.example.com that points to vpx.example.com
• A CNAME record named appsinternal.example.com that points to vpx.example.com

Then go to My Profile > API Tokens, and get your global API key.

On the Ansible host, create the file /etc/secrets/cloudflare.cfg:

dns_cloudflare_email = "your@cloudflare.account"
dns_cloudflare_api_key = "YOUR_GLOBAL_API_KEY"

Then set the file permission to 0600 and the directory permission to 0700.

Please note that you can also create an API token suited for DNS zone update only, which is recommended since it is more secure. However, you need at least version 2.3.1 of the Python Cloudflare API. In this case, your /etc/secrets/cloudflare.cfg content would look like this:

dns_cloudflare_api_token = "YOUR_CLOUDFLARE_API_TOKEN"

First run: Get certificate from Let's Encrypt

On the Ansible host, as root, launch the command for creating the Let's Encrypt certificate:

certbot certonly --rsa-key-size 2048 --dns-cloudflare --dns-cloudflare-credentials /etc/secrets/cloudflare.cfg -d vpx.example.com,apps.example.com,appsinternal.example.com --preferred-challenges dns-01

Create an Ansible playbook

In your Ansible host file, typically /etc/ansible/hosts, define a group that will include your delivery controllers:

[xdc_servers]
xdc1.lab.example.com
xdc2.lab.example.com

Install the ar_win_iis_lecert role either from Ansible Galaxy or from my github.

ansible-galaxy install rbicelli.ar_win_iis_lecert

or

git clone https://github.com/rbicelli/ar_win_iis_lecert.git

Then create the Ansible playbook /etc/ansible/win-xdc-updatecerts.yml for delivery controllers:

- hosts: xdc_servers
 become: yes
 vars:
  virtual: yes
  cert_name: vpx.example.com
 roles:
  - role: ar_win_iis_lecert

Set up the renewal hook

The renewal hook simply handles these tasks:

• Converts the certificate to PFX, which is the format supported by Windows
• Copies the certificate to NetScaler
• Calls the Ansible playbook responsible for copying and binding certificates to IIS on delivery controllers

Create the script /etc/letsencrypt/custom-hooks/iis-le-cert-renew.sh:

#!/bin/bash
DOMAIN=$1

#Convert Certificate to pfx
CERT_DIR="/etc/letsencrypt/live/$DOMAIN"

if [ -d $CERT_DIR ]; then
    openssl pkcs12 -export -out $CERT_DIR/cert.pfx -inkey $CERT_DIR/privkey.pem -in $CERT_DIR/cert.pem -passout pass:
else
  	echo "ERROR: Source certificate doesn't exist"
	exit 1
fi

#Install Certificate to delivery controllers
sudo -u ansible ansible-playbook /etc/ansible/win-xdc-updatecerts.yml

#Copy Certificate to NetScaler
#Note, you can remove this part in order to deploy certificates to IIS only
cd /opt/ns-letsencrypt
./ns-copy.sh $DOMAIN

Save it and make it executable:

chmod 750 /etc/letsencrypt/custom-hooks/iis-le-cert-renew.sh

Then edit the file /etc/letsencrypt/renewal/vpx.example.com, and add the following in the [renewalparams] section:

renew_hook = /etc/letsencrypt/custom-hooks/iis-le-cert-renew.sh vpx.example.com

Deploy certificates on IIS and NetScaler

After the initial certificate is issued, you have to deploy the certificate on IIS hosts and NetScaler. The most comfortable way is to manually run the renewal hook script from the Ansible host:

/etc/letsencrypt/custom-hooks/iis-le-cert-renew.sh vpx.example.com

It should copy the certificate on IIS and NetScaler. To be sure, log in to your IIS machines and check whether the certificate is present in the certificate store, is valid, and has the correct bindings.

Sometimes it is better to check directly with the browser

Then you can bind certificates in NetScaler.

Automatic renewal: Setup cron

After the first run, the renewal process should run smoothly without user intervention.

For automatic renewal, you can set up a cron job by simply adding a script called /etc/cron.daily/certbot-renew.sh with this content:

#!/bin/bash
certbot renew

Then reload or restart your cron daemon. When the certificate is close to expiration, the renewal process will take place, and previously defined renewal hooks will handle the certificate upload on IIS and NetScaler.

Troubleshooting certificates on IIS

The Ansible playbook creates a PowerShell script. It can be found in "%TEMP%\update-cert.ps1", where %TEMP% is the temporary directory of Ansible assigned for Windows administration.

After playbook execution, check whether the script exists and try to run it manually.

Conclusion

You can take inspiration from this guide to handle comparable tasks. For instance, if you remove the NetScaler part, you can achieve a solution for managing certificates for your entire fleet of IIS servers.

This solution can also be improved by moving the NetScaler certificate deployment entirely to Ansible.