Let's Encrypt with Octavia in OpenStack

I like using Catalyst Cloud to host some of my personal sites. In the past I used to use CAcert for my TLS certificates, but more recently I've been using Let's Encrypt for my TLS certificates as they're trusted in all browsers. Currently the LoadBalancer as a Service (LBaaS) in Catalyst Cloud doesn't have built in support for Let's Encrypt. I could use an apache2/nginx proxy and handle the TLS termination there and have that manage the Let's Encrypt lifecycle, but really, I'd rather use LBaaS.

So I thought I'd set about working out how to get Dehydrated (the Let's Encrypt client I've been using) to drive LBaaS (known as Octavia). I figured this would be of interest to other people using Octavia with OpenStack in general, not just Catalyst Cloud.

There's a few things you need to do. These instructions are specific to Debian:

1. Install and configure Dehydrated to create the certificates for the domain(s) you want.
   • apt install barbican
2. Create the LoadBalancer (use the API, ClickOps, whatever), just forward port 80 for now (see sample Apache configs below).
3. Save the sample hook.sh below to /etc/dehydrated/hook.sh, you'll probably need to customise it, mine is a bit more complicated!
4. Insert the UUID of your LoadBalancer in hook.sh where LB_LISTENER is set.
5. Create /etc/dehydrated/catalystcloud/password as described in hook.sh
6. Save OpenRC file from the Catalyst Cloud dashboard as /etc/dehydrated/catalystcloud/openrc.sh
7. Install jq, openssl and the openstack tools, on Debian this is:
   • apt install jq openssl python3-openstackclient python3-barbicanclient python3-octaviaclient
8. Add TLS termination to your LoadBalancer
9. You should be able to rename the latest certs /var/lib/dehydrated/certs/$DOMAIN and then run dehydrated -c to have it reissue and then deploy a cert.

As we're using HTTP-01 Challenge Type here, you need to have the LoadBalancer forwarding port 80 to your website to allow for the challenge response. It is good practice to have a redirect to HTTPS, here's an example virtual host for Apache:

<VirtualHost *:80>
    ServerName www.example.com
    ServerAlias example.com

    RewriteEngine On
    RewriteRule ^/.well-known/ - [L]
    RewriteRule ^/(.*)$ https://www.example.com/$1 [R=301,L]

    <Location />
        Require all granted
    </Location>
</VirtualHost>

Alias /.well-known/acme-challenge /var/lib/dehydrated/acme-challenges

<Directory /var/lib/dehydrated/acme-challenges>
        Options None
        AllowOverride None

        # Apache 2.x
        <IfModule !mod_authz_core.c>
                Order allow,deny
                Allow from all
        </IfModule>

        # Apache 2.4
        <IfModule mod_authz_core.c>
                Require all granted
        </IfModule>
</Directory>

And that should be all that you need to do. Now, when Dehydrated updates your certificate, it should update your LoadBalancer as well!

deploy_cert() {
    local DOMAIN="${1}" KEYFILE="${2}" CERTFILE="${3}" FULLCHAINFILE="${4}" \
          CHAINFILE="${5}" TIMESTAMP="${6}"
    shift 6

    # File contents should be:
    #   export OS_PASSWORD='your password in here'
    . /etc/dehydrated/catalystcloud/password

    # OpenRC file from the Catalyst Cloud dashboard
    . /etc/dehydrated/catalystcloud/openrc.sh --no-token

    # UUID of the LoadBalancer to be managed
    LB_LISTENER='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

    # Barbican uses P12 files, we need to make one.
    P12=$(readlink -f $KEYFILE \
        | sed -E 's/privkey-([0-9]+)\.pem/barbican-\1.p12/')
    openssl pkcs12 -export -inkey $KEYFILE -in $CERTFILE -certfile \
        $FULLCHAINFILE -passout pass: -out $P12

    # Keep track of existing certs for this domain (hopefully no more than 100)
    EXISTING_URIS=$(openstack secret list --limit 100 \
        -c Name -c 'Secret href' -f json \
        | jq -r ".[]|select(.Name | startswith(\"$DOMAIN\"))|.\"Secret href\"")

    # Upload the new cert
    NOW=$(date +"%s")
    openstack secret store --name $DOMAIN-$TIMESTAMP-$NOW -e base64 \
        -t "application/octet-stream" --payload="$(base64 < $P12)"

    NEW_URI=$(openstack secret list --name $DOMAIN-$TIMESTAMP-$NOW \
        -c 'Secret href' -f value) \
        || unset NEW_URI

    # Change LoadBalancer to use new cert - if the old one was the default,
    # change the default. If the old one was in the SNI list, update the
    # SNI list.
    if [ -n "$EXISTING_URIS" ]; then
        DEFAULT_CONTAINER=$(openstack loadbalancer listener show $LB_LISTENER \
            -c default_tls_container_ref -f value)

        for URI in $EXISTING_URIS; do
            if [ "x$URI" = "x$DEFAULT_CONTAINER" ]; then
                openstack loadbalancer listener set $LB_LISTENER \
                    --default-tls-container-ref $NEW_URI
            fi
        done

        SNI_CONTAINERS=$(openstack loadbalancer listener show $LB_LISTENER \
            -c sni_container_refs -f value | sed "s/'//g" | sed 's/^\[//' \
            | sed 's/\]$//' | sed "s/,//g")

        for URI in $EXISTING_URIS; do
            if echo $SNI_CONTAINERS | grep -q $URI; then
                SNI_CONTAINERS=$(echo $SNI_CONTAINERS | sed "s,$URI,$NEW_URI,")
                openstack loadbalancer listener set $LB_LISTENER \
                    --sni-container-refs $SNI_CONTAINERS
            fi
        done

        # Remove old certs
        for URI in $EXISTING_URIS; do
            openstack secret delete $URI
        done
    fi
}

HANDLER="$1"; shift
#if [[ "${HANDLER}" =~ ^(deploy_challenge|clean_challenge|sync_cert|deploy_cert|deploy_ocsp|unchanged_cert|invalid_challenge|request_failure|generate_csr|startup_hook|exit_hook)$ ]]; then
if [[ "${HANDLER}" =~ ^(deploy_cert)$ ]]; then
    "$HANDLER" "$@"
fi