Using Let’s Encrypt certs with Oracle Cloud Load Balancer — including auto renew feature
Oracle Cloud free tier includes a Load Balancer with up to 10 Mpbs bandwidth, enough for many projects, this article shows how to use SSL traffic encrypted with Let’s Encrypt free certs.
Continuing with my article My own dev/test cloud environment using Oracle Always Free instances I’ll extend it to use Let’s Encrypt certs with auto renew features because these certs are valid for 3 months and I want to renew them automatically. Following picture depicts the deployment diagram:
Unlike my previous example, instead of using NGINX as a reverse proxy for my apps now I am using Traefik due it allows on the fly publishing using Swarm labels.
Traefik includes a functionality to register LetsEncrypt certs but AFAIK there is no hook to redistribute this cert to other locations as Let’s Encrypt CertBot includes, so using a new Docker container including NGNIX and CertBot is enough.
Configuring Traefik reverse proxy
Lets configure first, Traefik reverse proxy, using this stack definition docker-compose-traefik.yml first create an external network used to communicate the reverse proxy and internal services exposed by Traefik and then deploy de stack.
$ docker network create -d overlay lb_network
$ docker stack deploy -c docker-compose-traefik.yml traefikWe included a whoami service at this stack to configure Oracle Cloud Load Balancer Backend Set Update Health Check (Networking->Load Balancers->Load Balancer Details->Backend Sets->Backend Set Details).
Once We have a Load Balancer configured and a Backend Set forwarding traffic to our Traefik instances (replicated) We need a public DNS entry associated to our Load Balancer, for example using NoIP free services We can define an A entry as:
Now We are ready to create and register a Let’s Encrypt cert associated with Oracle Cloud Load Balancer.
Docker image using CertBot, NGINX and OCI client
The step to create a Let’s Encrypt cert and associate it to our Oracle Cloud Load Balancer is implemented using a Docker container running in one of our two node Swarm clusters. Here a project structure to build our Docker image:
A Dockerfile for our image will include NGINX, Let’s Encrypt CertBot and Oracle OCI tool, see below:
Directory oci/ have Oracle CLI OCI credentials including a config, oci_api_key.pem and oci_api_key_public.pem files generated as is described at the article Administering OCI from WSL, following only steps starting at Connecting CLI to your tenancy section because all previous steps were made in our Dockerfile.
Once We have Docker images ready and uploaded at our docker registry, a docker-compose.yml file will be used to deploy a Let’s Encrypt CertBot stack:
We defined two services, once used to manually start a cert generation or debugging purpose named nginx, command: sleep 1d left our container running for one day and We could attach them using a docker exec command. The other service named renew will be used for a monthly trigger renewal process using Unix crontab.
Both services will intercept Web URLs like http://*/.well-known/acme-challenge/ which are used by Let’s Encrypt to validate our DNS/IP ownership.
Environment variables OCID and RENEWED_DOMAINS are used to customize our cert generation. OCID value is available at Networking->Load Balancers->Load Balancer Details (OCID Copy link)
Scripts defined at Docker file are, renew-certs.sh which call Let’s Encrypt CertBot:
CertBot hook calls to create-lb-certs.sh which will register our new cert at Oracle Cloud Load Balancer
Load Balancer certs are created with an unique name using the Domain Name and a serial number, We can’t replace or update an existing cert definition, If We trigger for the first time a cert creation using nginx service, console output will look like:
[opc@node2 ~]$ docker exec -ti letsencrypt_nginx.1.xz8ifbsorfkn35fsob4nuk022 bashroot@ffb330d973fb:/opt/letsencrypt# /renew-certs.sh
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Running post-hook command: /create-lb-certs.sh
Output from post-hook command create-lb-certs.sh:
dev-oci.servehttp.com-0002
{
"opc-work-request-id": "ocid1.loadbalancerworkrequest.oc1.iad......."
}
{
"opc-work-request-id": "ocid1.loadbalancerworkrequest.oc1.iad......."
}
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/dev-oci.servehttp.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/dev-oci.servehttp.com/privkey.pem
Your cert will expire on 2021-01-12. To obtain a new or tweaked version of this certificate in the future, simply run letsencrypt-auto again. To non-interactively renew *all* of your certificates, run "letsencrypt-auto renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
Once a new cert is registered the second step is to update the listener definition. We can check the steps by visiting Networking->Load Balancers->Load Balancer Details->Certificates console:
And that’s all, Oracle Cloud Load Balancer is ready to accept SSL traffic using a valid LetsEncrypt cert, We can test exposing another stack with Traefik, for example VS-Code web version, by Clicking at padlock icon
Finally a crontab Unix job at our master Swarn node will trigger a renew process monthly, for example:
[root@node1 yml]# crontab -l
52 2 1 * * root docker service scale letsencrypt_renew=0 && sleep 60 && docker service scale letsencrypt_renew=1Every month, the first day of the month at 2:52am.
December 2020 Update!!!
Above Dockerfile stop working this December because LetsEncrypt certbot clone from GitHub repo will no longer run on several Linux distros, they recommend to use oficial LetsEncrypt Docker image certbot/certbot, this Docker image is based on Alpine Linux then to get an extended image with Oracle OCI client I created to a new one based on the idea of this blog post “Get going quickly with Command Line Interface for Oracle Cloud Infrastructure using Docker container” by Lucas Jellema. Updated Dockerfile look like:
$ cat certbot/requirements.txt
oci
oci-cli
$ docker build --build-arg VERSION=v1.10.1 -t "certbot-oci:v1.10.1" ./certbotcreate-lb-cert.sh is similar:
but docker-compose.yml is more compact by using macros and new CertBot functionality:
a few comments:
- certbot service is only defined for debugging purpose or to create for the first time manually our cert.
- renew service will be called every month using crontab as is described above
- certbot can create one certificate for many domains, at the example I am using dev-oci.mydomain.com as primary and registry.mydomain.com as alternate domain, bot DNS entries must be pointed to our Load Balancer public IP.
- certs are stored at Oracle Cloud Object storage using S3 volume driver plugin
If We want to test above certs a sample registry stack using Traefik as inbound traffic controller look like:
Note that registry.mydomain.com only maps path prefix /v2/.
Registry UI frontend look like:
Testing Docker Registry + Traefik
[root@node2 letsencrypt]# docker pull hello-world:latest
Trying to pull repository docker.io/library/hello-world ...
latest: Pulling from docker.io/library/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:1a523af650137b8accdaed439c17d684df61ee4d74feac151b5b337bd29e7eec
Status: Downloaded newer image for hello-world:latest
hello-world:latest
[root@node2 letsencrypt]# docker tag hello-world:latest registry.mydomain.com/hello-world:latest
[root@node2 letsencrypt]# docker push registry.mydomain.com/hello-world:latest
The push refers to repository [registry.mydomain.com/hello-world]
......
[root@node2 letsencrypt]# docker rmi registry.mydomain.com/hello-world:latest hello-world:latest
Untagged: registry.mydomain.com/hello-world:latest
....
Deleted: sha256:9c27e219663c25e0f28493790cc0b88bc973ba3b1686355f221c38a36978ac63
[root@node2 letsencrypt]# docker run --rm registry.mydomain.com/hello-world:latest
Unable to find image 'registry.mydomain.com/hello-world:latest' locally
Trying to pull repository registry.mydomain.com/hello-world ...
latest: Pulling from registry.mydomain.com/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042
Status: Downloaded newer image for registry.mydomain.com/hello-world:latestHello from Docker!
This message shows that your installation appears to be working correctly.To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bashShare images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/For more examples and ideas, visit:
https://docs.docker.com/get-started/
Everything works fine again!!!
