The official gitlab helm chart for pages does not support a cert manager for *.pages.example.org as this is officially
not supported. Thus you have to create the
wildcard certificate manually
like described in the docs.
So how to update this regulary? I think there are multiple options (using a different tool then cert manager, running a cronjob in k8s doing acme.sh or others), but I choose today: a scheduled pipeline in gitlab.
Our setup uses acme.sh and hetzner dns (which is one of the acme.sh dns apis).
This requires that hetzner dns is capable of adding TXT DNS entries to the domain.
If you would do it manually, you would do the following steps.
Install acme.sh:
$ curl https://get.acme.sh | sh -s [email protected]
If you need a specific version (I like to PIN dependencies, as this is best practices) and store this in environment variable $ACME_SH_VERSION it looks like this (because BRANCH must match the version):
$ export ACME_SH_VERSION=3.0.1
$ curl -Ss https://raw.githubusercontent.com/acmesh-official/acme.sh/$ACME_SH_VERSION/acme.sh | BRANCH=$ACME_SH_VERSION sh -s -- --install-online -m [email protected]
This might run into:
Please install openssl first. ACME_OPENSSL_BIN=openssl
We need openssl to generate keys.
Pre-check failed, can not install.
so if you are on alpine:latest, run: apk add --no-cache openssl.
The log on alpine:latest (if you run acme.sh with --debug 2) shows an error that wget is missing -d and that sed cannot execute -i, so I added those, too:
$ apk add --no-cache sed wget openssl
The pipeline looks like this:
stages:
- deploy
deploy wildcard certificate:
stage: deploy
image: google/cloud-sdk:369.0.0-alpine # or use your deploy image
variables:
GKE_SERVICE_ACCOUNT_JSON: "$GKE_SERVICE_ACCOUNT_JSON" # The token must be set in CI/CD Variables in Gitlab if you need this in before_script
HETZNER_Token: "$HETZNER_Token" # The token must be set in CI/CD Variables in Gitlab
ACME_SH_VERSION: "3.0.1"
EMAIL: "[email protected]" # FIX THIS FOR YOURSELF!
WILDCARD_DOMAIN: "pages.example.org"
before_script: # create connection to k8s (this is gcp, change it to whatever way you need if k8s is hosted elsewhere!)
- gcloud auth activate-service-account --key-file=<( echo "$GKE_SERVICE_ACCOUNT_JSON")
- gcloud config set project my-gcp-project
- gcloud container clusters get-credentials my-k8s-cluster --region europe-west1
script: # expects to have a k8s connection
- apk add --no-cache curl openssl wget sed # we need curl to install and openssl, wget and sed in non-busybox version
- curl -Ss https://raw.githubusercontent.com/acmesh-official/acme.sh/$ACME_SH_VERSION/acme.sh | BRANCH=$ACME_SH_VERSION sh -s -- --install-online -m $EMAIL
- ~/.acme.sh/acme.sh --issue --dns dns_hetzner -d "${WILDCARD_DOMAIN}" -d "*.${WILDCARD_DOMAIN}" --fullchain-file /root/tls.crt --key-file /root/tls.key
- kubectl create secret tls gitlab-pages-tls --cert=/root/tls.crt --key=/root/tls.key --dry-run=true -o yaml | kubectl apply -f -
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule"'
when: always
- if: '$CI_PIPELINE_SOURCE != "schedule"'
when: manual
An execution looks like this:
[Sun Jan 23 11:40:45 UTC 2022] Using CA: https://acme.zerossl.com/v2/DV90
[Sun Jan 23 11:40:45 UTC 2022] Create account key ok.
[Sun Jan 23 11:40:45 UTC 2022] No EAB credentials found for ZeroSSL, let's get one
[Sun Jan 23 11:40:46 UTC 2022] Registering account: https://acme.zerossl.com/v2/DV90
[Sun Jan 23 11:40:49 UTC 2022] Registered
[Sun Jan 23 11:40:49 UTC 2022] ACCOUNT_THUMBPRINT='REDACTED-FIVE'
[Sun Jan 23 11:40:49 UTC 2022] Creating domain key
[Sun Jan 23 11:40:49 UTC 2022] The domain key is here: /root/.acme.sh/pages.example.org/pages.example.org.key
[Sun Jan 23 11:40:49 UTC 2022] Multi domain='DNS:pages.example.org,DNS:*.pages.example.org'
[Sun Jan 23 11:40:49 UTC 2022] Getting domain auth token for each domain
[Sun Jan 23 11:41:00 UTC 2022] Getting webroot for domain='pages.example.org'
[Sun Jan 23 11:41:00 UTC 2022] Getting webroot for domain='*.pages.example.org'
[Sun Jan 23 11:41:00 UTC 2022] Adding txt value: REDACTED-ONE for domain: _acme-challenge.pages.example.org
[Sun Jan 23 11:41:01 UTC 2022] Adding record
[Sun Jan 23 11:41:02 UTC 2022] Record added, OK
[Sun Jan 23 11:41:05 UTC 2022] The txt record is added: Success.
[Sun Jan 23 11:41:05 UTC 2022] Adding txt value: REDACTED-TWO for domain: _acme-challenge.pages.example.org
[Sun Jan 23 11:41:05 UTC 2022] Adding record
[Sun Jan 23 11:41:06 UTC 2022] Record added, OK
[Sun Jan 23 11:41:09 UTC 2022] The txt record is added: Success.
[Sun Jan 23 11:41:09 UTC 2022] Let's check each DNS record now. Sleep 20 seconds first.
[Sun Jan 23 11:41:30 UTC 2022] You can use '--dnssleep' to disable public dns checks.
[Sun Jan 23 11:41:30 UTC 2022] See: https://github.com/acmesh-official/acme.sh/wiki/dnscheck
[Sun Jan 23 11:41:30 UTC 2022] Checking pages.example.org for _acme-challenge.pages.example.org
[Sun Jan 23 11:41:30 UTC 2022] Domain pages.example.org '_acme-challenge.pages.example.org' success.
[Sun Jan 23 11:41:30 UTC 2022] Checking pages.example.org for _acme-challenge.pages.example.org
[Sun Jan 23 11:41:30 UTC 2022] Domain pages.example.org '_acme-challenge.pages.example.org' success.
[Sun Jan 23 11:41:30 UTC 2022] All success, let's return
[Sun Jan 23 11:41:30 UTC 2022] Verifying: pages.example.org
[Sun Jan 23 11:41:35 UTC 2022] Processing, The CA is processing your order, please just wait. (1/30)
[Sun Jan 23 11:41:42 UTC 2022] Success
[Sun Jan 23 11:41:42 UTC 2022] Verifying: *.pages.example.org
[Sun Jan 23 11:41:47 UTC 2022] Processing, The CA is processing your order, please just wait. (1/30)
[Sun Jan 23 11:41:55 UTC 2022] Success
[Sun Jan 23 11:41:55 UTC 2022] Removing DNS records.
[Sun Jan 23 11:41:55 UTC 2022] Removing txt: REDACTED-ONE for domain: _acme-challenge.pages.example.org
[Sun Jan 23 11:41:59 UTC 2022] Record deleted
[Sun Jan 23 11:41:59 UTC 2022] Removed: Success
[Sun Jan 23 11:41:59 UTC 2022] Removing txt: REDACTED-TWO for domain: _acme-challenge.pages.example.org
[Sun Jan 23 11:42:03 UTC 2022] Record deleted
[Sun Jan 23 11:42:03 UTC 2022] Removed: Success
[Sun Jan 23 11:42:03 UTC 2022] Verify finished, start to sign.
[Sun Jan 23 11:42:03 UTC 2022] Lets finalize the order.
[Sun Jan 23 11:42:03 UTC 2022] Le_OrderFinalize='https://acme.zerossl.com/v2/DV90/order/REDACTED_THREE/finalize'
[Sun Jan 23 11:42:11 UTC 2022] Order status is processing, lets sleep and retry.
[Sun Jan 23 11:42:11 UTC 2022] Retry after: 15
[Sun Jan 23 11:42:27 UTC 2022] Polling order status: https://acme.zerossl.com/v2/DV90/order/REDACTED_THREE
[Sun Jan 23 11:42:29 UTC 2022] Downloading cert.
[Sun Jan 23 11:42:29 UTC 2022] Le_LinkCert='https://acme.zerossl.com/v2/DV90/cert/REDACTED_FOUR'
[Sun Jan 23 11:42:32 UTC 2022] Cert success.
-----BEGIN CERTIFICATE-----
[[REDACTED]]
-----END CERTIFICATE-----
[Sun Jan 23 11:42:32 UTC 2022] Your cert is in: /root/.acme.sh/pages.example.org/pages.example.org.cer
[Sun Jan 23 11:42:32 UTC 2022] Your cert key is in: /root/.acme.sh/pages.example.org/pages.example.org.key
[Sun Jan 23 11:42:32 UTC 2022] The intermediate CA cert is in: /root/.acme.sh/pages.example.org/ca.cer
[Sun Jan 23 11:42:32 UTC 2022] And the full chain certs is there: /root/.acme.sh/pages.example.org/fullchain.cer
[Sun Jan 23 11:42:32 UTC 2022] Installing key to: /root/tls.key
[Sun Jan 23 11:42:32 UTC 2022] Installing full chain to: /root/tls.crt
And the kubectl output states:
$ kubectl create secret tls gitlab-pages-tls --cert=/root/tls.crt --key=/root/tls.key --dry-run=true -o yaml | kubectl apply -f -
secret/gitlab-pages-tls configured
It works! So now I can deploy pages on the helm chart deployed gitlab and they certificate works. If I setup a schedule in gitlab now for this branch, I will get recreation of the certificate every 30 days or anything else which is feasible to never have a certificate which expires (it is usually safe for 90 days, so scheduling for every 30 days is sufficient if you are fine with failing one time :)).
