Building Multi-CPU Architecture Docker Images for ARM and x86 (2): Building in GitLab CI

Photo by Pankaj Patel on Unsplash

In “Building Multi-CPU Architecture Docker Images for ARM and x86 (1): The Basics”, we covered the general workflow of building multi-arch Docker images with buildx/buildkit. However, to get it running on Gitlab CI, you will need some extra setup.

Enable Docker commands in GitLab CI/CD Jobs

There are three options to enable Docker commands in GitLab CI/CD Jobs:

• The shell executor
• Docker socket binding
• The Docker executor with the Docker image (Docker-in-Docker): requires either Docker executor or the Kubernetes executor

In this article, we will only cover the dind (Docker-in-Docker) option with k8s (Kubernetes) executor for setting up multi-arch building pipelines.

Deploy Kubernetes Executor Gitlab Runner

To deploy Kubernetes executor Gitlab Runner, you can use Helm to deploy the Helm Chart into your k8s cluster.

Firstly, we can create the config values file (`config.yaml`) for our deployment:

gitlabUrl: "https://gitlab.com/"
runnerRegistrationToken: "xxxxxx"
concurrent: 1
runners:
  tags: "k8s-runners"
  privileged: true
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "ubuntu:20.04"
        privileged = true
      [[runners.kubernetes.volumes.empty_dir]]
        name = "docker-certs"
        mount_path = "/certs/client"
        medium = "Memory"
      [runners.cache]
        Type = "gcs"
        Path = "gitlab_runner"
        Shared = false
        [runners.cache.gcs]
          AccessID = "xxx@xxxx.com"
          PrivateKey = "xxxxxxx"
          BucketName = "xxxxxx"

Here, you need to replace `runnerRegistrationToken` with your own registration token. The setup assumes using GCS (google cloud storage) as cache storage. If you use AWS s3, you can replace the[runners.cache] section with:

      [runners.cache]
        Type = "s3"
        Path = "runner"
        Shared = false
        [runners.cache.s3]
          ServerAddress = "s3.amazonaws.com"
          BucketName = "xxxxxx"
          BucketLocation = "eu-west-1"
          Insecure = false
          AuthenticationType = "access-key"
          AccessKey = "xxxx"
          SecretKey = "xxxxx"

Secondly, add Gitlab Runner Helm Chart repo (if you haven’t):

helm repo add gitlab https://charts.gitlab.io

Deploy the Gitlab Runner with config.yaml config file we created:

helm install --namespace <NAMESPACE> gitlab-runner -f config.yaml gitlab/gitlab-runner

Here, <NAMESPACE> is the Kubernetes namespace where you want to install the GitLab Runner.

Add ENV Variables to .gitlab-ci.yml

After deploying Kubernetes executor Gitlab Runner, we also need to add the following ENV variable to the Gitlab CI/CD Configuration File .gitlab-ci.yml :

variables:
  DOCKER_HOST: tcp://docker:2376
  DOCKER_TLS_CERTDIR: "/certs"
  DOCKER_TLS_VERIFY: 1
  DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"

Docker Image with `buildx` Pluigin

To run docker buildx build command in CI, you need a docker image containing docker command & `buildx` Plugin. We’ve covered how to make a such docker image in “Building Multi-CPU Architecture Docker Images for ARM and x86 (1): The Basics”.

you can also use my published pre-built docker image: data61/magda-builder-docker:latest .

Build Multi-arch Docker Images & Push to Gitlab Docker Registry

Before run docker buildx build , we need to create a “builder instance” with the following commands:

# Create a new docker context for builder instance to use
docker context create builder-context
# Create a builder instance named "builderx"
docker buildx create --name builderx --driver docker-container --use builder-context

Here is a complete example of building multi-arch Docker images and push to the Gitlab Docker registry:

# All env vars are auto-provided by GitLab CI at runtime
docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
# Create a new docker context for builder instance to use
docker context create builder-context
# Create a builder instance named "builderx"
docker buildx create --name builderx --driver docker-container --use builder-context
docker buildx build -t $CI_REGISTRY/xx/myimage:$CI_COMMIT_REF_SLUG -f [path to Dockerfile] --push --platform=linux/arm64,linux/amd64 [path to build context]

Retag & Copy Multi-arch Images to Docker Hub

In the CI pipeline, we often push docker images to the CI platform provided registry for testing & verification. But, eventually, we will want to retag & copy the multi-arch Docker image in Gitlab Registry to a production registry. e.g. Docker Hub.

With multi-arch images, we cannot use the old workflow to retag & push docker image to a different registry using docker tag & docker push command as they only work with one image rather than multiple images behinds one manifest registry entry.

To solve this issue, we can use regclient . Its command syntax is very simple. To retag & copy the image that we built & pushed to the Gitlab registry in the previous example and push to the Docker hub, we can run:

# Login to gitlab registry
docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
# Login to docker hub
docker login -u my-dockerhub-username -p $DOCKER_HUB_PASSWORD
regctl image copy $CI_REGISTRY/xx/myimage:$CI_COMMIT_REF_SLUG docker.io/xx/myimage:v1

Please note: my published pre-built docker image: data61/magda-builder-docker:latest has already included the regclient utility. If you use your own pre-built docker image, you will need to manually install it.

Next

Building Multi-CPU Architecture Docker Images for ARM and x86 (3): Building in GitHub Action CI