Photo by MF Evelyn on Unsplash

How to Deploy and Use Kube-fledged to Cache Images in Kubernetes

Fledging is the stage in a flying animal’s life between hatching or birth and becoming capable of flight. All birds are considered to have fledged when the feathers and wing muscles are sufficiently developed for flight. A young bird that has recently fledged but is still dependent upon parental care and feeding is called a fledgling. (Source: Wikipedia)

What is kube-fledged?

kube-fledged is a kubernetes add-on or operator for creating and managing a cache of container images directly on the worker nodes of a kubernetes cluster. It allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pulled). As a result, application pods start almost instantly, since the images need not be pulled from the registry. kube-fledged provides CRUD APIs to manage the lifecycle of the image cache, and supports several configurable parameters to customize the functioning as per one’s needs. (Repo URL: https://github.com/senthilrch/kube-fledged)

In this blogpost, I’ll explain (i) how to deploy kube-fledged, (ii) configuration flags, (iii) creating a cache of container images in a Kubernetes cluster and (iv) the various image cache actions supported

Prerequisites

• A running K8s cluster. You should have cluster-admin privileges on the cluster to deploy kube-fledged
• kubectl, make and git installed on a local Linux or Mac machine.

Deploy kube-fledged using YAML Manifests

These instructions deploy kube-fledged to a separate namespace called “kube-fledged”, using YAML manifests and pre-built images in Docker Hub.

• Clone the source code repository

mkdir -p $HOME/src/github.com/senthilrch
git clone https://github.com/senthilrch/kube-fledged.git $HOME/src/github.com/senthilrch/kube-fledged
cd $HOME/src/github.com/senthilrch/kube-fledged

• Deploy kube-fledged to the cluster

make deploy-using-yaml

• Verify if kube-fledged deployed successfully

kubectl get pods -n kube-fledged -l app=kubefledged
kubectl get imagecaches -n kube-fledged (Output should be: 'No resources found')

kube-fledged supports deploying via Helm Chart and Helm Operator as well. Refer to the README of the project (https://github.com/senthilrch/kube-fledged) for the steps to deploy using these methods.

Kube-fledged Configuration Flags

kube-fledged supports several configuration flags which can be used to customize the behaviour.

--image-pull-deadline-duration: Maximum duration allowed for pulling an image. After this duration, image pull is considered to have failed. default "5m"

--image-cache-refresh-frequency: The image cache is refreshed periodically to ensure the cache is up to date. Setting this flag to "0s" will disable refresh. default "15m"

--image-pull-policy: Image pull policy for pulling images into and refreshing the cache. Possible values are 'IfNotPresent' and 'Always'. Default value is 'IfNotPresent' (Image with no or ":latest" tag are always pulled)

--stderrthreshold: Log level. default is INFO

Create Image Cache

The first step in creating an image cache on the cluster, is to determine the list of images to be cached and onto which nodes those images should be cached. Let’s assume you need to:-

• cache images quay.io/bitnami/nginx:1.21.1 and quay.io/bitnami/tomcat:10.0.8 to all the nodes in the cluster
• cache images quay.io/bitnami/redis:6.2.5 and quay.io/bitnami/mariadb:10.5.11 only to nodes with label tier: backend

Create a file named kubefledged-imagecache.yaml with the following contents:

apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:
  # Name of the image cache.
  name: imagecache1
  # The namespace to be used for this image cache.
  namespace: kube-fledged
  labels:
    app: kubefledged
    component: imagecache
spec:
  # The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).
  cacheSpec:
  # Specifies a list of images (nginx:1.21.1 and tomcat:10.0.8) with no node selector, hence these images will be cached in all the nodes in the cluster
  - images:
    - quay.io/bitnami/nginx:1.21.1
    - quay.io/bitnami/tomcat:10.0.8
  # Specifies a list of images (redis:6.2.5 and mariadb:10.5.11) with a node selector, hence these images will be cached only on the nodes selected by the node selector
  - images:
    - quay.io/bitnami/redis:6.2.5
    - quay.io/bitnami/mariadb:10.5.11
    nodeSelector:
      tier: backend

Use kubectl to create the image cache

kubectl create -f kubefledged-imagecache.yaml

At this point, the image cache manifest is submitted to k8s api server. The api server creates a HTTP POST request and sends it to kubefledged-webhook-server. The webhook server validates the contents of ImageCache resource and returns back a successful response to the api server.

The api server then persists the ImageCache resource in etcd. kubefledged-controller then creates several k8s jobs on the worker nodes for pulling the images into the cache. One job is responsible for pulling one image into one node. Once all the jobs are completed successfully, the controller updates the status field of the ImageCache resource. Users can query the ImageCache resource and look at the status field to know if image cache was created successfully or not. If there are failures, the status field will also have error message and error description pin-pointing the reason for the failures

Verify the status of image cache creation

kubectl get imagecache imagecache1 -n kube-fledged -o yaml

Modify Image Cache

Once an image cache has been successfully created, it is possible to make changes to it by modifying the image cache manifest and re-submitting it to the cluster. Let’s assume you want to remove the image quay.io/bitnami/tomcat:10.0.8 from the image cache.

Edit the image cache manifest file kubefledged-imagecache.yaml as follows:-

apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:
  # Name of the image cache.
  name: imagecache1
  # The namespace to be used for this image cache.
  namespace: kube-fledged
  labels:
    app: kubefledged
    component: imagecache
spec:
  # The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).
  cacheSpec:
  # Specifies a list of images (nginx:1.21.1) with no node selector, hence these images will be cached in all the nodes in the cluster
  - images:
    - quay.io/bitnami/nginx:1.21.1
  # Specifies a list of images (redis:6.2.5 and mariadb:10.5.11) with a node selector, hence these images will be cached only on the nodes selected by the node selector
  - images:
    - quay.io/bitnami/redis:6.2.5
    - quay.io/bitnami/mariadb:10.5.11
    nodeSelector:
      tier: backend

Use kubectl to apply the changes to image cache

kubectl apply -f kubefledged-imagecache.yaml

kubefledged-controller will detect the changes to the ImageCache resource and determines that image quay.io/bitnami/tomcat:10.0.8 need to be removed from the image cache on the cluster. So, it creates jobs to delete the image from the image cache. The results of these actions are updated in the status field of the ImageCache resource.

Verify the status of image cache modification

kubectl get imagecache imagecache1 -n kube-fledged -o yaml

Purge Image Cache

If you decide to remove all the images from the image cache, this can be done by submitting a purge request. The purge request is submitted by annotating the ImageCache resource using the following command

kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/purge-imagecache=

Verify the status of purging the image cache

kubectl get imagecache imagecache1 -n kube-fledged -o yaml

Refresh Image Cache

Once an image cache is purged, it can be easily restored back by submitting a refresh request. The refresh request is submitted by annotating the ImageCache resource using the following command

kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=

Verify the status of Image Cache Refresh

kubectl get imagecache imagecache1 -n kube-fledged -o yaml

Conclusion

In this blog I explained how to use kube-fledged to create a cache of container images on a kubernetes cluster and how to perform different actions on the created image cache: view, modify, purge, refresh. Head to the project’s github repository: https://github.com/senthilrch/kube-fledged

If you want to learn more about kube-fledged, read my previous blog Kube-fledged: Cache Container Images in Kubernetes

👉 I tweet & blog regularly on Kubernetes and Cloud-Native Tech. Follow me on Twitter and Medium