From Kubernetes to Local Development

Use Podman to run Kubernetes manifests on your local machine

Published in

ITNEXT

6 min readFeb 22, 2024

I previously wrote about my workflow for running containers locally and generating Pod definitions to run in Kubernetes. However, it’s also useful to work in the opposite direction. I frequently want to take a Pod running in Kubernetes and run it on my local workstation, without the overhead of a Kubernetes cluster.

Podman supports this exact use case with the podman kube play command. This powerful Podman feature allows you to run Kubernetes manifests locally as Podman resources. In this article, I’ll show you how to get started with podman kube play, and I’ll explain some important considerations when using this functionality.

Run a Simple Pod Definition

The easiest way to get started with podman kube play is to run it using a basic Kubernetes Pod definition. The example below shows a simple Nginx Pod:

---
# Pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx-pod
    image: docker.io/nginx:latest

You can use the podman kube play command to transform this Pod definition into a locally running Podman Pod:

$ podman kube play Pod.yaml

Podman starts up a Pod with both the workload container and a special “podman-pause” container:

# The pod definition
$ podman pod ps
POD ID        NAME        STATUS      CREATED             INFRA ID      # OF CONTAINERS
89590e378cac  nginx-pod   Running     About a minute ago  46d13ea671fb  2

# The containers inside the pod
$ podman ps
CONTAINER ID  IMAGE                           COMMAND               CREATED             STATUS             PORTS       NAMES
46d13ea671fb  localhost/podman-pause:4.8.3-0                        About a minute ago  Up About a minute              89590e378cac-infra
148181794a3d  docker.io/library/nginx:latest  nginx -g daemon o...  About a minute ago  Up About a minute              nginx-pod-nginx-pod

Once you are done experimenting, you can clean up your running Pod with the --down flag:

$ podman kube play --down Pod.yaml
Pods stopped:
89590e378caccc591c3ebe38a31cf8ad17a00b31f22bc170021fbb27a2585b03
Pods removed:
89590e378caccc591c3ebe38a31cf8ad17a00b31f22bc170021fbb27a2585b03
Secrets removed:
Volumes removed:

Using the podman kube play command is perfect if you already have a Pod definition file. For example, your application may include a Pod definition for deploying your software. However, it’s more common to run an existing Pod in a Kubernetes cluster. In the next section, you’ll take a look at how to pull the definition for an existing Pod from a Kubernetes cluster, modify it, and run it locally using Podman.

Run an Existing Kubernetes Pod

I frequently experiment on my local machine with a workload that is already running in a Kubernetes cluster. I may have a complex application in Kubernetes, and I want to focus on a particular container without the overhead of running a cluster locally. While podman kube play makes this much easier, there are some caveats to consider when trying to run an existing Pod definition from a Kubernetes cluster.

To see this in action, simulate an existing Pod in the cluster by using kubectl apply on the Pod definition from the previous section:

$ kubectl apply -f Pod.yaml
pod/nginx-pod created

Next, use kubectl to obtain the YAML-formatted Pod definition:

$ kubectl get pod nginx-pod -o yaml > nginx-pod.yaml

The raw Pod definition from the Kubernetes API contains certain configuration that may conflict with Podman. You will likely encounter errors if you use podman kube play on the direct output of kubectl get pod:

$ podman kube play nginx-pod.yaml
Error: annotation "kubectl.kubernetes.io/last-applied-configuration"="{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"annotations\":{},\"name\":\"nginx-pod\",\"namespace\":\"default\"},\"spec\":{\"containers\":[{\"image\":\"docker.io/nginx:latest\",\"name\":\"nginx-pod\"}]}}\n" value length exceeds Kubernetes max 63

It appears that you can simply remove the annotation from the kubectl apply command. However, you will still encounter other issues. For example, podman kube play only supports certain volumes. Even the default Kubernetes Pod volumes, such as the default service account token, will cause issues:

$ podman kube play nginx-pod-edited.yaml
Error: failed to create volume "kube-api-access-96h5r": HostPath, ConfigMap, EmptyDir, Secret, and PersistentVolumeClaim are currently the only supported VolumeSource

You must remove all offending lines in the generated YAML to avoid these problems. This involves removing any long annotations and dropping any unsupported volumeMounts and volumes. The diff below shows all lines that were removed from the default output of kubectl get pod:

$ diff nginx-pod.yaml nginx-pod-edited.yaml
4,6d3
<   annotations:
<     kubectl.kubernetes.io/last-applied-configuration: |
<       {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"nginx-pod","namespace":"default"},"spec":{"containers":[{"image":"docker.io/nginx:latest","name":"nginx-pod"}]}}
20,23d16
<     volumeMounts:
<     - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
<       name: kube-api-access-96h5r
<       readOnly: true
44,62d36
<   volumes:
<   - name: kube-api-access-96h5r
<     projected:
<       defaultMode: 420
<       sources:
<       - serviceAccountToken:
<           expirationSeconds: 3607
<           path: token
<       - configMap:
<           items:
<           - key: ca.crt
<             path: ca.crt
<           name: kube-root-ca.crt
<       - downwardAPI:
<           items:
<           - fieldRef:
<               apiVersion: v1
<               fieldPath: metadata.namespace
<             path: namespace

The podman kube play command succeeds once the file has been cleaned up:

$ podman kube play nginx-pod-edited.yaml
Trying to pull docker.io/library/nginx:latest...
Getting image source signatures
Copying blob c7f80e9cdab2 skipped: already exists
Copying blob 18a869624cb6 skipped: already exists
Copying blob e1caac4eb9d2 skipped: already exists
Copying blob c3ea3344e711 skipped: already exists
Copying blob cc1bb4345a3a skipped: already exists
Copying blob da8fa4352481 skipped: already exists
Copying blob 88f6f236f401 skipped: already exists
Copying config e4720093a3 done   |
Writing manifest to image destination
Pod:
3ab0271b732fa096d3e304008f574ec00113b81c0125466d2364c22f0f72db71
Container:
0a11aad4c017bed2848c4f6da9a439b30fee8cfdc347e9ff72c089b6513c4d2a

Using podman kube play on YAML generated by kubectl get pod is the most common way that I leverage this command. It’s incredibly useful to find a Pod in a Kubernetes cluster, download the YAML definition, and run it locally on my machine. This avoids the overhead of running an entire Kubernetes cluster when I only need to experiment with a single Pod.

Run a Pod with Network and Storage

The previous examples used very simple workloads to illustrate the process of locally running a Kubernetes workload. However, production workloads are likely to include network and storage configurations.

The podman kube play command includes limited support for Kubernetes networking and storage constructs. It can expose ports and create volumes for certain volume types. To see this in action, create a more realistic Pod definition with a containerPort and a volume:

---
# nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - image: docker.io/nginx:latest
    imagePullPolicy: Always
    name: nginx-pod
    ports:
    - containerPort: 80
      protocol: TCP
    volumeMounts:
    - mountPath: /var/www
      name: nginx-pv
  volumes:
    - name: nginx-pv
      persistentVolumeClaim:
        claimName: nginx-pv

Podman automatically creates a named volume for the volume, but the containerPort requires some special attention. Podman won’t automatically publish this port, but you can force it to publish all defined ports with the --publish-all flag:

$ podman kube play --publish-all nginx-pod.yaml

Podman automatically creates a named volume for the persistentVolumeClaim:

$ podman volume ls
DRIVER      VOLUME NAME
local       nginx-pv

It also publishes the containerPort on the host:

$ podman ps
CONTAINER ID  IMAGE                               COMMAND               CREATED         STATUS         PORTS                    NAMES
72067ec84478  localhost/podman-pause:4.8.3-0                            25 seconds ago  Up 25 seconds  0.0.0.0:80->80/tcp       c1cfbbd00b5c-infra
cc8c69a77c4a  docker.io/library/nginx:latest      nginx -g daemon o...  24 seconds ago  Up 25 seconds  0.0.0.0:80->80/tcp       nginx-pod-nginx-pod

You can verify that the published port works by sending an HTTP request to port 80:

$ curl localhost
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>

Cleaning up Pod definitions with volumes requires additional consideration. Podman won’t delete the named volume by default to avoid any unexpected data loss. However, the --force flag will tell Podman to delete the volume when it tears down the resources:

$ podman kube play --down --force nginx-pod.yaml

Run a Deployment

I’ve covered Pod definitions so far because they’re easy to think about, but production Kubernetes workloads are rarely defined as a single Pod. A true workload typically uses a Deployment resource, and Podman supports both Deployments and DaemonSets.

First, create a Deployment definition for Nginx:

---
# NginxDeployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 6
  selector:
    matchLabels:
      app: nginx
  template:
    spec:
      containers:
        - image: docker.io/nginx:latest
          name: nginx
          ports:
            - containerPort: 80

Next, run podman kube play on the Deployment. Podman doesn’t support multiple replicas, so the replicas field is ignored and automatically set to a single replica:

$ podman kube play NginxDeployment.yaml
WARN[0000] Limiting replica count to 1, more than one replica is not supported by Podman
Pod:
c8e81d7e7b52e6e9779bff9aedad54309883f257182122017c92af63c3e6d525
Container:
fdf7e1c3277ba0ebdc05faf75fd9e46a5f34080397f4539f8ccfd145e453c1c3

That’s it! Podman makes it easy to run Deployment resources that you will likely find in a production Kubernetes environment.

Wrapping Up

The ability to easily move between Kubernetes and local development is important, and it isn’t a problem that is always solved by running an entire Kubernetes cluster on your local machine. Sometimes, it’s preferable to locally run a container using a small runtime, such as Podman.

In this article, you saw how the podman kube play command enables you to run Kubernetes manifests on your local machine. While it doesn’t support every option that you will find in a manifest, it supports enough core features to greatly ease the cognitive load of switching between Kubernetes and a local Podman installation.