Image by 200 Degrees from Pixabay

How to create Kubernetes YAML files

Developing on Kubernetes

Introduction

This article is intended as a guide for creating Kubernetes manifest files.

YAML stands for YAML ain’t markup language is used to create Kubernetes manifests. According to the definition:

YAML is a human friendly data serialization language for all programming languages.

Indeed reading YAML is rather straightforward, once you realize that indentation is the way to build data structures. It is especially easy if you are familiar with Python, F# or other indentation-aware languages.

Writing YAML, on the other hand, is much less fun. If you are familiar with schema-less REST API development, think about Kubernetes YAML in the same way as JSON payload for an API. There is no schema to validate against and all the knobs are exposed as data in the payload.

It is relatively straightforward to create a simple YAML file, but for production use, many more settings are required. Here is an example of a Kubernetes Deployment manifest. If you know Kubernetes a bit, most of the settings would make sense when you read the file. Now imagine that you need to create a few of those from scratch (ingress, service, config map, secrets…)

Source: Author — Blazor PWA Repo

As a developer, you will need to create several different types of manifests. Here is a list of Kubernetes resources, * indicates the resources typically created/maintained by developers. Please note that this only applies if you have a dedicated Kubernetes administrator who can take care of the rest. The list is abbreviated only to common resources and in production, scenarios will likely include various CRDs (Custom Resource Definitions).

Source: Author — Kubernetes Resources

If you are not confused, you are not paying attention.

Tom Peters

What you will learn

Let’s try to alleviate some of the confusion around writing YAML files and make the process easier and less painful.

After reading this article, you will learn:

  • different ways to create Kubernetes YAML files
  • how to automatically detect and correct errors during the development process
  • what development tools to use to make creating YAML files easier
  • learn about online tools specializing in generating different kinds of YAML files

Prerequisites

If you would like to follow along and get your hands dirty with YAML, you will need:

Please note that the repository uses a devcontainer with all the tools and configuration needed to run the examples build in. Because of this, the image is large, so please modify it accordingly before running the container.

Create vs generate

Initially, you might be tempted to generate as much of the boilerplate as possible. My recommendation is, don’t! Especially if you are new to Kubernetes or YAML, experiment, copy-paste from Kubernetes docs, but don’t use generators on day one.

Once you are familiar with the basics, progressively add tools that will make your life easier. There is good news; you will understand the basics pretty fast.

A good way to know if you are familiar enough with the YAML content of a specific resource if it is getting, well … boring. From here now, you should dive headfirst into the world of generators and helpers to keep your sanity and make your life easier.

#1 YQ

The first tool I want to talk about is yq. Yq is not a Kubernetes specific, it’s rather a “jack of all trades” of YAML. Learning this tool will help you query and manipulate YAML files directly from the command line. It helps with tasks, such as:

  • filtering YAML file for a specific value, for example retrieving an image name from a deployment file

Selecting values from YAML files is useful, but mastering yq will help mostly with bulk operations on multiple files and more complex transformations.

#3 Kubectl

It is easy to get started with generating YAML files for most of the resources with kubectl. You can use “ — dry-run=client -oyaml > yaml_file.yaml” flag on the “kubectl create” or “kubectl run” commands to generate most of the resources.

For example, to generate a YAML file for an nginx pod you can run:

kubectl run nginx — image=nginx — port=8080 — env=env=DEV — labels=app=nginx,owner=user — privileged=false — dry-run=client -oyaml > nginx-pod.yaml

This command will generate the following YAML:

YAML generated with kubectl

The file needs to be cleaned a bit, but it is a good starting point.

Now you could create a deployment using:

kubectl create deployment my-dep — image=nginx — dry-run=client -oyaml > deployment.yaml

and use yq to merge the two files.

This process can get complicated fast, but it’s easy to use shell scripts to automate most of the tasks.

Using the combination of kubectl and yq is great for starting a simple one-off project as well as help automate things in between.

If you are interested in kubectl tips & tricks, I have a growing list of useful commands in this gist.

#3 Docker-compose

Do you have a docker-compose.yaml file in your project? Generating Kubernetes manifests from the docker-compose file is possible using a tool called kompose.

Let’s see this in action. We will use a docker-compose file from the awesome-compose repository.

Here is a sample docker-compose file:

Source: Awesome-Compose repo

Now, let’s generate Kubernetes manifests using kompose:

kompose generates K8s manifests

This command takes the docker-compose as input and outputs generated Kubernetes YAML into the k8s-manifests folder.

kompose generated files

Using kompose is a good option if you already have a docker-compose file. There are often tweaks needed, but it takes you one step closer to having a decent starting point.

#4 VS Code with plugins

VS Code has 2 plugins that help with creating YAML files. Big thanks to Avi Nehama for suggesting it.

Kubernetes Templates

The template enables quick scaffolding of any Kubernetes resource.

Create yaml file, start typing the name of Kubernetes resource and hit TAB to insert a template. Keep cycling with TAB to fill the names in the required fields.

YAML

This extension from Red Hat runs a YAML server in the background and adds context aware smart completion to any Kubernetes resource.

Remember to activate it in the settings and reload VS Code. Add this line to the settings to enable Kubernetes completion on all YAML files.

"yaml.schemas": {
"Kubernetes": "*.yaml"
}

#5 CDK8s

Moving on from command line to programming territory. If you have to author a lot of YAML, but happen to know Python, Typescript, JavaScript, Java or Go, you can harness the power of a programming language to make the process of writing YAML much easier.

Introducing CDK8s

cdk8s is an open-source software development framework for defining Kubernetes applications and reusable abstractions using familiar programming languages and rich object-oriented APIs. cdk8s apps synthesize into standard Kubernetes manifests which can be applied to any Kubernetes cluster.

CDK8s works by exposing Kubernetes resources objects and using an object called constructs to further abstract and automate YAML files creation.

The real power behind this approach is the ability to:

  • create reusable components and abstractions that capture your requirements
  • use native programming language constructs to automate, test and validate the process of creating YAML

#6 NAML

If you happen to know Go and don’t like YAML at all and want to avoid it at all costs, this project might be something for you!

A very interesting approach designed by Kris Nova (Github profile) is a Go-centric tool called naml which works by creating Kubernetes manifests directly in Go and installing them on the cluster via CLI install command.

This tool can produce YAML similarly to CKD8s but works only with Go.

#7 Online Tools

Now and then a frustrated developer will create a simple Web UI with a form to gather input for generating YAML files. Those projects are usually short-lived and not maintained very well, so my advice is to stay away from online Kubernetes YAML generators.

There is one exception, an online editor by cilium specialized in creating Kubernetes resource called Network Policy

Closing Thoughts

We’ve seen various ways of creating Kubernetes YAML files, some of them using simple command-line tools, auto-generating files. Others exposing full programming languages.

This is great, but how do we make sure that our YAML is correct? What are some of the best practices for validating the content of the file? How can those practices be automated?

Stay tuned for the next article if you would like to know answers to those and similar questions.

--

--

Opinions: Multi-cloud is real, Microservices are hard, Kubernetes is the future, CLIs are good. Me: Love jogging with my dog and learning new things.