AWS CDK for EKS — Handling Helm Charts

Published in

ITNEXT

5 min readMay 20, 2021

In a previous post, I briefly covered how to use the AWS Cloud Development Kit (CDK) with Java, to build Amazon EKS clusters and use Kubernetes (k8s) manifests to install resources. In this post I will look at how the AWS CDK handles helm charts.

Helm

According to the helm documentation Helm is:

The package manager for Kubernetes

The documentation goes on to say:

Helm is the best way to find, share, and use software built for Kubernetes.

Helm is a mature application and is a graduated project from the Cloud Native Computing Foundation (CNCF). With helm charts, k8s users can easily create, version, share, and publish software that runs on k8s.

Charts

Helm packages are known as charts.

Helm uses a packaging format called charts. A chart is a collection of files that describe a related set of Kubernetes resources. A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on.

The cool thing about charts is that they can be used to install resources across the k8s cluster; they are not limited to a single namespace. And, the resources are all related to the scope and version of the package being installed.

Though they include other parts, charts are primarily composed of templates files and values files. The following example shows a template file for a k8s service.

kind: Service
apiVersion: v1
metadata:
  name: {{ .Values.service.name }}
  namespace: {{ .Values.service.namespace }}
  labels: {{ toYaml .Values.service.labels | nindent 4 }}
  {{- if .Values.service.annotations }}
  annotations:
    {{- toYaml .Values.service.annotations | nindent 4 }}
  {{- end }}
spec:
  ports:
    - port: {{ .Values.service.port }}
      targetPort: {{ .Values.service.targetPort }}
      protocol: TCP
      name: http
  type: {{ .Values.service.type }}
  selector: {{ toYaml .Values.service.selector | nindent 4 }}

The idea behind templates and values is that configurations are both uniform and flexible. The base config is supplied in the template; this reduces the undifferentiated heavy-lifting between instances of the chart. The values file provides the point-of-customization; users can supply their own configuration elements to customize the packages installed to clusters. This also follows the configuration practices of 12 Factor App Methodology. The following listing is the portion of the Values file that allows users to customize the service installed by the chart.

service:
  name: readonly
  namespace: readonly
  type: LoadBalancer
  port: 80
  targetPort: 8080
  labels: {owner: "jimmy", env: "dev", app: "readonly"}
  selector: {app: "readonly"}
  annotations: {service.beta.kubernetes.io/load-balancer-source-ranges: "10.0.0.1/32,192.168.0.1/32"}

The preceding YAML file uses flow-styles made possible with explicit indicators; this makes the YAML less dependent on whitespace and indentations. To consume this markup, the toYaml function is used in the template.

labels: {{ toYaml .Values.service.labels | nindent 4 }}

The toYaml function converts the format to YAML and the result is piped to the nindent function to indent the elements 4 spaces, after a newline. As seen in following snippet, created by the helm template readonly-go-http command, the YAML output is composed of the templates and the values.

kind: Service
apiVersion: v1
metadata:
  name: readonly
  namespace: readonly
  labels:
    app: readonly
    env: dev
    owner: jimmy
  annotations:
    service.beta.kubernetes.io/load-balancer-source-ranges: 10.0.0.1/32,192.168.0.1/32
spec:
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
      name: http
  type: LoadBalancer
  selector:
    app: readonly

The helm template command is very handy to verify the output of helm chart templates and values. This output can be applied directly to a Kubernetes cluster using the kubectl apply command.

K8s users usually install charts via the helm install command, like the following example.

helm install readonly jimmy-charts/readonly-go-http --debug -f readonly-go-http/values.yaml

In the previous example, readonly is the name of the k8s package installed. This is what the package will be known as on the cluster. The jimmy-charts/readonly-go-http argument is the helm repository and chart that is being used for the install instance. The — debug argument is usually not used during normal operation, but it’s handy when charts don’t install properly. The -f readonly-go-http/values.yaml is the file where custom values are supplied by the k8s user.

Helm Charts with AWS CDK

Using helm with the AWS CDK is similar to how k8s manifests are handled. The following example uses the HelmChart.Builder object to construct the helm chart entry for the ensuing AWS CloudFormation template produced by the cdk synth and cdk deploy commands.

HelmChart.Builder.create(this, "readonly")
                .cluster(cluster)
                .chart("readonly-go-http")
                .repository("https://git-helm.jimmyray.io")
                .values(YamlParser.parse(readonlyValues))
                .createNamespace(false)
                .version("0.1.1")
                .build();

The cluster argument points to the cluster object into which the helm chart is installed. The chart argument is the name of the helm chart that is stored in the helm chart repository: https://git-helm.jimmyray.io . The values argument is a parsed string of YAML values. Finally, the version argument points to the helm chart version.

Values

The final Values class stores the values YAML files in final Java text-block strings. As in the previous post, this YAML file could be stored in an external file and read in with a Java FileReader or InputStream.

public final class Values {
    public static final String readonlyValues = """
            namespace:
              name: readonly
              labels: {owner: "jimmy",env: "dev",app: "readonly"}
                        
            service:
              name: readonly
              namespace: readonly
              type: LoadBalancer
              port: 80
              targetPort: 8080
              labels: {owner: "jimmy", env: "dev", app: "readonly"}
              selector: {app: "readonly"}
              annotations: {service.beta.kubernetes.io/load-balancer-source-ranges: "10.0.0.1/32,192.168.0.1/32"}
...
             """;
...

The YamlParse.parse(…) method parses the values YAML and supplies it to the stack as an AWS CloudFormation Custom Resource. The following snippet is from the cdk synth command output.

readonly40AF6D61:
    Type: Custom::AWSCDK-EKS-HelmChart
...
      Chart: readonly-go-http
      Version: 0.1.1
      Values: '{"namespace":{"name":"readonly","labels":{"owner":"jimmy","env":"dev","app":"readonly"}},"service":{"name":"readonly","namespace":"readonly","type":"LoadBalancer","port":80,"targetPort":8080,"labels":{"owner":"jimmy","env":"dev","app":"readonly"},"selector":{"app":"readonly"},"annotations":{"service.beta.kubernetes.io/load-balancer-source-ranges":"10.0.0.1/32,192.168.0.1/32"}},"deployment":{"name":"readonly","namespace":"readonly","labels":{"owner":"jimmy","env":"dev","app":"readonly"},"matchLabels":{"app":"readonly"},"revisionHistoryLimit":3,"podSecurityContext":{"fsGroup":2000},"replicaCount":3,"podTemplateLabels":{"owner":"jimmy","env":"dev","app":"readonly"},"containerName":"readonly","image":{"repository":"public.ecr.aws/r2l1x4g2/go-http-server","pullPolicy":"IfNotPresent","tag":"v0.1.0-23ffe0a715"},"securityContext":{"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsNonRoot":true,"runAsUser":1000,"allowPrivilegeEscalation":false},"resources":{"limits":{"cpu":"200m","memory":"20Mi"},"requests":{"cpu":"100m","memory":"10Mi"}},"readinessProbe":{"tcpSocket":{"port":8080},"initialDelaySeconds":5,"periodSeconds":10},"livenessProbe":{"tcpSocket":{"port":8080},"initialDelaySeconds":15,"periodSeconds":20},"ports":[{"containerPort":8080}]}}'
...

Like the AWS CDK handling of k8s manifests, the helm charts are created during the deployment process, started by cdk deploy, using the AWS CDK assets. As a reminder, these assets are installed in the CDKToolkit stack during the cdk bootstrap operation. The CDkToolkit provides the helm handler to make the helm calls to the target cluster via AWS Lambda.

The installed helm chart includes the readonly package, including the Namespace, Deployment, and Service resources, as well as the ReplicaSet resource implicitly created by the applied Deployment. Finally, there is an AWS Classic Load Balancer (ELB) created by the Kubernetes AWS Cloud Provider.

k get all -n readonly

The ELB is outfitted with with an AWS security group, with inbound rules matching the CIDR ranges supplied in the service.beta.kubernetes.io/load-balancer-source-ranges annotation.

Summary

The AWS CDK handles helm charts and creates AWS Cloud Formation resources to apply packages as Kubernetes resources. The application of these packages are handled by AWS Lambda assets installed by the AWS CDK bootstrap operation. With the AWS CDK, k8s developers can build in the cloud using tools (like Java) with which they are already familiar. This openness to multiple languages improves adoption and eases the barrier to entry for cloud developers.

The example code for this post can be had from this GitHub repo.

The helm chart can be found in this GitHub repo.