Kubernetes over Ubuntu on VirtualBox

Kubernetes is a main stream. Lets create cluster for development on local environment

Click here to share this article on LinkedIn »

Environment:

• VirtualBox 5.2.6 on ArchLinux
• Ubuntu 16.04 on each VM.
• Kubernetes 1.9.3

Minimum requirements:

• 2CPU
• 2GB of ram
• disabled SWAP
• 20GB of HDD (just for test)

VM has two network interfaces. One behind NAT Network for internet connections, second Host-only for inter communication between VMs. Host-only network should be different then kubernetes virtual network. I choose 172.30.56.0/24. Kubernetes will have 3 nodes:

• kube-master (management, monitoring)
• kube-worker-1
• kube-worker-2

worker-1 and worker-2 will be used for all users pods.

For build kubernetes cluster I use kubeadm. All instruction how to setup kubeadm, kubectl described here.

Lets start:

1. install docker environment.
2. install kubelet, kubeadm, kubectl.
3. change cgroup-driver for kubelet and docker

For Docker:

cat << EOF > /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

For kubelet:

#edit kubeadm.service file
> vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# add to the end --cgroup-driver=systemd
> systemctl daemon-reload && systemctl restart kubelet docker

4. initializing master

> kubeadm init --apiserver-advertise-address=172.30.56.120 --pod-network-cidr=192.168.0.0/16

— apiserver-advertise-address=<IP adress> — change standard API IP adress.

— pod-network-cidr=192.168.0.0/16 — needed for bootstraping calico network.

5. copy kube configuration to home folder. Needed for management cluster through kubectl. Also you can copy this configuration to workers or to local machine if need manage cluster on different host than master.

> mkdir -p $HOME/.kube
> sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
> sudo chown $(id -u):$(id -g) $HOME/.kube/config

6. install virtual network. I choose Calico.

> kubectl apply -f https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/kubeadm/1.7/calico.yaml

7. Don’t forget copy token hash. It will be needed for adding workers to the cluster and login to dashboard. Token will be available 24 hours.

Don’t worry if token will expire kubeadm token create and kubeadm token list will help create new tokens for join nodes and login to dashboard.

Each worker should have docker software and kubeadm installed. For join node to cluster execute next command on each of VMs.

> kubeadm join --token 6a1c63.4d6b405ebfba2021 172.30.56.120:6443 --discovery-token-ca-cert-hash sha256:8fac8a430850de77f17e7eab0bdfbcd59107bf8f60edd3538e098f46e0d2287b

Show cluster members (on master):

> kubectl get nodes

8. By default master is not handle users pods but if cluster is small master can be place for pods.

> kubectl taint nodes --all node-role.kubernetes.io/master-

9. cluster is created and ready for use. Just start simple pod with linux inside.

> kubectl run -i --tty busybox --image=busybox --restart=Never -- sh

busybox — it is simple linux with minimum software inside. Good for creating containers from scratch.

Now kubernetes cluster is ready for pods. kubectl is main command for managing cluster.

Dashboard installation

Dashboard is not secure way for manage kubernetes cluster. Dashboard executing with full admin right. Be careful with sharing access to dashboard. Users can rape your cluster.

1. install dashboard.

> kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

Dashboard - UI with management possibilities. For metrics collection need install Heapster.

2. install heapster with influxDB for metric storage.

> git clone https://github.com/kubernetes/heapster.git
> cd heapster
> kubectl create -f deploy/kube-config/influxdb/
> kubectl create -f deploy/kube-config/rbac/heapster-rbac.yaml

3. after all pods successfully started start kubectl proxy for open dashboard web application.

> kubectl proxy

kube proxy start listen 8001 port on localhost. Put next line to browser.

http://localhost:8001/ui

kebernetes give you a logging page with two options: login with kubeconfig or with token. The easiest way use token. kubeadm token list will show all created tokens. Choose one of them to login to application.

Tip and tricks

• kube-dns doesn’t start if VM has less than 2 CPU.
• Sometimes after login dashboard return security error:

User "system:bootstrap:883ae1" cannot list configmaps in the namespace "default"

Just add user to cluster admin group:

> kubectl create clusterrolebinding --user system:bootstrap:883ae1 kube-system-cluster-admin --clusterrole cluster-admin

• describe system pods:

kubectl describe pod kube-apiserver-kube-master -n kube-system

• show logs from pod:

kubectl logs kube-dns-6f4fd4bdf-kr8wh -n kube-system

Enjoy.