A_Porcupine - A Blog about technology and other such geekery!

cover image of raspberry Pis lined up

🤖 Setting Up a Raspberry Pi 4 Kubernetes Cluster

March 07, 2020

I’ve been wanting to set up a Raspberry Pi powered Kuberenetes cluster for quite some time and I finally found time to get it working. If you’re anything like me, you often find yourself tearing things apart and re-building them, so to make the chore of re-building the cluster easier, I’ve been logging my efforts in the form of Ansible playbooks. This post is mostly just a write-up of how I went about building the cluster, but you may find it useful if you’re building your own.

At the end of this process you will have a single-plane Kubernetes Cluster running on Raspberry Pis. My Cluster will consist of 4 nodes (1 control-plane node and 3 workers), but you can add more workers.

1. Get The Equipment

This is what I went for:

2. Write an Operating System to the SD Cards

I chose an ARM64 build of Ubuntu Server (19.10 or 20.04 should both work) for my cluster, the primary reason for this is: it’s debian based (I like Debian) and an ARM64 build is available. If you opt for a different operating system, the Ansible scripts used in this post will need to be edited.

You can download images from the Ubuntu Raspberry Pi Download Page.

Write the image to your SD cards, if you don’t know how, Ubuntu provide instructions for applying the images to an SD card on their Thank You Page.

3. Prepare the Raspberry Pi Nodes

Before you can start to run the Ansible playbooks you need to set up a few things.

A. Enable SSH

  1. Insert the SD card into any computer and mount the first partition (named boot)
  2. Create an empty file in the boot partition with the name ssh - This enables remote SSH login

B. Change the default password of the ubuntu user

If you have a display, keyboard and the appropriate cable, connect each Raspberry Pi to the display and connect the Keyboard. Alternatively, find the IP address that the Pi has been assigned (your home router’s web interface / DHCP server logs can help you find this) and connect via SSH.

Log in to each Pi with the username ubuntu and password ubuntu and follow the prompts to change the password, just use something temporary as you’ll be deleting this user in a moment.

C. Configure Fixed IP addresses

  1. Once you have changed the passwords, you will be logged out, log in again

  2. Enter cat /sys/class/net/eth0/address and note down the MAC address of each Pi

    $ cat /sys/class/net/eth0/address
de:ad:be:ef:ca:fe
  3. On your home router or DHCP server, set up fixed addresses for each Raspberry Pi
  4. Restart each Pi to pick up the new IP addresses, you may also need to restart your home router / DHCP server

D. Install Python

One final step before we are ready to start issueing commands to the Raspberry Pis using Ansible is to install Python. This can be Python 2.7 or 3, here we are using Python 3.

  1. Log back in to each Pi

  2. Install Python with the following:

    $ sudo apt update && sudo apt install python3 -y
  3. Log out of each node

4. Prepare Your Host

We now have a little preparation to complete on the device you’ll be running the Ansible playbooks from. These steps apply to Linux and Mac.

A. Create an SSH Key Pair

If you don’t already have an SSH key pair, use ssh-keygen to generate a new one.

Running ssh-keygen without any arguments will generate an RSA key pair, you may want to read the docs and switch to a different key type for additional security. Accept the default location and optionally enter a passphrase.

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa): 
Created directory '/home/user/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/user/.ssh/id_rsa
Your public key has been saved in /home/user/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:abXLRkBBWL8F5HkhPnkvW8xxPpLSB9h7jqRZ/MhvEI8 user@penguin
The key's randomart image is:
+---[RSA 3072]----+
|       +=o+ .    |
|      .. + =o.   |
|        . O.+o. .|
|         + Bo=++ |
|        S +..OO+.|
|       . o .BEO..|
|          +o.+.o |
|         .    .. |
|              .. |
+----[SHA256]-----+

B. Install Ansible

For instructions on installing Ansible, check-out Installing Ansible.

Once installed run Ansible with the version flag to confirm it has been installed correctly:

$ ansible --version
ansible 2.9.4
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.7.6 (default, Jan 19 2020, 22:34:52) [GCC 9.2.1 20200117]

C. Get the Ansible Playbooks and Configure the Hosts file

  1. Clone the ansible-raspberry-pi-kubernetes repository
$ git clone https://github.com/aporcupine/ansible-raspberry-pi-kubernetes
  1. Edit the hosts file with your IP addresses and preferred hostnames. Enter one host under the control-plane section, this will be your ‘master’ node, whilst the others will be ‘workers’.

5. Create Your New User Account

In this step we’ll set up a new user, copy over a public key, and finally remove the ubuntu user.

A. Run user_setup/stage1

This stage will prompt you to enter the username and password for your new user, it will then create that user, and add your public key to it’s authorized keys file.

I’d recommend setting the username to the same as the user that you’re currently logged in to, otherwise you’ll need to add -u followed by the username to all subsequent Ansible calls.

From the directory that you cloned the ansible-raspberry-pi-kubernetes repo into, run the following and when prompted enter the password you set in step 3:

$ ansible-playbook user_setup/stage1.yaml -i hosts \
  --ask-become-pass --ask-pass -u ubuntu

B. Run user_setup/stage2

This step will kill any processes running under the default ubuntu user, and will then delete it.

From the directory that you cloned the ansible-raspberry-pi-kubernetes repo into, run the ansible command below. From now on when prompted for the BECOME password enter the password configured previously.

$ ansible-playbook user_setup/stage2.yaml -i hosts --ask-become-pass

6. Configure Nodes and Install Docker and Kubernetes

In this step we’ll do some node configuration (including setting the hostname) and then install Docker, Kubectl, Kublet and Kubeadm.

From the directory that you cloned the ansible-raspberry-pi-kubernetes repo into, run the following:

$ ansible-playbook configure_hosts/configure_hosts.yaml -i hosts --ask-become-pass

7. Initiate the Cluster

Our last step is to initiate the cluster, this step runs kubeadm init on the control-plane node, installs a network add-on, gets the join command from the control-plane node and runs it on all the other nodes.

In this step you’ll be asked whether you want to scheduled pods on your control-plane node, by default this is disabled, but it is recommended for small clusters.

From the directory that you cloned the ansible-raspberry-pi-kubernetes repo into, run the following:

$ ansible-playbook init_cluster/init_cluster.yaml -i hosts --ask-become-pass

8. Give it a test!

Log into your control-plane node, and run the following:

$ kubectl get nodes
NAME   STATUS   ROLES    AGE    VERSION
cl0    Ready    master   102m   v1.17.3
cl1    Ready    <none>   101m   v1.17.3
cl2    Ready    <none>   101m   v1.17.3
cl3    Ready    <none>   101m   v1.17.3

You should see each each of your nodes ready to go as shown, but it may take a few minutes until all nodes are ready.

Lets run a quick nginx deployment. Whilst logged into the control-plane node run the following to create the nginx.yaml file:

cat  <<EOF >nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 4
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
EOF

Apply the deployment:

$ kubectl apply -f nginx.yaml
deployment.apps/nginx created

Expose the deployment:

$ kubectl expose deployment/nginx --type=NodePort
service/nginx exposed

Check that the pods are running, you should see 4 listed as available:

$ kubectl get deployment/nginx
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   4/4     4            4           35s

Find out the port that nginx has been exposed on, in the example below it is 30910:

$ kubectl get service/nginx
NAME    TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
nginx   NodePort   10.108.201.143   <none>        80:30910/TCP   2m6s

Open a browser and enter the IP address of any of your nodes with the port provided, in our case http://10.42.1.200:30910 works. You should see the nginx welcome page.

Congratulations! You now have a Rapsberry Pi Kubernetes Cluster!

Tom Hanson

Hey, I'm Tom Hanson! I'm a Software Engineer currently building people tools at a UK based fintech! Follow me on Twitter!