Creating Multi-tenant Moodle service on Kubernetes using Operator Pattern
We are currently supporting WisdmLabs in building a hosted Moodle based e-learning platform on Kubernetes. Recently we completed an early prototype of creating multi-tenant Moodle environment on a Kubernetes cluster. Here we present the challenges we faced and the choices we made in implementing it.
Moodle is an open source e-learning platform implemented in PHP. It is installed by deploying it on a web server (Nginx, Apache). It uses relational database for storing its runtime state. Its base functionality can be extended by installing various plugins. These are available in Moodle plugins repository. Each plugin is essentially a zip file and installing it amounts to adding the plugin’s unzipped code at an appropriate sub-directory inside Moodle’s installation directory within the web server.
Here are the primary requirements for the Moodle environment running on Kubernetes in order to meet the goals of the planned hosted e-learning platform.
- Multi-tenancy: A single Kubernetes cluster should be able to host multiple Moodle instances. This requirement stemmed from the fact that creating a new cluster for every Moodle instance is expensive (for both, price and time). Also, with Kubernetes namespaces, it is possible to achieve multi-tenancy within a single cluster.
- Plugin installation: It should be possible to install plugins on a Moodle instance easily throughout its life cycle. For example, one may want to install few plugins as part of Moodle instance creation. Then few other plugins may need to be installed later, on an already running Moodle instance. Additionally, when Moodle version needs to be updated it should be possible to re-install all the plugins that were previously installed on that instance, after the version upgrade.
Key Challenges
- Multi-tenancy:
There are two primary challenges in implementing multi-tenant Moodle setup. First, each Moodle instance needs to be available on a separate port to ensure that traffic to/from different instances is appropriately isolated. The implication of this is, each Moodle instance’s web server configuration file, which defines the listening ports, needs to be updated with the chosen port number. Moreover, the web server process may need to be restarted for the configuration change to take effect. Second, in order for a Moodle instance to be publicly accessible, all of its resources (form actions links, URLs, etc.) need to be created with a publicly accessible endpoint embedded in them. Moodle code defines a configuration parameter (wwwroot) that can be set for this, but this parameter needs to be set before triggering Moodle installation.
- Plugin installation:
The main challenge in plugin installation is that each plugin has a unique sub-directory within the Moodle install directory in which it needs to be installed. When running Moodle in non-container form this process is simple — you download the required plugin’s zip file and inflate it in appropriate sub-directory inside the Moodle install directory. With containerized setup on Kubernetes this becomes tricky as the plugin installation sub-directory inside a running container needs to be accessed from outside of it.
Overall, as part of creating every new Moodle instance we need to perform following actions:
a) allocate a port, b) update web server configuration file for the instance to include the port number, c) start the web server, d) create publicly accessible endpoint for the instance, e) set the wwwroot variable, f) trigger installation, g) once the installation is complete install any plugins. Also, support plugin installation on an already running Moodle instance.
We realized that Kubernetes Operator Pattern will be the best fit here as all the above steps are essentially workflow actions that can be effectively implemented as Moodle instance state reconciliation logic within an Operator. Once such an Operator is available, an admin can use simple kubectl commands to create Moodle instances and manage plugins on them. This also will make it very easy for upper UI layer to interact with Kubernetes using just one interface.
KubePlus Moodle Operator:
Here is the link of this work-in-progress Moodle Operator — the KubePlus Moodle Operator.
The Operator consists of a Moodle Custom Resource and associated Custom Controller. Every Moodle instance requires a Mysql instance. Currently Mysql instance is being created as a separate deployment using standard Kubernetes yaml definition. The plan is to use a Mysql Operator for this in future.
What happens inside the Moodle Operator?
- The Spec of Moodle Custom Resource defines a field to include the list of plugins to be installed on the instance. For each supported plugin, the Operator knows exact location where the plugin needs to be installed on the Moodle install directory.
- When ‘kubectl create/apply’ command is received, the Operator reconciles the state of a Moodle instance by first creating the instance and then installing plugins, or just installing plugins if the instance already exists.
- For creating Moodle instances, the Operator uses our a custom Docker image consisting of Nginx and Moodle. The Nginx’s /etc/nginx/sites-available/default.conf file in this image is defined to listen on a port whose value is injected by the Operator as an environment variable. For each Moodle instance creation, the Operator allocates a port, creates Service (type NodePort) and Ingress objects, constructs the wwwroot using the Ingress path, and injects the selected port number and the wwwroot as environment variables when starting the container. For the configuration changes to Nginx to take effect, it defines Nginx reload action in the PostStart Container lifecycle hook within the Deployment object for that Moodle instance.
You can try the Moodle Operator using our KubePlus Plaform toolkit. The steps to deploy Moodle instances on Minikube are available here. If you run into any issues, please report them here. Our Moodle Operator is still a work-in-progress. We also want to add Moodle instance backup/restore functionality to the Operator.
To achieve the end goal of the planned hosted e-learning platform, Moodle is just one of the required Operators. It didn’t exist in the community as required and hence we ended up developing it ourselves. However, we will use some of the other community-built Operators to compose this end-to-end application platform. As mentioned, we will include a Mysql Operator. We also want to support https endpoints for each Moodle instance. For that we are considering to use either Traefik or cert-manager.
Summary:
In this post we outlined the challenges that we faced in developing multi-tenant Moodle service on Kubernetes. Building the Moodle Operator allowed us to provide significant ease of use for deploying and managing multiple Moodle instances on a cluster.
