Boost your API mocking workflow with Ollama and Microcks

The Microcks AI Copilot feature helps API Product Owners and Developers enrich their example dataset. It suggests new request/response pairs or events samples so that you don’t have to type them and can achieve a greater diversity of samples faster.

However, having a large language model (LLM) at hand that is deployed and accessible is not that common yet: it can be expensive and raise many security and privacy concerns when using a Cloud-based one, and it is nontrivial to deploy one locally. And that’s where Ollama is changing the game! Ollama is an open-source app that lets you run, create and share LLMs locally. Moreover, it has an extensive model library that now includes models like Llama2, Mistral and Gemma.

Recently, Ollama announced built-in compatibility with the OpenAI Chat Completions API, making it possible to use more tooling and applications with Ollama locally. This one rings a bell for me, as Microcks already works well with this OpenAI API!

A cute llama juggling with soap bubbles representing API styles managed by Microcks.

It’s time for a test ride: Use both Ollama and Microcks locally on our laptop, and have them boost our workflow to generate samples for API mocks and test suite. Let's see how to use Ollama 🦙 as our AI Copilot 🤖

Preamble: install Ollama

First, start installing Ollama and configuring it with a Mistral model. Go to https://ollama.com/download, choose the binary corresponding to your platform, and go through the installation process. Ollama is a Command-Line Interface tool so you can check it is correctly installed from a terminal session:

$ ollama --version
Warning: could not connect to a running Ollama instance
Warning: client version is 0.1.30

Retrieve the Mistral LLM using the Ollama pull command like below:

$ ollama pull mistral
pulling manifest 
pulling e8a35b5937a5... 100% ▕█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 4.1 GB
pulling 43070e2d4e53... 100% ▕█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏  11 KB
pulling e6836092461f... 100% ▕█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏   42 B
pulling ed11eda7790d... 100% ▕█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏   30 B
pulling f9b1e3196ecf... 100% ▕█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏  483 B
verifying sha256 digest 
writing manifest 
removing any unused layers 
success

It may take some time depending on your internet connection speed. You can check the models present in your local setup:

$ ollama list
NAME           ID           SIZE   MODIFIED       
llama2:latest  78e26419b446 3.8 GB 4 days ago     
mistral:latest 61e88e884507 4.1 GB 19 seconds ago

Finally, launch Ollama in the server mode — that way it will listen on local 11434 port for incoming API queries:

$ ollama serve
time=2024-04-02T17:04:39.661+02:00 level=INFO source=images.go:804 msg="total blobs: 11"
time=2024-04-02T17:04:39.665+02:00 level=INFO source=images.go:811 msg="total unused blobs removed: 0"
time=2024-04-02T17:04:39.666+02:00 level=INFO source=routes.go:1118 msg="Listening on 127.0.0.1:11434 (version 0.1.30)"
time=2024-04-02T17:04:39.671+02:00 level=INFO source=payload_common.go:113 msg="Extracting dynamic libraries to /var/folders/n9/_hd3r9c13sz76379nr9lg3p40000gn/T/ollama1945503273/runners ..."
time=2024-04-02T17:04:39.691+02:00 level=INFO source=payload_common.go:140 msg="Dynamic LLM libraries [metal]"

If you’re facing an “address already in use” error, it may be another previous Ollama instance running. Check your system tray if any, kill this one and relaunch the ollama serve command.

Install and configure Microcks

Now it’s time to install and configure Microcks. As of today, it will require some configuration in the properties files but nothing too complicated 😉 Starting cloning the microcks GitHub repository and moving to the /install/docker-compose folder:

$ git clone https://github.com/microcks/microcks --depth 10                                                                                                            ─╯
Cloning into 'microcks'...
remote: Enumerating objects: 1211, done.
remote: Counting objects: 100% (1211/1211), done.
remote: Compressing objects: 100% (854/854), done.
remote: Total 1211 (delta 412), reused 875 (delta 259), pack-reused 0
Receiving objects: 100% (1211/1211), 1.47 MiB | 14.04 MiB/s, done.
Resolving deltas: 100% (412/412), done.

$ cd microcks/install/docker-compose

We’ll need to edit 2 different properties files to enable AI Copilot and configure Ollama. The first one to edit is /config/application.properties. You’ll need to add this block of lines with the ai-copilot prefix. They can be added anywhere:

ai-copilot.enabled=true
ai-copilot.implementation=openai
ai-copilot.openai.api-key=ollama
ai-copilot.openai.api-url=http://host.docker.internal:11434/
ai-copilot.openai.timeout=90
ai-copilot.openai.maxTokens=10000
ai-copilot.openai.model=mistral

They allow us to tell Microcks that Ollama is an openai compliant implementation, that uses the mistral model and is located on the localhost (host.docker.internal is an alias).

We must also edit the /config/features.properties file to enable the AI Copilot feature in Microcks UI. Just add this new line somewhere in this file:

features.feature.ai-copilot.enabled=true

Now it’s time to launch Microcks and check everything is working! Use the docker-compose command below to launch Microcks and its containers:

$ docker-compose -f docker-compose-devmode.yml up -d                                                                                                                   ─╯
[+] Running 6/6
 ✔ Network docker-compose_default      Created                                                                                                                        0.0s 
 ✔ Container microcks-db               Started                                                                                                                        0.5s 
 ✔ Container microcks-postman-runtime  Started                                                                                                                        0.4s 
 ✔ Container microcks-kafka            Started                                                                                                                        0.5s 
 ✔ Container microcks                  Started                                                                                                                        0.6s 
 ✔ Container microcks-async-minion     Started                                                                                                                        0.8s

Have Mistral generate samples for you!

Start by opening a browser on http://localhost:8080 — this is where the Microcks UI can be reached. Within Microcks, navigate to the Microcks Hub menu entry, explore the samples provided by Microcks and install the Pastry API — 2.0 sample OpenAPI. You can use the Direct import method.

A Go green button should then appear to navigate to the API details. In the API details, you can expand the operation of your choice, click the ellipse icon on the right and choose AI Copilot samples 🪄

After a few seconds of analysis and computing, you should get suggestions for new samples from your Mistral model served by Ollama. This sample should be accurate, relevant, and conform to the OpenAPI schema of our API. So nice!

You now have to give the sample a name, ensure the Keep the sample box is ticked, and you can now add this sample to the ones of your API!

Giving it a try is easy:

$ curl -X GET 'http://localhost:8080/rest/API+Pastry+-+2.0/2.0.0/pastry/Cannoli' -H 'accept: application/json'
{"name":"Cannoli","description":"A tube of fried dough filled with sweet, creamy ricotta cheese and often chocolate chips.","size":"M","price":3.50,"status":"available"}%

🎉 You enriched your mocks and test suite dataset with a new sample! 🎉

The best thing is that you can do exactly the same thing for GraphQL API, gRPC services or Event Driven Architectures defined using AsyncAPI specification!

Wrap-up

Through this simple example, we demonstrated how Microcks + Ollama running the model of your choice is a powerful combo to streamline your workflow of producing relevant and comprehensive samples for your APIs and microservices. Such samples can then be reused by developers to quickly test their stuff in isolation, in QA environments to ease the pain of building representative environments, and during acceptance and end-to-end tests to validate API and services implementation.

These are huge benefits and bring greater consistency to your API development cycles ♻️ We look forward to more integration with Ollama!

Remember that we are an open community, which means that you, too, can jump on board to make Microcks even greater! Come and say hi! on our GitHub discussion or Discord server 👻, send some love through GitHub stars ⭐️ or follow us on Twitter, Mastodon, LinkedIn, and our YouTube channel!