How to deploy only changed packages in a Lerna Monorepo

Two ways o f handling the deploy phase of a monorepo containing multiple packages

A monorepo is a software development strategy where code for many projects is stored in the same repository, and Lerna is a library that provides tools to manage the packages contained in a Monorepo.

There are a lot of discussions about pros and cons of using Monorepos, in this article I will not discuss that but I will explain how I solved, together with my colleagues, the problem about deploying packages contained in a monorepo.

Reference repository

The lerna base project that I will use as example for this article is available at: https://github.com/mzuccaroli/lerna-deploy-packages, it is a simple Lerna repository generated with the “lerna init” command with three packages generated with the “lerna create” command. For more information about creating a Lerna repo see: https://github.com/lerna/lerna

The problem

If you are working with a complex project, involving microservices, I think that monorepos are great. With monorepo I solved a lot of developing problems that I had but a new couple of problems emerged:

1. After working on a feature, that potentially involves modifications on multiple microservices I want to easily deploy only the modificated packages.
2. When I deploy multiple Lerna packages, it’s quite possible that some packages depends on other ones in the same Monorepo so, I want to control the packages build and deploy priority

Together with my colleagues, we figure out two possible solutions for these problems: the first and easy one is fully based on Lerna features, the second one is quite “old school” and solves the problem via bash scripts.

Solution One: Lerna commands

Lerna itself offers a solution, it’s easy to apply but not so easy to find in documentation and see in working examples.

The only problem about this solution is that you will need lerna installed in your build/deploy machine and the lerna bootstrap command could be quite slow for big projects. If your pipeline system supports launching lerna and npm command that’s the solution for you.

Deploy only changed packages

The only command you need to know is “lerna run” , it simply runs an npm script in each package that contains that script.

So if you put a “deploy” script in the package.json of your packages that performs the deploy you can invoke all the scripts by running:

$ npx lerna run deploy

For running the script only for modified packages just run

$ npx lerna run deploy — since HEAD~1

this will “automagically” detect which packages are changed in the last “~N” commits, if you are using a good branching system like git flow the merge commits will identify exactly the changed packages.

Running this command requires that lerna is available and packages are ready to deploy so you can create a mini script that install lerna and install all the repo dependencies for your pipeline:

Determine packages order

Let’s assume that we have some example packages: package_1 is a microservice that has some coupling with package_2, this “coupling” situation between microservices is not ideal but sometimes happens so we need a solution.

If both packages are modified you must build & deploy package_2 before package_1.

Lerna offers a solution with a little “hack”: simply put package_2 in the devDependencies of package_1: Lerna will automatically detect packages dependencies and build in the right order.

You can try it with the example repo: make some modifications on the packages then run ./build_lerna.sh: you can notice that the deploy script of package_2 is invoked before package_1.

Solution Two: Bash script

If your deploy pipeline is not ready for launching Lerna and npm commands you can rely on the good old bash/git combination. That is quite “old school” and requires the creation and maintenance of little bash scripts but is very portable and has no requirements (except for git). So will not require the installation of lerna in the deploy environment.

Deploy only changed packages

Let’s create a bash script that will check modified packages and launch a build script for each one, a ready to use version of the script is available in the example repo.

the main steps of the script are:

Fetch the list of modified files

Determine which packages are changed according the file list

Launch the build.sh script for each package

The packages scripts

Remember that this solution assumes that every package has a ./build.sh script in the packages/package_name folder. An example that handle different deploy stages according the branch passed as argument to could be:

Determine packages order

This is a little hack that i applied for solving the packages priority issue: simply add an optional “build priority” field in the package.json file, all the packages that have no specified priority will assume a default value (ex. 50) then order packages by priority before run single builds:

package_2/package.json:

...
“build_priority”: 10
...

build_general.sh:

It’s not a very elegant solution but get the job done, let me know if you have any better ideas!

Now you can run your build_general.sh script and the output should be something like that:

Remember that you can download the working example with build scripts and demo packages in the reference repository: https://github.com/mzuccaroli/lerna-deploy-packages.

Note about the article

This article and the reference examples are inspired by real projects that run in production. The code of this article is not an “hello world” demo but real code that supports large projects. It’s not perfect but the two different solutions provided a robust deploy system for the projects for more than a year.