Automation-friendly Software Systems and How to Build Them
Tips on designing your software systems toward better automations
Why should I care?
Mere mention of “automation” makes us think of DevOps. After building an app or a service, we must manage its life cycle. Environment setup, provision, maintenance, upgrade, and decommission naturally come into play. Designing your app to better support the automation of that life cycle makes DevOps and your life easier.
There is, however, a more important angle to consider when it comes to automation-friendliness: making your customers happy. In the context of an SaaS or PaaS, your product will be an integral building block of customers’ own larger systems. Said customers will therefore need to manage the life cycles of your product, preferably in automated fashion. Even if you think you can get away without automation for yourself, for your customers it’s a different story.
This article offers guidelines for designing software systems on which you and your customers can build good automation.
Defining good automation?
Not all automation is created equal. The best automation, at its core, must be easy to understand and easy to fix. From the perspective of automation users, it can be a simple matter of tweaking configurations, modifying snippets of code in a standard CI/CD system (e.g., Travis or Jenkins), or working with an in-house admin app/dashboard. By having more code (toward Turing completeness), you gain flexibility at the expense of making automation more complicated.
Your automation should favor the config end of the spectrum, because SREs, support engineers, or new developers filling in for you while you’re on vacation are more likely to be users than the original developers themselves. The more code you have, the steeper the learning curve is, thus increasing the possibility of error.
Failures are inevitable and interrupt your automation with network breaks, incorrect credentials, or mere changes of mind. Under ideal circumstances, your automation should be able to fix these undesirable setbacks with ease. When good automation fails, the fix should be as simple as “re-run until it succeeds.” Bad automation, on the other hand, generates a tangle of temporary states that must be manually cleaned up before retrying.
Now that we have a viable automation metric, we can move on to the topic of design.
Let there be APIs
Any automation-friendly system should have APIs — that is, ways of manipulating the state of your app programmatically. The golden rule: Any action performable by a human should map onto an API.
Declarative APIs with reconciliation
Though having APIs is the bare minimum required for automation, they don’t necessarily make automation easier. Declarative APIs with reconciliation, however, take your automation to the next level.
In short, declarative APIs focus on the “what” (desired states) instead of the “how” (changes between states). With declarative APIs, users are dictators who only tell your system the desired states it should be in, as opposed to imperative APIs, through which users function as orchestrators of your system. While both may achieve the same results, their differences in focus impact user experiences of automation.
Let’s say, for instance, your system provides backup capabilities. A declarative API will allow users to designate the desired state of the backup, for example, there exists a backup file for every N-day period. Your system takes note of that and keeps ensuring that it happens (reconciliation). An imperative API will allow users to create a backup job and have it run every N days. In practice, systems like Puppet and Kubernetes are exemplars of the declarative design paradigm.
Declarative APIs are preferable because they help simplify automation and translate naturally into configurations of desired states. This reduces the amount of code you need to write and turns the question of automation into a more straightforward “config file management” problem. Plenty of effective solutions in this regard, such as Git and GitOps, are available. Moreover, declarative APIs relieve failure by relegating errors to your system’s reconciliation logic, which is better equipped to handle them. Automation must ensure the desired states are received by your system and watch how it transits into those states. Lastly, declarative APIs are born to be idempotent and re-run safe. With imperative APIs you might worry about re-running certain automation tasks, such as requesting new resources and generating duplicates, whereas declarative APIs provide your system with a list of what’s needed. You can keep sending the same list and the system will make sure everything on that list is fulfilled instead of creating duplicate lists.
Fewer orders
While declarative APIs leave your system well-primed for automation, there is more you can do.
Most people who shy away from declarative APIs cite their inability to handle complicated workflows requiring sequences of steps that must happen in a certain order.
Let’s use a resource creation scenario as an example. Say we have two APIs, one for provisioning storage and the other for creating credentials. A user must first call the storage API, wait for the API to interact with the underlying infrastructure, and get an ID back. The user then needs to pass that ID to the credential API to create credentials for new storage. It’s hard to express such operational logic with declarative APIs and configurations due to their lack of imperative order and steps. You can “cheat” by letting people manually configure the steps they want, but that just makes your APIs imperative. It’s akin to inventing your own mini-language and letting people write snippets of code with it. The resulting solution will have the same pros and cons as any other imperative systems.
Orders are the Achilles’ heel of declarative APIs. As a developer you can help by removing the need for orders. The following are two tricks you can try:
Move orders into your system
This can be done by wrapping imperative logics and merging a sequence of API calls into a higher-level “cover-all” API. In our example, your user desires to create storage and a credential for that storage. By packing everything into a “give me storage with a credential” API, your user only needs to call it once and let your system handle the specific steps required to return the final results.
Remove dependencies among steps
When two steps no longer depend on each other, orders become unnecessary. One way to achieve this is by making your APIs more deterministic. Steps depend on each other because they lack certain information from other steps to proceed. Looking back on our previous example, credential creation must wait for provisioning out of need for a storage ID, not the readiness of the new storage. If we have a seer who tells the credential API the future-storage-ID-to-be, it can create a credential entry (ID, username, password) right away, or start a reconciliation loop to create it once the storage is ready. It’s true that users will not be able to use the credential until the storage is ready, but it’s effectively the same result as the imperative solution. The only difference is that, in this case, users provision, get a credential, then wait, while in the imperative solution users provision and wait before getting a credential.
So where do you find a seer? If your API results are deterministic, you don’t need to. We typically need a seer to tell the future storage ID because it’s some random value generated by our system at the time of provisioning. It doesn’t need to be that way because you can let your user pass in an ID for your system to use. You don’t need to worry too much about user-generated IDs not being unique enough, since there are plenty of ways to help uniqueness such as namespacing (e.g., setting ID as {username}/{user-gen-id}) or just checking and rejecting duplicates (e.g., twitter username).
In summary, deterministic APIs translate running a sequence of imperative steps into sending declarative configurations once before waiting for all pieces to converge into desired states with the help of reconciliation loops.
Bonus: create a CLI
Declarative or not, CLI (command-line interface) is a great catalyst for automation. It allows people to automate your system with a few lines of script instead of creating a whole coding project — at least in the beginning. It’s a common practice these days and all major IaaS, PaaS, and SaaS services such as AWS, Azure, and IBM Cloud make use of one. Those interested can check out such frameworks as https://github.com/urfave/cli and https://github.com/pallets/click to save you a lot of hassle while building beautiful yet professional CLIs.
Header image credit: Pexels/Pixabay
If you enjoyed this article, follow me on Medium! I write about distributed systems and software architecture, such as:
- Minimalist Software Architecture. Lessons learned from building large-scale multi-region distributed systems
- Before you use Kustomize. The pros and cons of the popular Kubernetes config management tool Kustomize
