Better way to get secrets in serverless world

Most likely if you are actively using serverless technologies, You already implemented such kind of helper to retrieve some secret data from secret Manager at runtime. I thought there could be some pitfalls and area for improvement.

In this example, we will implement a helper to get secret values from AWS Secret Manager

You’re welcome on board to disassemble such kind of helper! We will use typescript. Based on the technique used here, it will be possible to use any programming language (well, almost).

Let’s define the initial “look” of our helper

Already could appear question. Why the hell is it so complicated ? Function returns a function ? Let’s figure it out together. Firstly we need to understand how we will use this helper.

HOF helps us to “create” multiple different function which will return secrets value by different secretId. It looks more accurate than using secretId as separate parameter for out helper. Without HOF it requires to extend logic of helper itself.

So far should not appear new questions. Let’s define main logic of our helper

There is nothing special, we receive SecretString, then we parse it and try to get values by provided keys.

At this point, we could stop it and go for a cup of coffee/tea and rewatch the best episodes of Rick and Morty, but as I said earlier, there is an area for improvement with own pitfalls.

When running a serverless function, it will stay active (a.k.a., hot) as long as you’re running it. Your container stays alive, ready and waiting for execution.

After a period of inactivity, your cloud provider will drop the container, and your function will become inactive, (a.k.a., cold).

source: https://www.serverless.com/blog/keep-your-lambdas-warm

Now we can benefit knowing that fact and don’t make additional calls to AWS Secrets Manager, cause it cost not only time but money as well. A simple Map will help us.

After successfully receiving data from secretManager we put it in map where key is secretId and value is a parsed string(secret values). Thus we avoid repeated calls to secretManager, cause we’r getting value from Map directly(In case if lambda warm of course).

You might thinking this is the end. But with such implementation, you can easily wipe yourself in the foot! Let’s pretend you have a super successful startup. Constant traffic leaves your lambdas constantly warm ! You look in the logs and understand that you need to change one of the secret values, because some API keys are defined wrong for prod env. You open aws console, change the values to the correct ones. Some time passes, but by the logs lambda still works incorrectly. That causes permanent warm and helper use “cached” secret values from Map. How can we fix it ? Correct, add some expiration time for our secret, let’s say that non of the values should not be “cached” in the map more than 10 minutes. Let’s implement it. That is how looks our final implementation of getSecretValues helper

I hope You find it helpful. I’m open for any kind of feedback. Feel free to share with Yours helpers/tricks which might be useful in serverless world. This helper is a part of my serverless-bonk-templet. Here You can find original source of helper.