Architecting Security & Governance Across your AWS Accounts Part 3: Data Security on AWS.
NOTE!: this is Part 3 of a multi-part series. For Part 1, please click here. for part 2, please click here
“Companies spend millions of dollars on firewalls, encryption and secure access devices, and it is money wasted; none of these measures address the weakest link in the security chain.” — Kevin Mitnick, “The World’s Most Famous Hacker”
According to the 2019 Cloud Security Report, the top cloud security concern of cybersecurity professionals is data loss and leakage. This information should not come as a surprise to any of us as there is no shortage of data breaches nowadays. Data has never been this valuable before as major corporations are willing to pay top dollar to obtain data about consumers. Even though I would love to talk about how precious our data is, we need to move on.
Before we start, let’s discuss why we even need to encrypt our data in the cloud (What people in the industry say about this?):
1- Compliance: You need to meet a compliance requirement, i.e., PCI-DSS
2- You’re trying to protect your data from your cloud provider’s other customers
3- You’re trying to protect your data from your cloud provider
4- It’s a security best practice
Let’s translate the above:
~ We need to minimize unauthorized physical access to data
~ We need to reduce unauthorized logical access to sensitive data
~ We need to achieve data integrity, confidentiality, and availability (CIA)
Certainty, we can’t cover all aspects of data security in an article or two as it a colossal information security domain, books were written about this subject. This post is written equip you as a security or cloud engineer with the knowledge necessary to effectively implement data security in AWS.
Aspirations for this post are to answer questions I had a few years ago when I started to work with AWS trying to understand how encryption and data security works. I remember asking myself questions of this kind; “ if I encrypted an EBS volume today and attached to an EC2 instance in a few months. How would this EC2 instance know about encryption keys used to encrypt the EBS volume? What about messaging services? How do they handle encryption? Does CloudHSM interact with other AWS services similar to KMS? Would KMS store my encryption keys? Would it store them in plaintext? Encryption is by no means an easy topic for me to explain. I don’t blame you if you get lost trying to understand it from me. Especially if you are new to the subject by my goal is to simplify it for you.
AWS has lots of storage offerings; this makes it very difficult for customers to understand how each service should be configured to store data securely. Data is either traveling over the wire or resting waiting for a transfer call
Encrypt Everything:
Seriously, encrypt everything! If your business is big enough for you to move to AWS, you understand why you need to encrypt everything. For the most part, all you need to do is to check a box or make an API call to enable encryption. It doesn’t take much to get up and running with encryption on AWS. However, we need to understand how each service talks to your key management system and how each service handles data encryption. With that, let’s start by looking at an essential AWS service in my opening when it comes to data security (AWS KMS)
AWS Key Management System
AWS Key Management Service (KMS) makes it easy for you to create and manage keys and control the use of encryption across a wide range of AWS services and in your applications. AWS KMS is a secure and resilient service that uses hardware security modules that have been validated under FIPS 140–2, or are in the process of being validated, to protect your keys. AWS KMS is integrated with AWS CloudTrail to provide you with logs of all key usage to help meet your regulatory and compliance needs.
What problem is KMS trying to solve?
Let’s say you want to encrypt some data in an EBS volume, and you easily apply AES 256 encryption using OpenSSL. That means you have an encryption key that you need to secure. You double encrypt the encryption key again to be sure. However, eventually, you need the plaintext key to be able to decrypt your encryption key(s) and finally your encrypted data. Where do you store that key? in a thumb drive? You invest in some Hardware Security Module where your key can never exist in plaintext outside of this super secure HSM. Your application needs to scale up and down, generate terabytes of data, and encrypt those. How are you going to use your on-prem HMS to pass this plaintext key securely to your EBS volume? You can’t. Using KMS allows you to talk AWS managed Cloud HSM meaning KMS is nothing more than an interface for you to store those plaintext keys. Your plaintext keys are replicated in thousands of HSMs to achieve high durability as those keys should never be lost. According to the AWS keys store in HMS are the most durable piece of information AWS guarantee for its customers. So using KMS means you comply with federal regulations as your key is stored in FIPS 140–2 complaint service. You can also have your application use the API to talk to your keys securely. Moreover, you will also achieve low latency since your data sits not too far away from your keys!
KMS at work.
Let’s say you have some critical customer data stored in your data center. You want to have two copies of the data stored in AWS, a copy in Amazon S3 and the other copy in an EBS volume.
To encrypt the data in EBS and S3, you’ve decided to use KMS. Let get this done, and we will explain things as we go as I think scenario-based illustrations make sense.
Step One: To use KMS, we need to create a customer master key. What is it and why it called customer master key?
A customer master key (CMK) is a logical representation of the actual cryptographic material that is used to encrypt your data encryption key. Wait, I can’t use this CMK to encrypt my data? No, you use the CMK to encrypt the data key that is used to encrypt your data.
What are the data keys and backing keys? The backing key is the actual cryptographic material doing the encryption for you fancy CMK. Backing keys are stored and secured in AWS approved hardware and use hardware security modules that have been validated under FIPS 140–2. The data key is what KMS use to encrypt and decrypt the data. Data keys are always stored encrypted with the data (as metadata). Wait, KMS does not store the data keys? No, the data keys are stored with the data in the case of S3 each object will have a unique data key, with EBS each volume would have its own data key.
Key Points about CMK:
CMK is called CMK because the “customer” manages some aspects of the key. AWS creates and manages master keys in your behalf as well that’s why there is a distinction:
1- The customer creates the key
2- The customer can rotate the key annually (there is a workaround for more frequent rotation)
3- The customer manages access to the key (who can rotate the key, delete the key, use the key for encryption operations)
4- You storage services and CMK must reside in the same region
5- You can’t export a CMK
6- If you delete the CMK you won’t be able to read data you encrypted using the deleted CMK (Use IAM/SCP to dent CMK deletion!)
7- CMK pricing is per key and usage
8- You can use OpenSSL to create your own cryptographic material (backing key) and then upload it to as you backing key for your CMK
9- Some AWS storage services can generate data keys for you
Let’s go get a new shiny CMK to encrypt our data.
We also need an Alias, CMK alias is nothing more than a pointed to your CMK. When you create a CMK AWS returns a CMK Id that you can reference when making use of the CMK.
Great use for the CMK alias is to leverage it to rotate your CMK keys more frequently.
Let’s say you have a compliance requirement were you need to rotate a KMS CMK key every three months. Since AWS only allows you to rotate the key every 12 months automatically. We can rotate the key by deleting it. However, remember if you delete the key, you won’t be able to read the data encrypted using the deleted key. CMK alias to the rescue. When you create your first CMK key, make sure you also create an alias for it by using the following command:
aws kms create-alias --target-key-id ca52d1a0-af36-4ab6-95b6-567fbf837e17 --alias-name "alias/Part3" --region us-east-1 --profile defaultWhen it’s time to rotate the key create a new CMK and point your alias to the new CMK Id. Keep in mind that the CMK alias should be hardcode in your applications for them to be able to decrypt data. Moving forward, KMS would use the newly created CMK Id/old alias to process exiting data keys.
Let’s create a bucket and upload a file to it and then explain how AWS went about applying encryption to the file.
As you can see, we didn’t have to worry about data key generation all we had to do is to tell our bucket which CMK should be used.
The Envelope Encryption:
Whenever we upload an object to our S3 bucket, a new encryption (data) key is generated by AWS and used to encrypt the object before it’s stored to disk.
Having the object stored encrypted means it would be computationally infeasible for an attacker in possession of the raw object is stored on disk to decrypt it anyone in possession of that key could decrypt it, so the data keys have to be stored securely, and in a relatively useless form, and that cloud be achieved by encrypting the data keys with the CMK.
To decrypt the object, you need to have the unique data key for that object, but even if you have it, it can’t be used to decrypt the data since it’s also been encrypted by the CMK.
To decrypt the data key, or at least have it used on your behalf, you have to have the customer master key or have a trust relationship with an entity that controls the customer master key and can use it on your behalf.
The data keys are tiny in size as KMS only support symmetric encryption where a single key is used to encrypt and decrypt the data. Data keys are only a few tens of bytes each, which is why the CMK isn’t used to encrypt large objects, but just the tiny data keys. When the customer master key is rotated, the object-specific encryption (data) keys are decrypted with the old backing key and re-encrypted with the new backing key. The new data versions are stored, and the old versions are discarded.
Since the data keys are small in size, when we rotate a CMK AWS isn’t taxed heavily. AWS has to only work with the small data keys instead of astronomical objects using the data keys.
Several analogies could be drawn to an apartment complex, where keys are stored in a common lockbox, where changing the lockbox lock would restrict future access to individual apartments by restricting access to the individual apartment keys by restricting access to the common lockbox. The analogy breaks down because physical keys are easily copied, among other reasons… but it’s apparent, in this illustration, that changing the lockbox lock (relatively simple operation) would be unrelated to changing the lock on one or more apartments (a much more substantial operation).
The bottom line, in this scenario, they create a new secret key to encrypt/decrypt each object you upload, then store that secret key in an encrypted form… periodically, and transparently to you, they change their stored representation of that secret key as a preventative measure. ~ Michael — sqlbot Stack Overflow
Deeper Dive:
Let’s simulate the above by playing the role of an S3 bucket request data keys to encrypt and decrypt a text file. Hopefully, this process helps us understand how things work behind the scene.
1- Generating the data keys; this step takes place when we upload an object to s3. AWS generates a plain text key for us to use for encrypting the data and CiphertextBlob that we would use to decrypt the data. The CiphertextBlob is stored with the object as metadata in the case of S3.
We need to keep track of the CiphertextBlob and the plaintext for later operations.
I am saving those in separate files:
CiphertextBlob_base64_datakey.txt
Plaintext_base64_datakey.txt
Let’s decoded the Plaintext_base64_datakey.txt file as KMS expect us to decode base64 encoded files. Finally, let’s encrypt some data using OpenSSL and the decoded plaintext key.
The decoded and base64 plaintext data keys have done their jobs help us encrypt our secret now it’s time for us to get rid of them! We don’t need them anymore. As the encrypted version (CiphertextBlob) has all the required information if we ever needed to decrypt the data.
Reversing the about (No screenshots this time ):
To decrypt the encrypted file using KMS we need to run the following commands
1- Decode the base64 CiphertextBlob_base64_datakey.txt file:
cat CiphertextBlob_base64_datakey.txt | base64 --decode > CiphertextBlob_decoded_datakey.txt2- Ask KMS to decrypt your cipher-blob:
aws kms decrypt --ciphertext-blob CiphertextBlob_decoded_datakey.txt --region us-east-1 --profile defaultThis command should return the required plaintext to decrypt the data. We will first same it in a file and then encoded it and finally run the following command with the encoded file against our encrypted data file as follow:
cat my_super_important_secret.txt | openssl enc -d -aes256 -k plaintext_decode.txtThis command should return “Architecting Security & Governance Across your AWS Accounts” which we saved in our cat my_super_important_secret.txt file
KMS Access Policies:
Accidentally or maliciously deleting a CMK cloud be more dangerous than not using it to encrypt data keys in the first places. Unfortunately, you can’t recover the key or read the encrypted data ever again.
There are two methods for managing the security of KMS keys. You can have your security team create those keys in your AWS security account and then share them with development accounts. This approach reduces the stress that comes with not knowing where those keys are created, managed, etc. My preferred method is to allow developers to create those keys in their dev accounts, which will help you be more granular with key access management. You can always have a service control policy that prevents users from deleting those keys.
This technique also limits the blast radius as only users in that account should be able to access KMS keys. However, you can always use key policies only to allow few individual access to KMS keys. Overall, I like to see keys sit closer to the data as it reduces complexities when developers need to troubleshoot issues around data and encryption keys.
For managing your KMS key policies, I think you should look at AWS documentation as I think it does a great job
Since our KMS discussion above covered lots of ground when it comes to encrypting data at rest. Let’s close the loop by quickly summarizing how AWS can help you with encrypted data traveling over the wire.
AWS Data in Transport Security
This means ensuring data is encrypted and secured when traveling between internal and external endpoints. Data should be encrypted using protocols such as TLS, IPsec, and MACsec with keys that you control.
Network Layer Encryption:
Today, AWS has three different options for customers to encrypt traffic within virtual private clouds. The following screenshot stolen from AWS documentation summarizes those offerings.
Keep in mind that AWS has recently announced that all traffic between endpoints within your VPC is encrypted by default, way to go AWS.
Web Layer Encryption:
AWS has this incredible service called ACM (AWS Certificate Manager) that help you provision and push private and public certificates to your cloud resources. ACM would also manage the rotation of those certifications, and you can also bring you encryption keys similar to KMS.
ACM is trying to solve some of the following two problems:
1- Provisioning TLS certificates for use with public-facing AWS Cloud resources:
~ ALB/NLB
~ Amazon CloudFront
~ Amazon API Gateway
2- Handling the painful part of PKI:
~ Key pair and cets signing request generation and distribution
~ Encryption and secure storage of private keys
~ Managed renewal and development of certs
You can import your own certificates or use ACM when using the following services:
