- Kubernetes CoreDNS - Mon, Dec 4 2023
- Update container images with Copa - Mon, Nov 27 2023
- Deploying stateful applications with Kubernetes StatefulSets - Wed, Nov 1 2023
Store secrets in AWS Secrets Manager
In my example, we will store database credentials in AWS Secrets Manager. Launch the AWS management console, and follow these steps:
- Search for AWS Secrets Manager, and then click the service name.
- Click the Store a new secret button.
You will now see a page to add a new secret, as shown in the screenshot:
- Do the following:
- Select a secret type. I used Credentials for Amazon RDS database.
- Enter the database credentials.
- Select AWS managed key.
- Select your instance. I chose RDS for SQL server instance.
- Give a descriptive name to your secret, and click Next.
- On the next page, configure secret rotation. This is highly recommended in production environments, but I will keep it disabled for this post and simply click Next.
- Review your settings, and click the Store button to save your database credentials.
- Click the Retrieve secret value button to get the current secret value, as shown in the screenshot below.
You can view the secret value either as a key/value pair or in plaintext. If you have enabled secret rotation, you will see the current secret value.
Store AWS keys in an AWS profile
Once you have your database credentials stored in Secrets Manager, you can use the access keys in the IAM user's AWS profile to access the secret value from Secrets Manager. Later in this guide, I will show you a better way to access secrets without using access keys.
To use the access key method, attach the built-in SecretsManagerReadWrite policy or a custom policy to your IAM user. I have defined a more restrictive policy that allows only the list and read secret value permissions.
You can use PowerShell or the AWS CLI to retrieve a secret using access keys. To install the AWSPowerShell.NetCore module, run the following command:
Install-Module -Name AWSPowerShell.NetCore
Once the PowerShell module is installed, run the following commands to set up your AWS profile for PowerShell:
Set-AWSCredential -AccessKey 'your_access_key' -SecretKey 'your_secret_key' -StoreAs 'TestApp_Profile' Initialize-AWSDefaultConfiguration -ProfileName 'TestApp_Profile' -Region 'ap-southeast-1' Get-AWSCredential -ListProfileDetail
The above commands will store the AWS access keys in %userprofile%\AppData\Local\AWSToolkit\RegisteredAccounts.json using an encrypted format.
If you prefer the AWS CLI, run the following command on a Windows machine using an elevated command prompt:
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi
Now launch a new command prompt and run the aws configure command to set up your AWS profile, as shown in the screenshot below.
On Windows, the configuration file for the AWS CLI is %userprofile%\.aws\credentials, which is known as the shared credentials file. It is important to note that the AWS access keys are stored in plaintext in the shared credentials file.
Access secrets in Secrets Manager using AWS access keys
Now that you have saved your AWS access keys in a profile, you can use PowerShell or the AWS CLI (depending upon which profile you configured in the previous step).
To retrieve the database credentials from Secrets Manager with PowerShell, run the following command:
Get-SECSecretValue -SecretId 'dev/SqlServer/TestApp' -Region ap-southeast-1
Don't forget to specify your secret name with the -SecretId parameter in the above command. You can also specify your region with the -Region parameter. The screenshot shows that secret string value contains all the information needed to connect to your SQL server database in JSON format. You can deserialize this secret string and use it in an application.
If you are working with the AWS CLI, you can retrieve the same secret string with this command:
aws secretsmanager get-secret-value --secret-id dev/SqlServer/TestApp --region ap-southeast-1
Your secret is now safely stored in the AWS Secrets Manager. However, you are still using AWS access keys to retrieve secret values, so there is still a risk that your access keys get compromised. Read on to work around this problem.
Access secrets without access keys using IAM AssumeRole
Instead of directly using the access keys to retrieve secrets, we will configure an EC2 instance to assume an IAM role to retrieve temporary access keys with permission to access the secrets from Secrets Manager.
To create an IAM role, follow these steps:
- Open Identity and Access Management in the AWS management console.
- Click Roles in the left pane, and then click the Add Role button.
- Under the trusted entity type, select the AWS service option, and under use case, select EC2 instance (or another AWS service, depending on where your application is hosted).
- On the add permissions page, select the SecretsManagerReadWrite policy or the custom policy that we discussed in the previous section. I will choose my custom policy since it uses more restrictive read permissions.
- Now, give a descriptive name to your role and click the Create Role button at the bottom.
- Finally, we need to attach this IAM role to our EC2 instance. Open the EC2 dashboard, navigate to your EC2 instance > Actions > Security > Modify IAM role, as shown in the screenshot below.
Your IAM role is now ready with suitable permissions to read secrets from Secrets Manager. Let's try to access our database credentials again. This time, we run the command on our EC2 instance, which assumes the IAM role we attached previously.
Get-AWSCredential -ListProfileDetail Get-SECSecret -SecretId dev/SqlServer/TestApp -Region ap-southeast-1
Similarly, you can access the secret value with the AWS CLI.
aws configure list-profiles aws secretsmanager get-secret-value --secret-id dev/SqlServer/TestApp --region ap-southeast-1
Please note if your EC2 instance and Secrets Managers are in different regions, you need to use the -region parameter (--region, in case of AWS CLI) to specify a region. You can see in the screenshots that there are no access keys stored in profiles, but you can still retrieve the database credentials from Secrets Manager with the help of assume role.
If you have configured secret rotation in your AWS account, you might run into an error, as shown below:
Could not connect to the endpoint URL: https://secretsmanager.ap-southeast-1.amazonaws.com/
aws secretsmanager get-secret-value --secret-id dev/SqlServer/TestApp
You will get a similar error in PowerShell:
Get-SECSecretValue : Name resolution failure attempting to reach service in region ap-southeast-1 (as supplied to the -Region parameter or from configured shell default). Unable to connect to the remote server.
Get-SECSecret -SecretId dev/SqlServer/TestApp
This error occurs because for secret rotation to work properly, you must have a VPC endpoint configured for Secrets Manager. But your endpoint isn't allowing traffic from the EC2 instance. To fix this error, all you need to do is modify the security group of the Secrets Manager endpoint to allow requests from the EC2 instance. See the following screenshot for reference:
Your EC2 instance will now be able to reach Secrets Manager.
In this tutorial, you learned how to store and access secrets in AWS Secrets Manager using access keys and IAM AssumeRole. With the help of this AWS service, you can securely store all kinds of security relevant information, such as database passwords or SSH keys.