- EC2 Image Builder: Build your golden VM images on AWS - Wed, Jan 19 2022
- Configuring DFS Namespaces for Amazon FSx for Windows file servers - Fri, Jan 7 2022
- AWS Systems Manager Session Manager: Securely connect EC2 instances - Wed, Dec 22 2021
Prerequisites
Before we get started with CloudWatch, you have to follow these steps in AWS:
- Create an EC2 instance.
- Ensure that the EC2 instance can reach the SSM endpoint either via public internet or a VPC Endpoint.
- Attach an IAM role to the EC2 instance that includes the following policies:
- CloudWatchAgentServerPolicy: This policy enables the EC2 instance to push the logs and metrics to the Amazon CloudWatch service.
- AmazonSSMManagedInstanceCore: This policy enables the EC2 instance to read parameters stored in the SSM parameter store and to have them registered under the SSM managed instances, so you can Run Commands against it.
- Ensure that the SSM agent is installed in this EC2 instance.
Install the Amazon CloudWatch Agent on an EC2 instance
Navigate to the AWS Systems Manager console.
On the left menu, select Node Management > Run Command. Then click Run command.
RunCommand in Systems Manager Console
In the Command document search bar, choose Document prefix name, then Equals, and then specify the name of the document as AWS-ConfigureAWSPackage.
Search for the command document
Then specify the following command parameters:
- Action: Install
- Installation Type: Uninstall and Reinstall
- Name: AmazonCloudWatchAgent
- Version: Latest
- Additional Arguments: No additional arguments needed
Specify command parameters
Specify command parameters
Under the Targets section, specify the method for selecting the instances in which you want to install the CloudWatch agent. There are three methods:
- Instance Tags: If the instances you're using are tagged, you can specify one or more instance tag key–value pairs to identify the instances on which the tasks will run.
- Manually: Manually select the instances on which the tasks will run.
- Resource Group: If the instances are gathered under, you can specify the resource group on which the tasks will run.
Specify the targets
If you don't want to store the outputs of this command in an S3 document, you can just uncheck Enable an S3 bucket under Output options.
Finally, click Run.
Once you click Run, you are redirected to a new page where you can monitor the status of the running tasks.
Status of the running tasks
Using this method for installing the CloudWatch agent is highly recommended for instances on scale, where you can install the agent on all the instances at once. The command document we used to install the CloudWatch agent is suitable for Linux, Windows, and MacOS. So you don't have to worry about segregating the different platforms while provisioning the agent.
Configure the CloudWatch Agent
Once the agent is installed, the next step is to configure it to push the logs and metrics to CloudWatch.
To configure the CloudWatch agent, you need to create a configuration file. You can create it by running the CloudWatch Agent Configuration Wizard, which you can start by entering the following command:
Linux
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
Windows:
cd "C:\\Program Files\\Amazon\\AmazonCloudWatchAgent" amazon-cloudwatch-agent-config-wizard.exe
All you need to do is answer the wizard questions to generate a customized configuration file for the server you would like to monitor. Otherwise, you can create the configuration file manually by following this guide: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html
It's recommended to store the configuration file as an SSM parameter store in AWS Systems Manager because it provides version control for your stored parameter; therefore, if you modified the value of the parameter, it would create a new version of the parameter while retaining a history of the previous versions.
If you're following the wizard to generate the configuration file, you have the option to store it on SSM. Just ensure you answer Yes when prompted whether to store the file in Systems Manager Parameter Store. However, if you're creating it manually, you can just store it right away as a parameter in the Systems Manager Parameter Store.
If you're creating the configuration file manually, you can store it as a parameter in SSM by following these steps:
- Navigate to AWS Systems Manager.
- Under the Application Management section, choose Parameter Store.
- On the Parameter Store console, click Create Parameter.
- Specify a Name for the parameter, copy the content of the configuration file, paste in a Value, and you can keep the rest of the options as defaults.
If you use the wizard to write the configuration file to the SSM parameter store, ensure that the IAM role attached to the EC2 instance includes CloudWatchAgentAdminPolicy instead of CloudWatchAgentServerPolicy.
If you're storing the configuration file locally, the configuration file config.json is stored in /opt/aws/amazon-cloudwatch-agent/bin/ on Linux servers and in C:\Program Files\Amazon\AmazonCloudWatchAgent on Windows Servers.
Once the agent is created, you can use it to configure the CloudWatch agent. You can do so by either running this command on the instance or using the Run Command feature of AWS Systems Manager.
To run it locally, use the following commands:
Linux:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:<Parameter\_Name\_in\_SSM>
Windows:
& 'C:\\Program Files\\Amazon\\AmazonCloudWatchAgent\\amazon-cloudwatch-agent-ctl.ps1' -a fetch-config -m ec2 -s -c ssm:<Parameter\_Name\_in\_SSM>
If you're storing the configuration file locally, then you need to replace ssm with file in the command and specify the file path, e.g., file:'C:\Program Files\Amazon\AmazonCloudWatchAgent\config.json'
To do this with Run Command:
Navigate to Run Command the same way we did earlier.
In the Command document search bar, choose Document prefix name, then Equals, and then specify the name of the document AmazonCloudWatch-ManageAgent.
image6
Search for the CloudWatch-Manage Agent command document
Then specify the following command parameters:
- Action: configure
- Mode: ec2
- Optional Configuration Source: ssm
- Optional Configuration Location: The name of the parameter that stores the CloudWatch agent configuration
- Optional Open Telemetry Collector Configuration Source: default
The rest of the sections are the same as in the command document we ran earlier.
image7
Configure CloudWatch-ManageAgent Command Parameters
Finally, you can click Run and wait until it configures the CloudWatch agent on the instances you specified.
As of CloudWatch agent version 1.247347.3, there's an embedded AWS OpenTelemetry Collector that enables you to monitor with open telemetry without needing multiple agents. If you need to configure the Open Telemetry Collector, you can change its command parameter to SSM and specify the parameter that stores this configuration in the Optional Open Telemetry Collector Configuration Location field.
Verify Amazon CloudWatch Agent Configuration
Once configured, you can verify whether the configurations specified in the configuration file have taken place successfully by doing the following:
- Logs: Navigate to the CloudWatch console and select Logs > Log groups. Check whether the log groups you specified in the configuration file are there, with streams inside them and logs pushed to these streams.
- Metrics: Navigate to the CloudWatch console and select Metrics > All Metrics. You should see it under Custom Namespaces. Open this namespace and see if the metrics specified in the configuration file are being pushed.
Conclusion
In this article, we've gone through the installation and configuration of the Amazon CloudWatch agent. If you have any further questions, please mention them in the comments.
Subscribe to 4sysops newsletter!
DISCLAIMER: This article represents my own viewpoints, not those of my employer, Amazon Web Services.
