How to use CloudWatch to Monitor Memory and access log

In this article, I will discuss how you can use CloudWatch to monitor your AWS EC2 instance’s RAM and Apache access log. CloudWatch is a monitoring tool provided by AWS and it does provide different metrics by default, which means you don’t need to do any customization for these default metrics monitoring. But if you want to monitor RAM or want to view the apache access log then you need to install any additional tool on your EC2 instance which will then show those data in CloudWatch. But before that, let’s discuss what is CloudWatch.

What is CloudWatch

CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, and visualizes it using automated dashboards so you can get a unified view of your AWS resources, applications, and services that run in AWS and on-premises.

So basically CloudWatch is monitoring tool and in next section I will show how you can use CloudWatch to monitor custom metrics as well.

How to Setup CloudWatch Agent

To monitor custom metrics from your AWS EC2 instance, you need to install a tool called CloudWatch agent. This tool actually sends data to CloudWatch periodically and from this data, CloudWatch shows us a graph of those data.

To begin the process at first create an EC2 instance in AWS by following my other article here. After creating the instance then login to that instance using Putty or Terminal based on your OS. After that, please set up Apache and PHP so that we can monitor the apache access log. I have an article written on that which you can view here.

So far you have completed the Setup of the Apache 2.4 server, which means we continue further of our tutorial. Next, we want to install CloudWatch Agent, which will send data to CLoudWatch on a periodical interval. This example is developed on Amazon Linux 1 operating system. If you are using a different OS then these commands might not work properly.

At first we need to download the CloudWatch agent RPM file from S3 bucket into our EC2 instance.

cd ~
wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

After that, we are going to install the RPM package which will eventually install our CloudWatch Agent.

sudo rpm -U ./amazon-cloudwatch-agent.rpm

After installing the CloudWatch Agent, we need to create a config file. This config file will tell the agent which metrics to monitor and the time interval. So let’s first open the file using VI for editing.

sudo vi /opt/aws/amazon-cloudwatch-agent/bin/config.json

Once you open the file “config.json” in edit mode then please paste the content as given below. This is a tested code block that does keep track of the apache access log and CPU memory_used_percent parameter.

{
    "logs": {
        "logs_collected": {
            "files": {
                "collect_list": [
                    {
                        "file_path": "/var/log/httpd/access_log",
                        "log_group_name": "access_log"
                    }
                ]
            }
        }
    },
    "metrics": {
        "metrics_collected": {
            "mem": {
                "measurement": [
                    "mem_used_percent"
                ],
                "metrics_collection_interval": 30
            } 
        }
    }
}

Once you have pasted that content in the “config.json” file then next, we need to start the CloudWatch Agent by running the command below. This command takes the “config.json” file as a parameter.

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

If you reached so far without any error that means, you have successfully installed the CloudWatch agent on your EC2 instance. Next, wait 5 mins and then go to CloudWatch to view these new matrices.

I have created a Video of this example as well, which you can watch and run these commands exactly one by one.

Conclusion

It’s important to monitor your CPU memory utilization and apache access log. By default, CloudWatch doesn’t monitor these metrics, but using this example now you can easily monitor these two matrices. As a system administrator or DevOps engineer, it’s really important to monitor as many metrics as possible, and using this example with the config parameter, you can now monitor any EC2 instance via CLoudWatch.

Looking for smartwatch? Check my other article related to cheap smartwatches.

Atiqur Rahman

I am MD. Atiqur Rahman graduated from BUET and is an AWS-certified solutions architect. I have successfully achieved 6 certifications from AWS including Cloud Practitioner, Solutions Architect, SysOps Administrator, and Developer Associate. I have more than 8 years of working experience as a DevOps engineer designing complex SAAS applications.

This Post Has 2 Comments

  1. sarath

    how to collect multiple ec2 instance utilized memory details

Leave a Reply to Atiqur Rahman Cancel reply