Monitor your Amazon EC2 instances using CloudWatch Metrics

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers. It provides you with complete control of your computing resources and lets you run on Amazon’s proven computing environment but it’s needed to handling it quite well.

How can you monitor your EC2 instances and with which metrics and what is the best EC2 type for my application?

AWS EC2 Instance Type- Overview

All the EC2 instance types have to come up with various combinations like CPU, Storage, Memory and other networking capacities. It also gives flexibility in selecting the right mix of resources for respective applications. 

  • General purpose instances provide a balance of compute, memory and networking resources, and can be used for a variety of diverse workloads”. 

 

General Purpose

Type

t2 (tiny or turbo)

m5 (medium)

Description

High frequency Intel Xeon processors


CPU Burstable Performance


Lowest-cost general purpose instance type


Optimized for changing workloads

High frequency Intel Xeon processors


Balanced





Optimized for consistent workloads

 

  • Compute optimized instances are ideal for compute bound applications that benefit from high performance processors”
 

Compute Optimized

Type 

c5 (compute)

Description

High ratio of compute to memory


Optimized for compute-intensive workloads


Deliver cost-effective high performance at a low price per compute ratio

 

  • Memory optimized instances are designed to deliver fast performance for workloads that process large data sets in memory”

 

Memory Optimized

Type

r4 (RAM)

x1e (xtreme)

Description

High Frequency Intel Xeon processor


Optimized for memory-intensive applications


Support for Enhanced Networking

High Frequency Intel Xeon processor


Optimized for high-performance databases, memory-intensive databases, and other memory-intensive applications

 

  • Accelerated computing instances use hardware accelerators, or co-processors, to perform functions [...], graphics processing, or data pattern matching more efficiently than is possible in software running on CPUs.”
 

Accelerated Computing

Type

p3 (pictures)

Description

High frequency Intel Xeon and GPU


Optimized for functions, graphics processing, and other GPU uses


Supports NVLink for peer-to-peer GPU communication

 

  • Storage optimized instances are designed for workloads that require high, sequential read and write access to very large data sets on local storage. They are optimized to deliver tens of thousands of low-latency, random I/O operations per second to applications”

 

Storage Optimized

Types

h1 (HDD)

i3 (IOPS)

d2 (dense)

Description

HDD storage


High disk throughput


High frequency Intel Xeon processors v4


Enhanced Networking up to 25Gbps

SSD storage


High Random I/O performance and High Sequential Read throughput


High frequency Intel Xeon processors v4


Enhanced Networking up to 25Gbps

HDD storage


Highest disk throughput


High frequency Intel Xeon processors v3

Enable Available CloudWatch Metrics  

By default, some metrics are unavailable through CloudWatch so we need to enable them.

For Ubuntu EC2, execute the following lines:

More information about the installation on Amazon Linux and Red Hat Enterprise Linux here.

 

Then download, install and configure the monitoring scripts:

Inside the package, we’ll be interested in mon-put-instance-data.pl script. It will collect memory, swap and disk space utilization data on the current system and then send it to CloudWatch. We’ll principally use MemoryUtilization and DiskSpaceUtilization.

Let’s set up a cronjob sending our metrics periodically. Add the following line in your crontab –e

Then access your metrics in CloudWatch with the prefix System/Linux.

Monitor your EC2 with CloudWatch

EC2 instances allow users to increase or decrease resource capacity on demand. It can also be integrated into AWS components and features such as Auto Scaling Group which automatically increase and decrease your EC2 instances number inside your ASG (Auto Scaling Group). It can also be grouped inside clusters and managed by either ECS (Amazon EC2 Container Service) or Kubernetes (EKS). 

EC2 key metrics are generally into three types :

  • CPU
  • Network
  • Disk I/O 

Let’s dig into it...

 

CPU CloudWatch Metrics


For your EC2 instance :

Name

Description

Unit

CloudWatch Metric Access

CPUUtilization

Percentage of allocated EC2 compute units that are currently in use on the instance

Percent

AWS/EC2

 

For your EC2 burstable performance instances :

Name

Description

Units

CloudWatch Metric Access

CPUCreditUsage

( T2 Instances)

Number of CPU credits spent by the instance for CPU utilization

Credits (vCPU-minutes)

AWS/EC2

CPUCreditBalance

( T2 Instances)

Number of earned CPU credits that an instance has accrued since it was launched or started

Credits (vCPU-minutes)

AWS/EC2

CPUSurplusCreditBalance (T2 Unlimited only)

Number of surplus credits that have been spent

Credits (vCPU-minutes)

AWS/EC2

CPUSurplusCreditsCharged (T2 Unlimited only)

Number of spent surplus credits that are not paid down by earned CPU credits.

Credits (vCPU-minutes)

AWS/EC2

 

Network CloudWatch Metrics

 

Name

Description

Units

CloudWatch Namespace

NetworkIn

Number of bytes received on all network interfaces by the instance

Bytes

AWS/EC2

NetworkOut

Number of bytes sent out on all network interfaces by the instance.

Bytes

AWS/EC2

NetworkPacketsIn

Number of packets received on all network interfaces by the instance.

Count

AWS/EC2

NetworkPacketsOut

Number of packets sent out on all network interfaces by the instance.

Count

AWS/EC2

 

Disk I/O CloudWatch Metrics

 

Name

Description

Units

CloudWatch Namespace

DiskReadOps/DiskWriteOps

Completed read/write operations from all instance store volumes available to the instance in a specified period of time

Count

AWS/EC2

DiskReadBytes/DiskWriteBytes

Bytes read from/written to all instance store volumes available to the instance

Bytes

AWS/EC2

 

You also can go deeper by checking what is the best CloudWatch metrics to watch for your EBS, the Amazon Elastic Block Store alongside your EC2.

 

Now that you understand better EC2 types, and which metrics are important to monitor, you may check what is best for your workflow.

Lanvert Matthieu

Lanvert Matthieu

Matthieu is a Site Reliability Engineer (SRE) at Padok. He specializes in DevOps technologies such as AWS, Kubernetes, Docker, Gitlab, and CloudWatch.

What do you think? Leave your comments here !