In this blog, I will try to cover the following topics and try to explain more about AWS Elastic Container Service which is a highly scalable, fast and high-performance container management service.
- Why Docker Containers?
- ECS Cluster Management
- EC2 Container Registry
- ECS Services
- Auto-Scaling in ECS
- Monitoring, Logging and Notification
Why Docker Containers?
- Lightweight, Open Source and Secure
- Portable and efficient in comparison to VM
- Empower Developer creativity
- Eliminates Environmental Inconsistencies
- Ability to scale quickly
- Reduces time to market of your application
Services evolve to microservices
Why Container Cluster Management System is needed?
- Provides clustering layer for controlling the deployment of your containers onto the underlying hosts
- Manages container lifecycle within the cluster
- Scheduling Containers across the cluster
- Scaling containers
What is AWS ECS (EC2 Container Service)?
- Amazon EC2 Container Service (ECS) is a highly scalable, fast and high performance container management service.
- Easily run, stop and manage Docker containers on cluster of Amazon EC2 instances.
- Schedules the placement of Docker containers across your cluster based on resource needs, availability and requirements.
Components of ECS
- Cluster – Logical group of container instances
- Container Instance – EC2 instance in which ECS agents runs and is registered to cluster.
- Task Definition – Description of application to be deployed
- Task – An instantiation of task definition running on container instance
- Service – Runs and maintains predefined tasks simultaneously
- Container – Docker Container created during task instantiation
ECS Architecture Overview
Key Components of ECS Architecture
Agent Communication Service – Gateway between ECS agents and ECS backend cluster management engine
API – Provides cluster state information
Cluster Management Engine – Provides cluster coordination and state management
Key/Value Store – It is used to store cluster state information
ECS Agent –
- It runs on EC2(Container) instances
- ECS cluster is collection of EC2(Container) Instances
- ECS agent is installed on each of EC2(Container) Instances
- ECS agent registers instance to centralised ECS service
- ECS agent handles incoming requests for container deployment
- ECS agent handles the lifecycle of container
EC2 Container Registry (Amazon ECR)
- It is an AWS managed Docker container registry Service.
- Stores and Manages Docker Images
- Hosts images in a highly available and scalable architecture
- It is integrated with ECS.
- No upfront fee, cheap and pay only for the data stored.
Creating ECS Cluster
Cluster can be created using
- AWS Console (Manual method)
- AWS ECS CLI (Manual method)
- Cloud Formation Template (IAC and Recommended method)
Cloud Formation Example
aws cloudformation create-stack –stack-name dev-ecs-stack –template-body file://master.yaml –parameters file://parameter_dev.json –capabilities CAPABILITY_IAM
ECS Task Definition
Task Definition is similar to docker-compose.
Task definition can consist 1 or more container definitions
It defines
- Docker Images to use
- Port and Drive Volume Mapping
- CPU and memory to use with container
- Whether containers are linked
- Environmental variable which is required to be passed to container.
ECS services
- Allows you to run and maintain a specified/desired number of tasks.
- If any task fails or stop for any reason, ECS service scheduler launches another task of your task definition to maintain desired task count.
Deploying ECS Cluster
- Create Security groups at instance and load balancer level.
- Create an Application Load Balancer
- Create a Launch configuration with ECS optimised AWS AMI
- Create a Autoscaling group, which specifies the desired number of instances
- Create a task definition
- Create a target group and ecs service
Sample ECS architecture
ECS Instance Level Auto Scaling
ECS provides cluster-level parameters which can give the cluster utilization Statistics
- Memory Reservation – Current % of reserved memory by cluster
- Memory Utilization – Current % of utilized memory by cluster
- CPU Reservation – Current % of reserved CPU by cluster
- CPU Utilization – Current % of utilized CPU by cluster
CloudWatch Alarms on the above parameters enables to Scale Up/Down the ECS cluster
ECS Service Level Autoscaling
- ECS also provides the facility to scale up/down the number of tasks in the service.
- Tasks can be autoscaled on following ECS service parameters
- CPU Utilization – Current % CPU utilization by ECS service
- Memory Utilization – Current % Memory Utilization by ECS
CloudWatch Alarms on the above parameters enables to Scale Up/Down the service.
ECS Auto Scaling Overview
Monitoring and Logging
CloudWatch
- Use Cloudwatch Logs to centralized all container service logs
- Follow “ecs/stackname/servicename” Log Group Format.
- Get notification in slacks channel about the Cloudwatch ECS Alarms and Events via AWS Lambda function.
Key Advantages of ECS Service
- Easy Cluster Management – ECS sets up and manages clusters made up of Docker containers. It launches and terminates the containers and maintains complete information about the state of your cluster.
- Auto Scaling – Instance as well as Service level.
- Zero-downtime deployment – service updation follows Blue-Green deployments.
- Resource Efficiency – A containerized application can make very efficient use of resources. You can choose to run multiple, unrelated containers on the same EC2 instance in order to make good use of all available resources.
- AWS Integration – Your applications can make use of AWS features such as Elastic IP addresses, resource tags, and Virtual Private Cloud (VPC)
- Service Discovery – used for internal Service to service communication.
- Fargate technology – automatically scale, load balance, and manage scheduling of your containers.
- Secure – Your tasks run on EC2 instances within an The tasks can take advantage of IAM roles, security groups, and other AWS security features.
Key Challenges of ECS Service
- Supported by only AWS.
- Application level custom monitoring is not available.
