Slide 1
Slide 1 text
Lancers x ECS
Jason
Slide 2
Slide 2 text
Agenda
For scaling of service
- Problems
- Solutions
Slide 3
Slide 3 text
Problems for scaling of service
Slide 4
Slide 4 text
2
Problems for scaling of service
Slide 5
Slide 5 text
SRE: 2
Problems for scaling of service
Slide 6
Slide 6 text
4
Problems for scaling of service
Slide 7
Slide 7 text
SRE: 4
Problems for scaling of service
Slide 8
Slide 8 text
4
Problems for scaling of service
Slide 9
Slide 9 text
Service:
Lancers/Quant
LancersTop/Pook
Sub Service: 20 +
Problems for scaling of service
Slide 10
Slide 10 text
Problems for scaling of service
Programming Languages
Slide 11
Slide 11 text
Terraform + Ansible
Problems for scaling of service
Slide 12
Slide 12 text
Solutions for scaling of service
Slide 13
Slide 13 text
ɾWhat is ECS
ɾWhy ECS?
ɾSystem Architecture
ɾCreate Dockerfile
ɾCreate ECS
ɾRelease
ɾLog
Slide 14
Slide 14 text
What is ECS
Slide 15
Slide 15 text
Solutions for scaling of service
What is ECS
Service Definition
Task Definition
Service
ECR
Cluster
Slide 16
Slide 16 text
Solutions for scaling of service
Terraform + ECS
VPC/ALB/ECR
Security Group
EC2/ECS
(Cluster, Service,
Task Definition)
※ I plan to use terraform when I Migrate to fargate.
ECS-CLI
Slide 17
Slide 17 text
Why ECS?
Slide 18
Slide 18 text
Solutions for scaling of service
Why ECS?
Speed
Pull -> Build -> Up
Quickly & Disposability
Slide 19
Slide 19 text
Solutions for scaling of service
Why ECS?
Immutable Infrastructure
1. Troubleshooting
2. OSS Upgrade
3. Development -> Staging -> Production
Infrastructure as Code
!Stop! Manual Operation!
Slide 20
Slide 20 text
I. Codebase
One codebase tracked in revision control, many deploys
II. Dependencies
Explicitly declare and isolate dependencies
III. Config
Store config in the environment
IV. Backing services
Treat backing services as attached resources
V. Build, release, run
Strictly separate build and run stages
VI. Processes
Execute the app as one or more stateless processes
VII. Port binding
Export services via port binding
VIII. Concurrency
Scale out via the process model
IX. Disposability
Maximize robustness with fast startup and graceful shutdown
X. Dev/prod parity
Keep development, staging, and production as similar as possible
XI. Logs
Treat logs as event streams
XII. Admin processes
Run admin/management tasks as one-off processes
The Twelve-Factor App
Solutions for scaling of service
※https://12factor.net
Slide 21
Slide 21 text
System Architecture
Slide 22
Slide 22 text
Solutions for scaling of service
Pook System Architecture
Api
Admin
Job
Batch
S3
CloudFront
ALB ALB
Nginx
Redis
RDS
Slide 23
Slide 23 text
Solutions for scaling of service
Dynamic port mapping
ALB
Nginx
80:80
Nginx
80:80
ALB
Nginx
0:80
Nginx
0:80
Slide 24
Slide 24 text
Create Dockerfile
&
Docker compose
Slide 25
Slide 25 text
Solutions for scaling of service
Dockerfile
API_Base API
FROM xxxx:xx
MAINTAINER hoge
RUN yum -y install \
ImageMagick
…
FROM API_Base:xx
MAINTAINER hoge
ARG CORE_SRC
ARG CORE_DEST
COPY ./docker_v2/pook_api/service.sh /app/
RUN chmod 755 /root/service.sh
COPY ${CORE_SRC} ${CORE_DEST}
RUN chown lancers:lancers -R ${CORE_DEST}
EXPOSE 3000
Slide 26
Slide 26 text
Solutions for scaling of service
How to Start Service?
command: /app/service.sh ${RAILS_ENV} api
command: /app/service.sh ${RAILS_ENV} delayed_job
docker-compose.yml
Api Job
=
Slide 27
Slide 27 text
Solutions for scaling of service
Docker compose
services:
pook_api:
image: ${API_IMAGE}:${API_TAG}
build:
context: .
dockerfile: ./docker_v2/pook_api/Dockerfile
args:
- CORE_SRC=./xxx/xxx
- CORE_DEST=/xxx/xxx/xxx
logging:
driver: awslogs
options:
awslogs-group: ecs-api-log
awslogs-region: ap-northeast-1
awslogs-stream-prefix: api
hostname: pook-api
ports:
- "0:3000"
mem_limit: ${API_MEM_LIMIT}
command: /app/service.sh ${RAILS_ENV} api
container_name: pook_api
Slide 28
Slide 28 text
Create ECS
Slide 29
Slide 29 text
Solutions for scaling of service
Service Definition
Task Definition
Service
Cluster
Slide 30
Slide 30 text
Solutions for scaling of service
Create ECS Cluster
$ ecs-cli up \
> --region ap-northeast-1 \
> --instance-role \
> --instance-type t2.medium \
> --vpc \
> --subnets , \
> --security-group \
> --size 2 \
> --keypair \
> --no-associate-public-ip-address \
> --launch-type EC2 \
> --cluster \
> --ecs-profile
Slide 31
Slide 31 text
Solutions for scaling of service
Create Task Definition & Service
$ ecs-cli compose \
> --file docker-compose.api.yml \
> --ecs-profile \
> --project-name pook_api \
> service up \
> --container-name pook_api \
> --container-port 3000 \
> —target-group-arn
Slide 32
Slide 32 text
Release
Slide 33
Slide 33 text
Release
Solutions for scaling of service
$ ecs-cli compose \
> --file docker-compose.api.yml \
> --ecs-profile \
> --project-name pook_api \
> service up
Slide 34
Slide 34 text
Solutions for scaling of service
Release
Slide 35
Slide 35 text
Solutions for scaling of service
Release
Slide 36
Slide 36 text
Deploy Tool
Solutions for scaling of service
Slide 37
Slide 37 text
Log
Slide 38
Slide 38 text
Solutions for scaling of service
Log
CloudWatch Logs
ElasticSearch Service
AWS Lambda S3
https://github.com/jasonsmithj/cloudwatchlog-to-s3
Slide 39
Slide 39 text
Solutions for scaling of service
Slide 40
Slide 40 text
Solutions for scaling of service
Log
$ pip install awslogs
$ awslogs get -w -S —timestamp
xxx 2018-02-14T01:32:05.194Z [4e627818-23d8-445b-93bd-d1f63bc5bf3b] Started GET "/
ping" for 10.4.106.143 at 2018-02-14 10:32:05 +0900
xxx 2018-02-14T01:32:20.182Z [0964b77c-7613-4414-90d3-3fae9066b289] Started GET "/
ping" for 10.4.6.250 at 2018-02-14 10:32:20 +0900
xxx 2018-02-14T01:32:20.210Z [6a46343d-6407-4263-b8d1-bcbbaf959598] Started GET
"/ping" for 10.4.106.143 at 2018-02-14 10:32:20 +0900
https://qiita.com/Jason/items/cecad88311eeeb9b5d9e
Slide 41
Slide 41 text
Solutions for scaling of service
etc…
Delete Old ECR Images
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content