Lancers x ECS

Transcript

1. Lancers x ECS Jason
2. Agenda For scaling of service - Problems - Solutions
3. Problems for scaling of service
4. 2 Problems for scaling of service
5. SRE: 2 Problems for scaling of service
6. 4 Problems for scaling of service
7. SRE: 4 Problems for scaling of service
8. 4 Problems for scaling of service
9. Service: Lancers/Quant LancersTop/Pook Sub Service: 20 + Problems for scaling of service
10. Problems for scaling of service Programming Languages
11. Terraform + Ansible Problems for scaling of service
12. Solutions for scaling of service
13. ɾWhat is ECS ɾWhy ECS? ɾSystem Architecture ɾCreate Dockerfile ɾCreate ECS ɾRelease ɾLog
14. What is ECS
15. Solutions for scaling of service What is ECS Service Definition Task Definition Service ECR Cluster
16. Solutions for scaling of service Terraform + ECS VPC/ALB/ECR Security Group EC2/ECS (Cluster, Service, Task Definition) ※ I plan to use terraform when I Migrate to fargate. ECS-CLI
17. Why ECS?
18. Solutions for scaling of service Why ECS? Speed Pull -> Build -> Up Quickly & Disposability
19. Solutions for scaling of service Why ECS? Immutable Infrastructure 1. Troubleshooting 2. OSS Upgrade 3. Development -> Staging -> Production Infrastructure as Code !Stop! Manual Operation!
20. I. Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages VI. Processes Execute the app as one or more stateless processes VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown X. Dev/prod parity Keep development, staging, and production as similar as possible XI. Logs Treat logs as event streams XII. Admin processes Run admin/management tasks as one-off processes The Twelve-Factor App Solutions for scaling of service ※https://12factor.net
21. System Architecture
22. Solutions for scaling of service Pook System Architecture Api Admin Job Batch S3 CloudFront ALB ALB Nginx Redis RDS
23. Solutions for scaling of service Dynamic port mapping ALB Nginx 80:80 Nginx 80:80 ALB Nginx 0:80 Nginx 0:80
24. Create Dockerfile & Docker compose
25. Solutions for scaling of service Dockerfile API_Base API FROM xxxx:xx MAINTAINER hoge RUN yum -y install \ ImageMagick … FROM API_Base:xx MAINTAINER hoge ARG CORE_SRC ARG CORE_DEST COPY ./docker_v2/pook_api/service.sh /app/ RUN chmod 755 /root/service.sh COPY ${CORE_SRC} ${CORE_DEST} RUN chown lancers:lancers -R ${CORE_DEST} EXPOSE 3000
26. Solutions for scaling of service How to Start Service? command: /app/service.sh ${RAILS_ENV} api command: /app/service.sh ${RAILS_ENV} delayed_job docker-compose.yml Api Job =
27. Solutions for scaling of service Docker compose services: pook_api: image: ${API_IMAGE}:${API_TAG} build: context: . dockerfile: ./docker_v2/pook_api/Dockerfile args: - CORE_SRC=./xxx/xxx - CORE_DEST=/xxx/xxx/xxx logging: driver: awslogs options: awslogs-group: ecs-api-log awslogs-region: ap-northeast-1 awslogs-stream-prefix: api hostname: pook-api ports: - "0:3000" mem_limit: ${API_MEM_LIMIT} command: /app/service.sh ${RAILS_ENV} api container_name: pook_api
28. Create ECS
29. Solutions for scaling of service Service Definition Task Definition Service Cluster
30. Solutions for scaling of service Create ECS Cluster $ ecs-cli up \ > --region ap-northeast-1 \ > --instance-role <IAM ROLE> \ > --instance-type t2.medium \ > --vpc <VPC ID> \ > --subnets <Subnet ID>,<Subnet ID> \ > --security-group <Security Group ID> \ > --size 2 \ > --keypair <KeyPair> \ > --no-associate-public-ip-address \ > --launch-type EC2 \ > --cluster <Cluster Name> \ > --ecs-profile <ECS Profile Name>
31. Solutions for scaling of service Create Task Definition & Service $ ecs-cli compose \ > --file docker-compose.api.yml \ > --ecs-profile <ECS Profile Name> \ > --project-name pook_api \ > service up \ > --container-name pook_api \ > --container-port 3000 \ > —target-group-arn <Target Group>
32. Release
33. Release Solutions for scaling of service $ ecs-cli compose \ > --file docker-compose.api.yml \ > --ecs-profile <ECS Profile Name> \ > --project-name pook_api \ > service up
34. Solutions for scaling of service Release
35. Solutions for scaling of service Release
36. Deploy Tool Solutions for scaling of service
37. Log
38. Solutions for scaling of service Log CloudWatch Logs ElasticSearch Service AWS Lambda S3 https://github.com/jasonsmithj/cloudwatchlog-to-s3
39. Solutions for scaling of service
40. Solutions for scaling of service Log $ pip install awslogs $ awslogs get <Log Group Name> -w -S —timestamp xxx 2018-02-14T01:32:05.194Z [4e627818-23d8-445b-93bd-d1f63bc5bf3b] Started GET "/ ping" for 10.4.106.143 at 2018-02-14 10:32:05 +0900 xxx 2018-02-14T01:32:20.182Z [0964b77c-7613-4414-90d3-3fae9066b289] Started GET "/ ping" for 10.4.6.250 at 2018-02-14 10:32:20 +0900 xxx 2018-02-14T01:32:20.210Z [6a46343d-6407-4263-b8d1-bcbbaf959598] Started GET "/ping" for 10.4.106.143 at 2018-02-14 10:32:20 +0900 https://qiita.com/Jason/items/cecad88311eeeb9b5d9e
41. Solutions for scaling of service etc… Delete Old ECR Images