Lancers x ECS

Lancers x ECS

291429be59bd97d8a3c21c17c55c7a22?s=128

takayuki.hiratsuka

February 21, 2018
Tweet

Transcript

  1. Lancers x ECS Jason

  2. Agenda For scaling of service - Problems - Solutions

  3. Problems for scaling of service

  4. 2 Problems for scaling of service

  5. SRE: 2 Problems for scaling of service

  6. 4 Problems for scaling of service

  7. SRE: 4 Problems for scaling of service

  8. 4 Problems for scaling of service

  9. Service: Lancers/Quant LancersTop/Pook Sub Service: 20 + Problems for scaling

    of service
  10. Problems for scaling of service Programming Languages

  11. Terraform + Ansible Problems for scaling of service

  12. Solutions for scaling of service

  13. ɾWhat is ECS ɾWhy ECS? ɾSystem Architecture ɾCreate Dockerfile ɾCreate

    ECS ɾRelease ɾLog
  14. What is ECS

  15. Solutions for scaling of service What is ECS Service Definition

    Task Definition Service ECR Cluster
  16. Solutions for scaling of service Terraform + ECS VPC/ALB/ECR Security

    Group EC2/ECS (Cluster, Service, Task Definition) ※ I plan to use terraform when I Migrate to fargate. ECS-CLI
  17. Why ECS?

  18. Solutions for scaling of service Why ECS? Speed Pull ->

    Build -> Up Quickly & Disposability
  19. Solutions for scaling of service Why ECS? Immutable Infrastructure 1.

    Troubleshooting 2. OSS Upgrade 3. Development -> Staging -> Production Infrastructure as Code !Stop! Manual Operation!
  20. I. Codebase One codebase tracked in revision control, many deploys

    II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages VI. Processes Execute the app as one or more stateless processes VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown X. Dev/prod parity Keep development, staging, and production as similar as possible XI. Logs Treat logs as event streams XII. Admin processes Run admin/management tasks as one-off processes The Twelve-Factor App Solutions for scaling of service ※https://12factor.net
  21. System Architecture

  22. Solutions for scaling of service Pook System Architecture Api Admin

    Job Batch S3 CloudFront ALB ALB Nginx Redis RDS
  23. Solutions for scaling of service Dynamic port mapping ALB Nginx

    80:80 Nginx 80:80 ALB Nginx 0:80 Nginx 0:80
  24. Create Dockerfile & Docker compose

  25. Solutions for scaling of service Dockerfile API_Base API FROM xxxx:xx

    MAINTAINER hoge RUN yum -y install \ ImageMagick … FROM API_Base:xx MAINTAINER hoge ARG CORE_SRC ARG CORE_DEST COPY ./docker_v2/pook_api/service.sh /app/ RUN chmod 755 /root/service.sh COPY ${CORE_SRC} ${CORE_DEST} RUN chown lancers:lancers -R ${CORE_DEST} EXPOSE 3000
  26. Solutions for scaling of service How to Start Service? command:

    /app/service.sh ${RAILS_ENV} api command: /app/service.sh ${RAILS_ENV} delayed_job docker-compose.yml Api Job =
  27. Solutions for scaling of service Docker compose services: pook_api: image:

    ${API_IMAGE}:${API_TAG} build: context: . dockerfile: ./docker_v2/pook_api/Dockerfile args: - CORE_SRC=./xxx/xxx - CORE_DEST=/xxx/xxx/xxx logging: driver: awslogs options: awslogs-group: ecs-api-log awslogs-region: ap-northeast-1 awslogs-stream-prefix: api hostname: pook-api ports: - "0:3000" mem_limit: ${API_MEM_LIMIT} command: /app/service.sh ${RAILS_ENV} api container_name: pook_api
  28. Create ECS

  29. Solutions for scaling of service Service Definition Task Definition Service

    Cluster
  30. Solutions for scaling of service Create ECS Cluster $ ecs-cli

    up \ > --region ap-northeast-1 \ > --instance-role <IAM ROLE> \ > --instance-type t2.medium \ > --vpc <VPC ID> \ > --subnets <Subnet ID>,<Subnet ID> \ > --security-group <Security Group ID> \ > --size 2 \ > --keypair <KeyPair> \ > --no-associate-public-ip-address \ > --launch-type EC2 \ > --cluster <Cluster Name> \ > --ecs-profile <ECS Profile Name>
  31. Solutions for scaling of service Create Task Definition & Service

    $ ecs-cli compose \ > --file docker-compose.api.yml \ > --ecs-profile <ECS Profile Name> \ > --project-name pook_api \ > service up \ > --container-name pook_api \ > --container-port 3000 \ > —target-group-arn <Target Group>
  32. Release

  33. Release Solutions for scaling of service $ ecs-cli compose \

    > --file docker-compose.api.yml \ > --ecs-profile <ECS Profile Name> \ > --project-name pook_api \ > service up
  34. Solutions for scaling of service Release

  35. Solutions for scaling of service Release

  36. Deploy Tool Solutions for scaling of service

  37. Log

  38. Solutions for scaling of service Log CloudWatch Logs ElasticSearch Service

    AWS Lambda S3 https://github.com/jasonsmithj/cloudwatchlog-to-s3
  39. Solutions for scaling of service

  40. Solutions for scaling of service Log $ pip install awslogs

    $ awslogs get <Log Group Name> -w -S —timestamp xxx 2018-02-14T01:32:05.194Z [4e627818-23d8-445b-93bd-d1f63bc5bf3b] Started GET "/ ping" for 10.4.106.143 at 2018-02-14 10:32:05 +0900 xxx 2018-02-14T01:32:20.182Z [0964b77c-7613-4414-90d3-3fae9066b289] Started GET "/ ping" for 10.4.6.250 at 2018-02-14 10:32:20 +0900 xxx 2018-02-14T01:32:20.210Z [6a46343d-6407-4263-b8d1-bcbbaf959598] Started GET "/ping" for 10.4.106.143 at 2018-02-14 10:32:20 +0900 https://qiita.com/Jason/items/cecad88311eeeb9b5d9e
  41. Solutions for scaling of service etc… Delete Old ECR Images

  42. None
  43. None