Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lancers x ECS

Lancers x ECS

takayuki.hiratsuka

February 21, 2018
Tweet

More Decks by takayuki.hiratsuka

Other Decks in Technology

Transcript

  1. Lancers x ECS
    Jason

    View full-size slide

  2. Agenda
    For scaling of service
    - Problems
    - Solutions

    View full-size slide

  3. Problems for scaling of service

    View full-size slide

  4. 2
    Problems for scaling of service

    View full-size slide

  5. SRE: 2
    Problems for scaling of service

    View full-size slide

  6. 4
    Problems for scaling of service

    View full-size slide

  7. SRE: 4
    Problems for scaling of service

    View full-size slide

  8. 4
    Problems for scaling of service

    View full-size slide

  9. Service:
    Lancers/Quant
    LancersTop/Pook
    Sub Service: 20 +
    Problems for scaling of service

    View full-size slide

  10. Problems for scaling of service
    Programming Languages

    View full-size slide

  11. Terraform + Ansible
    Problems for scaling of service

    View full-size slide

  12. Solutions for scaling of service

    View full-size slide

  13. ɾWhat is ECS
    ɾWhy ECS?
    ɾSystem Architecture
    ɾCreate Dockerfile
    ɾCreate ECS
    ɾRelease
    ɾLog

    View full-size slide

  14. Solutions for scaling of service
    What is ECS
    Service Definition
    Task Definition
    Service
    ECR
    Cluster

    View full-size slide

  15. Solutions for scaling of service
    Terraform + ECS
    VPC/ALB/ECR
    Security Group
    EC2/ECS
    (Cluster, Service,
    Task Definition)
    ※ I plan to use terraform when I Migrate to fargate.
    ECS-CLI

    View full-size slide

  16. Solutions for scaling of service
    Why ECS?
    Speed
    Pull -> Build -> Up
    Quickly & Disposability

    View full-size slide

  17. Solutions for scaling of service
    Why ECS?
    Immutable Infrastructure
    1. Troubleshooting
    2. OSS Upgrade
    3. Development -> Staging -> Production
    Infrastructure as Code
    !Stop! Manual Operation!

    View full-size slide

  18. I. Codebase
    One codebase tracked in revision control, many deploys
    II. Dependencies
    Explicitly declare and isolate dependencies
    III. Config
    Store config in the environment
    IV. Backing services
    Treat backing services as attached resources
    V. Build, release, run
    Strictly separate build and run stages
    VI. Processes
    Execute the app as one or more stateless processes
    VII. Port binding
    Export services via port binding
    VIII. Concurrency
    Scale out via the process model
    IX. Disposability
    Maximize robustness with fast startup and graceful shutdown
    X. Dev/prod parity
    Keep development, staging, and production as similar as possible
    XI. Logs
    Treat logs as event streams
    XII. Admin processes
    Run admin/management tasks as one-off processes
    The Twelve-Factor App
    Solutions for scaling of service
    ※https://12factor.net

    View full-size slide

  19. System Architecture

    View full-size slide

  20. Solutions for scaling of service
    Pook System Architecture
    Api
    Admin
    Job
    Batch
    S3
    CloudFront
    ALB ALB
    Nginx
    Redis
    RDS

    View full-size slide

  21. Solutions for scaling of service
    Dynamic port mapping
    ALB
    Nginx
    80:80
    Nginx
    80:80
    ALB
    Nginx
    0:80
    Nginx
    0:80

    View full-size slide

  22. Create Dockerfile
    &
    Docker compose

    View full-size slide

  23. Solutions for scaling of service
    Dockerfile
    API_Base API
    FROM xxxx:xx
    MAINTAINER hoge
    RUN yum -y install \
    ImageMagick

    FROM API_Base:xx
    MAINTAINER hoge
    ARG CORE_SRC
    ARG CORE_DEST
    COPY ./docker_v2/pook_api/service.sh /app/
    RUN chmod 755 /root/service.sh
    COPY ${CORE_SRC} ${CORE_DEST}
    RUN chown lancers:lancers -R ${CORE_DEST}
    EXPOSE 3000

    View full-size slide

  24. Solutions for scaling of service
    How to Start Service?
    command: /app/service.sh ${RAILS_ENV} api
    command: /app/service.sh ${RAILS_ENV} delayed_job
    docker-compose.yml
    Api Job
    =

    View full-size slide

  25. Solutions for scaling of service
    Docker compose
    services:
    pook_api:
    image: ${API_IMAGE}:${API_TAG}
    build:
    context: .
    dockerfile: ./docker_v2/pook_api/Dockerfile
    args:
    - CORE_SRC=./xxx/xxx
    - CORE_DEST=/xxx/xxx/xxx
    logging:
    driver: awslogs
    options:
    awslogs-group: ecs-api-log
    awslogs-region: ap-northeast-1
    awslogs-stream-prefix: api
    hostname: pook-api
    ports:
    - "0:3000"
    mem_limit: ${API_MEM_LIMIT}
    command: /app/service.sh ${RAILS_ENV} api
    container_name: pook_api

    View full-size slide

  26. Solutions for scaling of service
    Service Definition
    Task Definition
    Service
    Cluster

    View full-size slide

  27. Solutions for scaling of service
    Create ECS Cluster
    $ ecs-cli up \
    > --region ap-northeast-1 \
    > --instance-role \
    > --instance-type t2.medium \
    > --vpc \
    > --subnets , \
    > --security-group \
    > --size 2 \
    > --keypair \
    > --no-associate-public-ip-address \
    > --launch-type EC2 \
    > --cluster \
    > --ecs-profile

    View full-size slide

  28. Solutions for scaling of service
    Create Task Definition & Service
    $ ecs-cli compose \
    > --file docker-compose.api.yml \
    > --ecs-profile \
    > --project-name pook_api \
    > service up \
    > --container-name pook_api \
    > --container-port 3000 \
    > —target-group-arn

    View full-size slide

  29. Release
    Solutions for scaling of service
    $ ecs-cli compose \
    > --file docker-compose.api.yml \
    > --ecs-profile \
    > --project-name pook_api \
    > service up

    View full-size slide

  30. Solutions for scaling of service
    Release

    View full-size slide

  31. Solutions for scaling of service
    Release

    View full-size slide

  32. Deploy Tool
    Solutions for scaling of service

    View full-size slide

  33. Solutions for scaling of service
    Log
    CloudWatch Logs
    ElasticSearch Service
    AWS Lambda S3
    https://github.com/jasonsmithj/cloudwatchlog-to-s3

    View full-size slide

  34. Solutions for scaling of service

    View full-size slide

  35. Solutions for scaling of service
    Log
    $ pip install awslogs
    $ awslogs get -w -S —timestamp
    xxx 2018-02-14T01:32:05.194Z [4e627818-23d8-445b-93bd-d1f63bc5bf3b] Started GET "/
    ping" for 10.4.106.143 at 2018-02-14 10:32:05 +0900
    xxx 2018-02-14T01:32:20.182Z [0964b77c-7613-4414-90d3-3fae9066b289] Started GET "/
    ping" for 10.4.6.250 at 2018-02-14 10:32:20 +0900
    xxx 2018-02-14T01:32:20.210Z [6a46343d-6407-4263-b8d1-bcbbaf959598] Started GET
    "/ping" for 10.4.106.143 at 2018-02-14 10:32:20 +0900
    https://qiita.com/Jason/items/cecad88311eeeb9b5d9e

    View full-size slide

  36. Solutions for scaling of service
    etc…
    Delete Old ECR Images

    View full-size slide