Quipper on Containers

7ebb44070077adbf77dececc7802e58b?s=47 Kazuyuki Honda
September 21, 2016

Quipper on Containers

2016/09/21 JAWS UG コンテナ支部 #6

7ebb44070077adbf77dececc7802e58b?s=128

Kazuyuki Honda

September 21, 2016
Tweet

Transcript

  1. 3.
  2. 4.

    Mexico City London Tokyo Manila Jakarta Sales team only Sales

    and dev team International Offices No one here (yet).
  3. 7.

    Docker is one of the container impls. Ubuntu LXD Microsoft®

    Windows® Server Container Hyper-V® Container
  4. 9.
  5. 12.

    Docker on Elastic Beanstalk • For short batch using worker

    tier • Avoid darkness of .ebextension • Avoid Amazon Linux ◦ We want to use Ubuntu
  6. 13.

    Load Test on Elastic Container Service • Easy to setup

    locust cluster ◦ Consists of 2 Tasks ▪ 1 master ▪ Multiple slave ◦ Manage instances using Spot Fleet ▪ Cheap but scalable
  7. 14.

    nginx-build on CircleCI • Easy to build custom nginx build

    ◦ Bundle custom modules such as ngx_mruby ◦ Patch for security issue ▪ Update only openssl but not nginx
  8. 15.

    Ansible playbook test on CircleCI • Run playbook to clean

    environment • Run playboot test in parallel
  9. 22.
  10. 26.
  11. 28.

    Awesome features of Heroku • Deploy via git • Buildpack

    • Multiple Dyno Size • One-off dyno • Review Apps
  12. 30.

    What problem do we have to resolve? • Performance ◦

    Latency from Japan and USA (400ms/req) ◦ Needed more powerful dyno • Security ◦ At least we need fixed outbound IP address • Cost
  13. 31.

    We want to keep awesome features • Deploy using git

    from CircleCI • Buildpack • Multiple Dyno Size • One-off dyno • CLI compatibility
  14. 32.
  15. 34.

    Similarity to Heroku • Deploy via git • Buildpack •

    CLI sub command ◦ Ex: heroku conifg:set <-> deis config:set • Interactive console not yet supported ◦ Create own tool using Docker API
  16. 37.
  17. 38.
  18. 39.
  19. 40.

    Components of PaaS for Web app • Router • Controller

    • Slug Builder • Container Registry • Object Storage • Scheduler • Key Value Store • Database • Git Repository • Logger
  20. 42.
  21. 43.

    Elastic Load Balancing Router Controller Builder Applications etcd AZ #1

    AZ #2 Production Cluster 1st Gen. PosggreSQL On Amazon RDS Docker Registry On Amazon S3
  22. 46.

    Elastic Load Balancing Router/Applications AZ #1 AZ #2 Controller /Builder

    etcd Production Cluster 2nd Gen. PosggreSQL On Amazon RDS Docker Registry On Amazon S3
  23. 48.

    Fleet is too simple • Fleet is not resource aware

    scheduler • Fleet deploy new container in instance which has the least containers • Sometimes make it unbalanced workload of clusters
  24. 49.

    Bad scheduling of fleet High High Normal Low Low Low

    Instance 1 Instance 2 Instance 3
  25. 50.

    We hope resource aware scheduling High High Normal Low Low

    Instance 1 Instance 2 Instance 3 Low
  26. 52.

    Elastic Load Balancing Router/Apps (Shared) AZ #1 AZ #2 Controller

    /Builder High-Load Apps (Dedicated) etcd Production Cluster 3rd Gen PosggreSQL On Amazon RDS Docker Registry On Amazon S3
  27. 54.

    Tag based Deployment • Tag = Fleet metadata • Set

    fleet metadata by cloud-init #cloud-config Coreos: ... fleet: public-ip: $private_ipv4 metadata: dataPlane=true,high=true #cloud-config Coreos: ... fleet: public-ip: $private_ipv4 metadata: dataPlane=true,routerMesh=true,high=false For dedicated For shared
  28. 55.

    Elastic Load Balancing Router AZ #1 AZ #2 Controller /Builder

    Applications etcd Staging Cluster PosggreSQL On Amazon RDS Docker Registry On Amazon S3
  29. 57.

    Blue Green Deployment 1. Start new containers 2. Notify to

    etcd 3. Update nginx.conf by confd 4. Reload router (nginx) 5. Update routing 6. Destroy old containers
  30. 58.

    Use nginx as reverse proxy for PaaS • Easy to

    handle naked domain • Resolve slow client issue • Can support HTTP/2 • Flexible routing ◦ A/B test, 10% release, etc ...
  31. 60.

    Fleet is too simple (again) • Fleet is not resource

    aware scheduler • Fleet deploy new container in instance which has the least containers • Sometimes make it unbalanced workload of clusters
  32. 64.

    End