Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Quipper on Containers
Search
Kazuyuki Honda
September 21, 2016
Technology
0
1.2k
Quipper on Containers
2016/09/21 JAWS UG コンテナ支部 #6
Kazuyuki Honda
September 21, 2016
Tweet
Share
More Decks by Kazuyuki Honda
See All by Kazuyuki Honda
How to Write Complex Data Pipeline in Ruby
hakobera
2
4.7k
はじめての BQ GAS
hakobera
10
3.6k
MQTT.IO
hakobera
9
1.7k
Logging Technique for HTML5 Single Page Application with Fluentd
hakobera
11
10k
Other Decks in Technology
See All in Technology
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
soudai
PRO
50
20k
スタートアップに選択肢を 〜生成AIを活用したセカンダリー事業への挑戦〜
nstock
0
220
Core Audio tapを使ったリアルタイム音声処理のお話
yuta0306
0
190
PO初心者が考えた ”POらしさ”
nb_rady
0
210
Flutter向けPDFビューア、pdfrxのpdfium WASM対応について
espresso3389
0
130
React開発にStorybookとCopilotを導入して、爆速でUIを編集・確認する方法
yu_kod
1
280
Getting to Know Your Legacy (System) with AI-Driven Software Archeology (WeAreDevelopers World Congress 2025)
feststelltaste
1
130
How Do I Contact HP Printer Support? [Full 2025 Guide for U.S. Businesses]
harrry1211
0
120
品質と速度の両立:生成AI時代の品質保証アプローチ
odasho
1
370
Yahoo!しごとカタログ 新しい境地を創るエンジニア募集!
lycorptech_jp
PRO
0
120
整頓のジレンマとの戦い〜Tidy First?で振り返る事業とキャリアの歩み〜/Fighting the tidiness dilemma〜Business and Career Milestones Reflected on in Tidy First?〜
bitkey
3
16k
20250707-AI活用の個人差を埋めるチームづくり
shnjtk
4
3.9k
Featured
See All Featured
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.5k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
50
5.5k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Bash Introduction
62gerente
613
210k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
970
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
Automating Front-end Workflow
addyosmani
1370
200k
Git: the NoSQL Database
bkeepers
PRO
430
65k
Code Review Best Practice
trishagee
69
18k
Embracing the Ebb and Flow
colly
86
4.7k
Transcript
on Containers )
@hakobera DevOps Engineer
None
Mexico City London Tokyo Manila Jakarta Sales team only Sales
and dev team International Offices No one here (yet).
Products
Container != Docker
Docker is one of the container impls. Ubuntu LXD Microsoft®
Windows® Server Container Hyper-V® Container
When do you use container on your system?
Now!
Why do you use container?
Portability and Beyond Your Cloud Limits
Docker on Elastic Beanstalk • For short batch using worker
tier • Avoid darkness of .ebextension • Avoid Amazon Linux ◦ We want to use Ubuntu
Load Test on Elastic Container Service • Easy to setup
locust cluster ◦ Consists of 2 Tasks ▪ 1 master ▪ Multiple slave ◦ Manage instances using Spot Fleet ▪ Cheap but scalable
nginx-build on CircleCI • Easy to build custom nginx build
◦ Bundle custom modules such as ngx_mruby ◦ Patch for security issue ▪ Update only openssl but not nginx
Ansible playbook test on CircleCI • Run playbook to clean
environment • Run playboot test in parallel
Do you also use container for Web application?
Yes! But it’s not so easy
History of Quipper and Container
Quipper is Container Native Company
Quipper use container from beginning
Back to 2012 First Product First Container is ...
None
It’s too early
No source repository Only CTO only knows
From 2013 Second Container Choice is ...
None
Heroku is great!
Awesome features of Heroku • Deploy via git • Buildpack
• Multiple Dyno Size • One-off dyno • Review Apps
At April, 2015 We are acquired by Recruit Co. Ltd
What problem do we have to resolve? • Performance ◦
Latency from Japan and USA (400ms/req) ◦ Needed more powerful dyno • Security ◦ At least we need fixed outbound IP address • Cost
We want to keep awesome features • Deploy using git
from CircleCI • Buildpack • Multiple Dyno Size • One-off dyno • CLI compatibility
None
Your PaaS, Your Rules Open Source Application Platform
Similarity to Heroku • Deploy via git • Buildpack •
CLI sub command ◦ Ex: heroku conifg:set <-> deis config:set • Interactive console not yet supported ◦ Create own tool using Docker API
Similarity to Heroku is important Because we’re deploying one source
Source: http://docs.deis.io/en/latest/understanding_deis/architecture/
None
None
None
Components of PaaS for Web app • Router • Controller
• Slug Builder • Container Registry • Object Storage • Scheduler • Key Value Store • Database • Git Repository • Logger
Where is the best place to deploy each component?
Late 2015
Elastic Load Balancing Router Controller Builder Applications etcd AZ #1
AZ #2 Production Cluster 1st Gen. PosggreSQL On Amazon RDS Docker Registry On Amazon S3
Hard to Scale Out
April 2016
Elastic Load Balancing Router/Applications AZ #1 AZ #2 Controller /Builder
etcd Production Cluster 2nd Gen. PosggreSQL On Amazon RDS Docker Registry On Amazon S3
Noisy Neighbor AND Cost Increase
Fleet is too simple • Fleet is not resource aware
scheduler • Fleet deploy new container in instance which has the least containers • Sometimes make it unbalanced workload of clusters
Bad scheduling of fleet High High Normal Low Low Low
Instance 1 Instance 2 Instance 3
We hope resource aware scheduling High High Normal Low Low
Instance 1 Instance 2 Instance 3 Low
Aug 2016 〜
Elastic Load Balancing Router/Apps (Shared) AZ #1 AZ #2 Controller
/Builder High-Load Apps (Dedicated) etcd Production Cluster 3rd Gen PosggreSQL On Amazon RDS Docker Registry On Amazon S3
Tag based Deployment High HighI Normal Low Low Low high=false
high=true
Tag based Deployment • Tag = Fleet metadata • Set
fleet metadata by cloud-init #cloud-config Coreos: ... fleet: public-ip: $private_ipv4 metadata: dataPlane=true,high=true #cloud-config Coreos: ... fleet: public-ip: $private_ipv4 metadata: dataPlane=true,routerMesh=true,high=false For dedicated For shared
Elastic Load Balancing Router AZ #1 AZ #2 Controller /Builder
Applications etcd Staging Cluster PosggreSQL On Amazon RDS Docker Registry On Amazon S3
Other Awesome Features
Blue Green Deployment 1. Start new containers 2. Notify to
etcd 3. Update nginx.conf by confd 4. Reload router (nginx) 5. Update routing 6. Destroy old containers
Use nginx as reverse proxy for PaaS • Easy to
handle naked domain • Resolve slow client issue • Can support HTTP/2 • Flexible routing ◦ A/B test, 10% release, etc ...
Problem as of now
Fleet is too simple (again) • Fleet is not resource
aware scheduler • Fleet deploy new container in instance which has the least containers • Sometimes make it unbalanced workload of clusters
etcd stability • etcd sometimes get high CPU load
In the future ...
Deis Workflow (Deis v2) • https://github.com/deis/workflow • Based on Kubernates
End
We’re hiring! https://www.quipper.com/career/Japan/ https://www.wantedly.com/companies/quipper