Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Next FRESH! Applications with Amazon ECS
Search
stormcat24
November 20, 2015
Programming
13
8.5k
Next FRESH! Applications with Amazon ECS
2015/11/20 JAWS-UGコンテナ支部 #3 #jawsug_ct
stormcat24
November 20, 2015
Tweet
Share
More Decks by stormcat24
See All by stormcat24
素早く賢く失敗するDeveloper Productivityの実現を目指して
stormcat24
4
4.7k
KubernetesのマニフェストをそれなりにCIしたい
stormcat24
4
1.3k
令和時代のSaaS開発
stormcat24
1
250
History in 5 years of CircleCI and CyberAgent
stormcat24
3
810
Kubernetes Handson Osaka
stormcat24
5
550
Kubernetes Handson
stormcat24
5
4.2k
DockerとKubernetesでアプリケーション開発にコンテナをフル活用!
stormcat24
0
290
Base Image Journey 2018
stormcat24
29
130k
kotlin-fest
stormcat24
13
17k
Other Decks in Programming
See All in Programming
毎日13時間もかかるバッチ処理をたった3日で60%短縮するためにやったこと
sho_ssk_
1
240
アクターシステムに頼らずEvent Sourcingする方法について
j5ik2o
4
320
「Chatwork」Android版アプリを 支える単体テストの現在
okuzawats
0
180
PHPで学ぶプログラミングの教訓 / Lessons in Programming Learned through PHP
nrslib
3
340
Webエンジニア主体のモバイルチームの 生産性を高く保つためにやったこと
igreenwood
0
340
コンテナをたくさん詰め込んだシステムとランタイムの変化
makihiro
1
140
17年周年のWebアプリケーションにTanStack Queryを導入する / Implementing TanStack Query in a 17th Anniversary Web Application
saitolume
0
250
どうして手を動かすよりもチーム内のコードレビューを優先するべきなのか
okashoi
3
280
rails stats で紐解く ANDPAD のイマを支える技術たち
andpad
1
300
Stackless и stackful? Корутины и асинхронность в Go
lamodatech
0
880
Zoneless Testing
rainerhahnekamp
0
120
なまけものオバケたち -PHP 8.4 に入った新機能の紹介-
tanakahisateru
1
120
Featured
See All Featured
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
17
2.3k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
1
100
Writing Fast Ruby
sferik
628
61k
Designing on Purpose - Digital PM Summit 2013
jponch
116
7k
Making the Leap to Tech Lead
cromwellryan
133
9k
YesSQL, Process and Tooling at Scale
rocio
169
14k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
3
170
Build The Right Thing And Hit Your Dates
maggiecrowley
33
2.4k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
169
50k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Building Your Own Lightsaber
phodgson
103
6.1k
Being A Developer After 40
akosma
87
590k
Transcript
Next FRESH! Applications with Amazon ECS 2015/11/20 JAWS-UGίϯςφࢧ෦ #3 #jawsug_ct
@stormcat24
Who? ‣ Akinori Yamada ( @stormcat24 ) ‣ http://stormcat.hatenablog.com/ ‣
CyberAgent, Inc ‣ Ameba౷ׅຊ෦ FRESHάϧʔϓ ‣ ServerSide / DevOps ‣ ͖ͳݴޠScala
Agenda ‣ AmebaFRESH! ‣ Architecture & ECS ‣ Blue Green
Deployment ‣ Diet Docker Image ‣ Local Development ‣ ecs-formation ‣ Others
AmebaFRESH!
‣ ੜ์ૹಈը৴ϓϥοτϑΥʔϜ ‣ جຊແྉ ‣ PCϒϥβ / iOS / Android
‣ εϚϗφΠζυ͞ΕͨUI ‣ ߴ࣭ͳಈը৴ ‣ 201512݄Ұൠެ։༧ఆ ‣ ݱࡏΫϩʔζυެ։த ‣ ※AbemaTVͱผͷαʔϏεͰ͢YO
None
None
None
None
‣ 2015/04͔ΒϓϩδΣΫτ։࢝ ‣ ϓϩδΣΫτϝϯόʔ30໊ ‣ Serverside ☓ 6 ‣ Frontend
☓ 6 ‣ iOS ☓ 4 ‣ Android ☓ 4 ‣ Designer ☓ 3
‣ Movie & Broadcasting ‣ RTMP Publishing ‣ HTTP Live
Streaming (HLS)
‣ Frontend ‣ Node.js v4 ‣ TypeScript1.6 ‣ React /
Flux ‣ SPA + SSR ‣ iOS ‣ Swift2.0 ‣ Android ‣ Kotlin
‣ Serverside & Infrastructure ‣ Amazon Web Services (AWS) ‣
Go1.5 ‣ Docker1.9.0 ‣ Microservices ‣ RESTful API (goji)
‣ Tools ‣ Slack + Hubot (ChatOps) ‣ Github Enterprise
‣ CircleCI (Enterprise) ‣ Crashlytics ‣ JIRA Agile ‣ Mackerel ‣ ࡳଋ
Architecture & ECS
Architecture ‣ ࢥ ‣ جຊํ ‣ Task Definitionͷߏ ‣ ClusterͱServiceͷѻ͍
ࢦ͍ͯ͠Δ͜ͱ ‣ ۃྗϝϯςΛೖΕͳ͍ ‣ θϩμϯλΠϜϦϦʔε → Blue Green Deployment ‣
Πϯϑϥ͍ࣺͯͯೖΕସ͑Δ → Immutable Infrastructure ‣ ૉૣ͘සൟͳϦϦʔε ‣ γεςϜݶΓͳ͘ૄ݁߹ͷํ͕ྑ͍ → Microservices ‣ ίϯςφͩ → Docker
Microservices ‣ ಛఆͷ։ൃݴޠʹґଘ͠ͳ͍ʢ͖Δ͠ʣ ‣ ௨৴ϓϩτίϧ ‣ RESTFul API (HTTP) ‣
ͦͷ͏ͪgRPC(HTTP2)͍ͨ͠ ‣ αʔϏεͷཻΛఆΊΔͷΓ͍͠ ‣ γεςϜతͳυϝΠϯྖҬͰ͚ΔͷҰͭͷࢦඪ ‣ ޙ͔ΒผαʔϏεͱͯ͠Γམͱ͢OK
Infrastructure ‣ AWS + EC2 Container Service(ECS)Λ࠾༻ ‣ ͪΐ͏Ͳٕज़ݕূதʹECSͷ౦ژϦʔδϣϯ͕དྷͨ ‣
࠷ݶͷίϯςφߏཧͱεέδϡʔϦϯά͕Ͱ͖Εे ‣ ECSҎ֎ʹ͍͍ͨͷ͕͋ͬͨ͠ ‣ Lambda ‣ RDS Aurora
جຊํ ‣ MicroservicesຖͰίϯςφΛߏ͢ΔTask DefinitionΛͭ͘Δ ‣ 1ECS Clusterʹ͖ͭ1Service ‣ 1Clusterʹ1ͭͷAutoScaling Group
‣ ಈը৴αʔόಛघͳͷͰྫ֎ ‣ Clusterؒ௨৴Internal ELBΛར༻
Task Definitionͷํ ‣ ϩάͷసૹʹtd-agentΛ͏ ‣ ֤ίϯςφϩάϗετʹϚϯτ ‣ ϗετʹϚϯτ͞ΕͨϩάΛtd-agentͰసૹ ‣ logging
driver·ͩಋೖͯ͠ͳ͍ ‣ Internal Service(API)Ͱ͋ͬͯجຊNginxΛ௨͢ ‣ ΞΫηεϩάग़͢ͷָͩ͠
Task Definition(Service API)
Task Definition(WEB+API)
Task Definition(Job)
Task Definition(movie)
جຊతʹ͜ΕΒͷΈ߹Θͤ
cluster = serviceͷσϝϦοτ ‣ ϦιʔεޮతʹϕετͰͳ͍ ‣ Cluster : Service =
1:N ʹൺΔͱ ‣ Πϯελϯε૿͑Δʹ͋Δ ‣ ։ൃڥͰt2.microΛ༗ޮ׆༻͍ͯ͠Δ ‣ nanoΠϯελϯεщ ƅшƅщ)ŜŹŖƃ
cluster = service ʹͯ͠Δཧ༝ ‣ ࢹ֮తʹΘ͔Γ͍͢ʢϏΪφʔϑϨϯυϦʔʣ ‣ Service୯ҐͰIAM roleΛઃఆͰ͖ͳ͍ ‣
IAM RoleͰݫີͳݖݶ੍ޚΛ͢ΔʹΓͳ͍ ‣ Service୯ҐͰͷSecurity GroupʹະରԠ ‣ ͨͩ͠ɺELBલఏʹͯ͠͠·͑ղܾͰ͖Δ
Blue Green Deployment
2AutoScalingύλʔϯ ‣ BlueɺGreenܥ౷ͷClusterΛ࡞Δ ‣ ͦΕͧΕ͕AutoScalingGroupʹଐ͢Δ ‣ api1-blue, api1-green Έ͍ͨͳ ‣
AutoScalingGroup୯ҐͰELBΛΓସ͑Δ
None
2AutoScalingͷಛ ‣ ͱͯ҆શ ‣ DeployޙͷϩʔϧόοΫ༰қ ‣ DeployલͷStandbyܥ౷ͷΥʔϜΞοϓඞཁ ‣ 10લʹ͓͖͍ͬͯͨ ‣
ίετͷέΞ͕ॏཁ ‣ DeployޙɺStandbyʹͳͬͨܥ౷Λམͱ͢
Diet Docker Image
Πϝʔδখ͍͞΄Ͳྑ͍ ‣ docker build࣌ؒˣ ‣ CI࣌ؒˣ ‣ Registry͔ΒͷΠϝʔδμϯϩʔυ࣌ؒˣ ‣ AutoScaleͰ࡞͞ΕͨΠϯελϯε͕αʔϏεΠϯ͢Δ
·Ͱͷ࣌ؒ↓
docker hub ‣ hub.docker.com ‣ ଟ͘ͷެࣜΠϝʔδ ‣ αΠζΛ͗མͱͨ͠ͷ͔ΓͰͳ͍ ‣ 1GB͑ΔΠϝʔδσϒ
ෆཁͳͷআ͢Δ ‣ ෆཁͳϑΝΠϧΛݟམͱ͞ͳ͍ ‣ ϏϧυͷͨΊʹੜͨ͡࢈ۀഇغΛআ ‣ npm cache clear ‣
rm -rf ~/.grade ‣ apt-get clean ‣ Data VolumeΛ͏ʢϙʔλϏϦςΟམͪΔʣ
RUNͷճΛݮΒ͢ ‣ RUNͷ͚ͩΠϝʔδͷϨΠϠʔ͕ॏͳΓɺΠ ϝʔδ༰ྔ૿͑Δ ‣ && ͰνΣΠϯͯ͠ɺRUNͷճΛݮΒ͢ ‣ ͍docker buildͷ߹ɺ్தͰࣦഊ͢ΔͱRUN
ͷ಄͔ΒΓͳ͓͠ͳͷͰফ͢Δ
RUNͷճΛݮΒ͢ FROM ubuntu:15.10 RUN apt-get update RUN apt-get install -y
curl RUN apt-get apt-get clean FROM ubuntu:15.10 RUN apt-get update && \ apt-get install -y curl && \ apt-get apt-get clean
ܰྔΠϝʔδΛ͏ ‣ ͗མͱ͞ΕͨܰྔΠϝʔδΛ͏ ‣ ࠷ۙΦϑΟγϟϧͰslimΠϝʔδ͕͋Δͷ͋Δ ‣ e.g. Node, Go ‣
busyboxઈ͍ܰ
‣ ࠷ۙܰྔΠϝʔδ(slim)༻ҙ͞ ΕͯΔ ‣ Docker HubΛීஈ͔Β८ճ͓ͯ͠ ͜͏
‣ ٻΊΒΕΔϙʔλϏϦςΟˢ ‣ GoͰ͋Ε࣮ߦϑΝΠϧΛࡌͤΔ ͚ͩ ‣ busyboxͷதͰϏϧυΩπΠ ʢқߴʣ
ݮྔʹΑΔࢥΘ͵ฐ ‣ x509: failed to load system roots and no
roots provided ‣ ίϯςφ͔ΒHTTPS௨৴͕Ͱ͖ͳ͘ͳΔ ‣ apt-get install -y ca-certificates Ͱղܾ ‣ ֎෦πʔϧͷґଘʢΞϓϦ͔ΒͷΩοΫʣ
ϕʔεΠϝʔδΛͭ͘Δ ‣ apt updateɺapt-get install ͏Μ͵ΜΛऴΘΒͤͨ ͷ ‣ ຖͬͯͨΒCIͷ͕࣌ؒ૿͑Δʢdocker buildͷ
҆ఆʣ ‣ ΞϓϦଆͷDockerfileͰຊ࣭ͷॲཧʹઐ೦ͤ͞Δ
Local Development
ϩʔΧϧ։ൃͰ͏ͷ ‣ docker-machine + VirtualBox ‣ docker-compose
docker-machine ‣ VirtualBox, AWS, Azure, Digital Ocean্ʹ DockerϗετΛߏங͢Δ ‣ ͔͋ͨϩʔΧϧ্ʹDockerڥ͕͋Δ͔ͷΑ͏
ʹDockerͷૢ࡞͕Ͱ͖Δ
docker-machine ‣ docker-machine + VirtualBoxΛબ ‣ Vagrantࣺͯͨ ‣ DockerʹൺΔͱ͍ࣺͯίετˢ ‣
ΞϓϦέʔγϣϯɺϛυϧΣΞؚΊ͍ͯͭͰϩʔΧϧͰ֬ ೝͰ͖ΔΑ͏ʹ ‣ VirtualBoxͷϙʔτϑΥϫʔυར༻
ϚγϯϦιʔεඞཁ ‣ ϩʔΧϧϑϧDockerͩͱɺٻΊΒΕΔϚγϯεϖο Ϋߴ͘ͳΔ ‣ 16GBͳ͍ͱ݁ߏਏ͍ ‣ αʔόαΠυΤϯδχΞʹඞਢ ‣ ίϯςφ͍ͬͺ্͍͛ͯɺշదʹTwitter͢Δͷॏཁ
׳Εා͍
docker-compose ‣ Docker ToolboxͷҰ෦ʢݩʑfigʣ ‣ Dockerίϯςφ܈ͷߏΛYAMLͰཧ ‣ docker-compose up -d
Ͱىಈ
σʔλετΞDocker ‣ ϩʔΧϧͰσʔλετΞDockerίϯςφར༻ ‣ library/mysql, library/redis ‣ VagrantΑΓߴͰؾܰʹεΫϥοϓϏϧυ
DBϚΠάϨʔγϣϯॏཁ ‣ ڥἧͬͯσʔλෆඋ͋Δͱҙຯ͕ແ͍ ‣ FRESH! Ͱ goose ‣ https://bitbucket.org/liamstask/goose/ ‣
SQL͚ͩ͡Όͳ͘ɺGoͰϚΠάϨʔγϣϯॻ͚Δ
ecs-formation
ecs-formation ‣ https://github.com/stormcat24/ecs-formation ‣ docker-composeͷΑ͏ʹɺYAMLϑΝΠϧͰίϯςφ ͷߏΛཧ͢Δ ‣ ࣌ecs-cliͱ͔ແ͔ͬͨͷͰ࡞ͬͨ ‣ aws-sdk-goར༻
ecs-formation features ‣ Task Definitionsͷߋ৽ ‣ Clusterʹஔ͢ΔServiceͷߋ৽ ‣ Blue-Green Deploymentͷ࣮ߦ
Task Definitions (task/jawsug-api.yml) nginx: image: registry.jawsug.local:5000/jawsug/nginx:latest ports: - 80:80 environment:
SERVER_NAME: jawsug.example.com volumes: - /var/log/container/nginx:/var/log/nginx links: - api memory: 512 cpu_units: 512 essential: true
Services on cluster (service/jawsug-cluster.yml) api-service: task_definition: jawsug-api desired_count: 1
Blue Green Deployment (bluegreen/jawsug-cluster.yml) blue: cluster: jawsug-cluster-blue service: api-service autoscaling_group:
jawsug-cluster-blue green: cluster: jawsug-cluster-green service: api-service autoscaling_group: jawsug-cluster-green primary_elb: jawsug-api-primary standby_elb: jawsug-api-standby
ecs-formationͷӡ༻ ‣ ecs-formationઐ༻ͷϦϙδτϦΛͭ͘Δ ‣ Task, Service, BlueGreenͷఆٛͷYAML ‣ masterʹϚʔδ͞ΕΔͱTask Definitionߋ৽
‣ service updateHubot -> CircleCI͔Β
Update Task Definitions push merge PR webhook test update task
notification webhook ecs-formation repository Amazon ECS ※࠷৽ͷTask Definitionͷόʔδϣϯʹߋ৽͞ΕΔ
Update Service(Deploy) webhook test update service deploy webhook ecs-formation repository
Amazon ECS ※࠷৽ͷTask DefinitionͷόʔδϣϯΛར༻͠ɺServiceΛߋ৽͢Δ push deploy branch notification
Others
Others ‣ AMI ‣ Private Registry ‣ CircleCI + Docker
‣ Terraform ‣ Mackerel
EC2-Optimized AMI ‣ Current version 2015.09.b ‣ Amazon Linuxϕʔε ‣
Docker + ECS Agent ‣ ศར͚ͩͲࣾͰ໘ݟͯ͘Εͳͦ͞͏ͩͬͨ
Ubuntu ‣ FRESH!UbuntuΛ࠾༻ ‣ DockerͷΠϯετʔϧ → ηϧϑαʔϏε ‣ ECS AgentͷಋೖͱαʔϏεԽʢUpstartʣ
→ ηϧϑαʔϏε ‣ ੵۃతͳDockerͷΞοϓσʔτ → ਓப ‣ apt-get install docker-engine=1.9.0-0~trusty
Private Registry ‣ S3ΛόοΫΤϯυʹɺPrivate RegistryΛ࡞ΕΔ ‣ registry:2.2.0 ‣ konradkleine/docker-registry-frontend:v2
None
CircleCI + Docker ‣ ΞϓϦέʔγϣϯ1ϦϙδτϦʹ1Dockerfile ‣ nginxtd-agentͷΑ͏ͳϛυϧΣΞܥผ్ઐ ༻ϦϙδτϦ ‣ CircleCIͰmasterϏϧυ࣌ʹdocker
build + push
Build Docker Image push merge PR webhook test docker build
private registry docker push notification webhook
Terraform ‣ ΠϯϑϥߏஙͷͨΊͷΦʔέετϨʔγϣϯπʔ ϧ ‣ ଟ࠼ͳProviderΛఏڙ ‣ AWSͰͷΠϯϑϥߏஙʹར༻
Terraformཧͯ͠Δͷ ‣ EC2 ‣ Security Group ‣ Route53 (ΠϯλʔφϧυϝΠϯͷΈʣ ‣
ECS Cluster ‣ AutoScaling Groupͷىಈߏ
Terraformཧͷߟ͑ํ ‣ ӡ༻ʹΑͬͯঢ়ଶ͕มΘΔͷཧʹ͔ͳ͍ ‣ ELB ‣ AutoScaling Group ‣ εΫϥοϓʴϏϧυʹ͕͔͔࣌ؒΔͷ
‣ RDS ‣ ElastiCache ‣ EC2CloudinitͰߏ͢Δ ‣ Provisioning͠ͳ͍ɻ͍ࣺͯ
ΫϦςΟΧϧͳͷආ͚Δ ‣ Route53 ‣ Ͳ͔ͬͷϓϩδΣΫτͰɺϨϏϡʔ͕ܗ֚ԽˠηϧϑϚ ˠେࣄނ͕͋ͬͨΒ͍͠ ‣ roadworkerઐ༻ϦϙδτϦ༻ҙ͠ɺݫॏʹΫϩεϨ Ϗϡʔ͢Δ ‣
IAM
Terraformͷӡ༻ ‣ tfϑΝΠϧGHE্Ͱཧ ‣ tfstateS3্ʹอ࣋ ‣ શͯΛҰͭͷtfstateͰཧ͠ͳ͍ ‣ dev/shared/staging/production/load ͘Β͍
‣ CircleCI ‣ PRͰ terraform plan ͷࠩνΣοΫ ‣ planͰݫີͳνΣοΫͰ͖ͳ͍ͷͰͦΕͳΓʹ৺؟ඞཁ ‣ hubot -> CircleCIͰ terraform apply
Mackerel ‣ ࢹجຊతʹMackerel ‣ ݟ͍͢ɺ͖Ε͍ ‣ ࠷ۙDockerͷϝτϦΫε͕औΕΔΑ͏ʹͳͬͨ
Mackerel
Mackerel
Mackerel ˡίϯςφ୯Ґͷ ϝτϦΫε
࠷ޙʹॴײ ‣ ECSͪΖΜपลπʔϧἧ͖ͬͯͯɺDocker ຊ൪ӡ༻ͷෑډ͕େ͖͘Լ͕͍ͬͯΔ ‣ ΜͰΔͳΒͱΓ͋͑ͣࢼͯ͠ΈΑʁ
Thank you for listening