&$4Λར༻ͨ͠σϓϩΠ؀ڥ ,PIFJ4V[VLJ

.F • @eagletmt • ΫοΫύου ٕज़෦։ൃج൫άϧʔϓ • Docker Λར༻ͨ͠ΞϓϦέʔγϣϯαʔόج൫ͷ ։ൃɺӡ༻

Ξ΢τϥΠϯ • ΫοΫύουͱ Docker • Docker ΞϓϦͷσϓϩΠํ๏ v1 • ECS ҎલͷσϓϩΠํ๏ • Docker ΞϓϦͷσϓϩΠํ๏ v2 • ECS Λར༻ͨ͠σϓϩΠπʔϧ Hako ͷ։ൃ • Hako Λ࢖ͬͨσϓϩΠ؀ڥ

ΫοΫύουͱ%PDLFS • 2014೥9݄ࠒ͔Β Docker ͷར༻Λݕ౼ • ࣗ෼͕৽نʹ։ൃ͢Δখ͍͞ Web API ΞϓϦέʔ γϣϯΛύΠϩοτϓϩδΣΫτͱ͠ɺ12݄͔Βར ༻։࢝

ΫοΫύουͱ%PDLFS • 2015೥͔Βɺ৽نͷ Web ΞϓϦέʔγϣϯ͸جຊ తʹ Docker Λར༻ͨ͠؀ڥ (v1) Ͱಈ࡞ • 2016೥6݄ݱ࣌఺ʹ͓͍ͯɺ • ຊ൪Քಈ͍ͯ͠ΔαʔϏε͕ 11 ݸ • εςʔδϯά༻్ʹ 14 ݸ • ࣾ಺αʔϏεʹ 8 ݸ

ΫοΫύουͱ%PDLFS • 2015೥9݄ࠒ͔Β ECS Λར༻ͨ͠৽ߏ੒ (v2) Λݕ౼ • 2016೥1݄͔Βࣾ಺αʔϏεͰར༻։࢝ • 5݄͔Βຊ൪Ͱ΋ར༻։࢝͠ɺݱ࣌఺Ͱ • ຊ൪Քಈ͍ͯ͠ΔαʔϏε͕ 2 ݸ • εςʔδϯά༻్ʹ 10 ݸ • ࣾ಺αʔϏεʹ 21 ݸ

%PDLFSΞϓϦͷσϓϩΠ ํ๏W

&$4Λ࢖͍࢝ΊΔલ • σϓϩΠํ๏Λߟ͑ɺϗετଆʹσϓϩΠεΫϦϓτ Λ༻ҙ͢Δ͜ͱʹͨ͠ • ඞཁͳ͜ͱ • Web ΞϓϦέʔγϣϯͷσϓϩΠΛແఀࢭͰߦ͏ • σϓϩΠ͸શ։ൃऀ͕ߦ͏͜ͱ͕Ͱ͖Δ • ։ൃऀʹ͸ൿಗ஋ (DB ύεϫʔυͳͲ) Λݟͤͳ͍ • ൿಗ஋͸؀ڥม਺ͱͯ͠౉͢

σϓϩΠํ๏ W • 1ͭͷϗετʹ1ͭ·ͨ͸ෳ਺ͷ Docker ίϯςφ • ϗετଆʹ nginx Λ1ͭىಈ͠ɺHost ϔομͰίϯςφ ʹৼΓ෼͚ • σϓϩΠ࣌͸ɺίϯςφΛ৽نʹىಈ͠ɺnginx ͷઃఆ Λมߋͯ͠Ϧϩʔυ͠ɺݹ͍ίϯςφΛఀࢭ͢Δ • ৄ͘͠͸ http://techlife.cookpad.com/entry/ 2015/04/20/134758

σϓϩΠํ๏ W OHJOY BQQTFSWFS DPOUBJOFS W EFQMPZTDSJQU

σϓϩΠํ๏ W OHJOY BQQTFSWFS DPOUBJOFS W DPOUBJOFS W EFQMPZTDSJQU docker run

σϓϩΠํ๏ W OHJOY BQQTFSWFS DPOUBJOFS W DPOUBJOFS W EFQMPZTDSJQU IFBMUIDIFDL

σϓϩΠํ๏ W OHJOY BQQTFSWFS DPOUBJOFS W DPOUBJOFS W EFQMPZTDSJQU nginx -s reload

σϓϩΠํ๏ W OHJOY BQQTFSWFS DPOUBJOFS W EFQMPZTDSJQU docker stop

WͷΑ͔ͬͨ఺ • Πϯϑϥ͕ Itamae ͰΞϓϦέʔγϣϯຖʹϓϩϏ δϣχϯά͢Δख͕ؒͳ͘ͳͬͨ • ڞ௨ͷϗετͷ্ʹ༷ʑͳίϯςφ͕ಈ͘ • ΞϓϦέʔγϣϯαʔόͷΦʔτεέʔϧ͕͠΍͢ ͘ͳͬͨ • શ͘ಉ͡ํ๏ͰɺͲΜͳΞϓϦέʔγϣϯ΋εέʔ ϧΠϯɾεέʔϧΞ΢τͰ͖Δ

Wͷ࢒೦ͩͬͨ఺ • σϓϩΠઌͷϗετͷࢦఆ͕ਓྗ • EC2 λάʹࢦఆ͍ͯͨ͠ • ৽نͷαʔϏεΛ௥Ճ͢Δͱ͖͸ɺۭ͍ͯͦ͏ͳΠϯε λϯεʹλάΛ͚ͭΔ͔ɺ৽نʹىಈ͢Δඞཁ͕͋ͬͨ • ΞϓϦέʔγϣϯαʔόҎ֎ʹ΋ඞཁͳ΋ͷ͸͋Δ • RDSɺRoute53ɺELBɺCapistrano ͷઃఆ • ؀ڥม਺ͷ௥Ճ

%PDLFSΞϓϦͷσϓϩΠ ํ๏W

&$4Խ • ECS Λར༻͢Δ͜ͱͰɺσϓϩΠઌͷ੍ޚΛ೚ͤ Δ͜ͱ͕Ͱ͖Δ • EC2 λά͔Βͷղ์ • ECS Խʹ͋ͨͬͯൿಗ஋ͷѻ͍Λߟ͑Δ • task definition ͸Ӆͭͭ͠ɺผͷετϨʔδ͔Β ൿಗ஋Λ஫ೖ͢Δඞཁ͕͋Δ

&$4Խ • Route53 ΍ͦͷଞͷઃఆ͸ ECS ͚ͩͰ͸׬݁͠ͳ ͍ • ECS Λར༻ͭͭ͠ɺൿಗ஋ͷ஫ೖ΍ؔ࿈͢ΔϦιʔ ε΍ઃఆΛૢ࡞͢ΔσϓϩΠπʔϧ͕ඞཁ • => Hako ͷ։ൃ

)BLP • Docker ίϯςφΛσϓϩΠ͢ΔͨΊͷπʔϧ • ࠓͷͱ͜Ζ ECS ͷΈʹରԠ • ΞϓϦέʔγϣϯͷఆٛΛ YAML Ͱهड़ • ൿಗ஋ΛؚΉ؀ڥม਺Λѻ͑Δ • σϓϩΠ࣌ʹ༷ʑͳॲཧΛࠩ͠ࠐΉ͜ͱ͕Ͱ͖Δ

ઃܭϙϦγʔ • Ͱ͖Δ͚ͩ੩తͳఆٛʹ͢Δ • ఆٛ (ઃఆ) ͸ YAML Ͱɺίʔυ͸ Ruby Ͱ • ఆٛ͸ͦͷ··ͰɺRuby Ҏ֎Ͱ΋࣮૷Ͱ͖Δܗ • ϓϥΨϒϧʹ͢Δ • Docker ʹඞཁͳ΋ͷͷΈίΞʹؚΊΔ • ͦΕҎ֎ͷίʔυ͸ϓϥάΠϯͱͯࠩ͠͠ସ͑Մೳʹ ͢Δ

ΞϓϦέʔγϣϯఆٛ scheduler: <<: !include schedulers/internal_default.yml desired_count: 1 app: image: 01234567890.dkr.ecr.us-east-1.amazonaws.com/camo cpu: 128 memory: 128 port: 8080 env: $providers: - <<: !include env_providers/etcenv.yml root: /envs/hako/camo CAMO_KEY: '#{CAMO_KEY}' CAMO_KEEP_ALIVE: 'true' additional_containers: front: !include containers/front.yml consul-register: !include containers/consul_register_internal.yml scripts: - type: jenkins_tag job: docker-camo - !include scripts/route53_subdomain_shared.yml - !include scripts/nginx_omniauth.yml - type: consul_register

σϓϩΠํ๏ W • Hako::Schedulers::Ecs • ECS ্ʹ serviceɺtask definitionɺELB Λ࡞੒ • σϓϩΠ࣌ʹ͸ɺtask definition ͱ service Λߋ৽ • ͓ͦΒ͘ ECS ͷී௨ͷ࢖͍ํ

σϓϩΠํ๏ W • YAML ͷఆٛͱ DescribeTaskDefinition ͷ݁ՌΛൺ ֱ • ඞཁʹԠͯ͡ RegisterTaskDefinition Ͱߋ৽ • CreateService / UpdateService Ͱࢦ͍ͯ͠Δ task definition Λߋ৽ • CreateService ࣌ʹ͸ ELB ΋࡞੒͢Δ

σϓϩΠํ๏ W &-# UBTLW TFSWJDF IBLP

σϓϩΠํ๏ W &-# UBTLW UBTLW TFSWJDF IBLP 3FHJTUFS5BTL%FpOJUJPO 6QEBUF4FSWJDF

σϓϩΠํ๏ W &-# UBTLW TFSWJDF IBLP

σϓϩΠํ๏ W ࣾ಺αʔϏε൛ • ͜ͷํ๏ͩͱ service ຖʹ ELB ͕ඞཁ • ࣾ಺޲͚ͷࡶαʔϏεͰ΋ຖճ ELB Λ࡞Δ…? • ڞ௨ͷ ELB Λ࢖͍ͭͭɺϦόʔεϓϩΩγͷઃఆΛ ͍͍͔Μ͡ʹมߋ͢ΔΑ͏ͳผͷϞʔυΛ࣮૷ • ࣾ಺ͷผͷج൫ʹґଘ͍ͯ͠ΔͷͰίʔυ͸ඇެ։ • Hako ͳΒσϓϩΠ෦෼͚ͩࠩ͠ସ͑Δ͜ͱ͕Մೳ

શମߏ੒ W ࣾ಺αʔϏε൛ OHJOY &-# UBTL UBTL UBTL UBTL SFWFSTFQSPYZ DPOUBJOFS JOTUBODFT

σϓϩΠํ๏ W ࣾ಺αʔϏε൛ OHJOY UBTLW DPOTVM IBLP TFSWJDFW

σϓϩΠํ๏ W ࣾ಺αʔϏε൛ OHJOY UBTLW DPOTVM IBLP TFSWJDFW UBTLW TFSWJDFW 3FHJTUFS5BTL%FpOJUJPO $SFBUF4FSWJDF

σϓϩΠํ๏ W ࣾ಺αʔϏε൛ OHJOY UBTLW DPOTVM IBLP TFSWJDFW UBTLW TFSWJDFW SFHJTUFS

σϓϩΠํ๏ W ࣾ಺αʔϏε൛ OHJOY UBTLW DPOTVM IBLP TFSWJDFW UBTLW TFSWJDFW DPOTVMUFNQMBUF nginx -s reload

σϓϩΠํ๏ W ࣾ಺αʔϏε൛ OHJOY DPOTVM IBLP UBTLW TFSWJDFW %FMFUF4FSWJDF EFSFHJTUFS DPOTVMUFNQMBUF nginx -s reload

ൿಗ஋ͷѻ͍ • ؀ڥม਺ʹม਺ͷΑ͏ʹຒΊࠐΉ͜ͱ͕Ͱ͖Δ • DATABASE_URL: 'mysql2://#{user}:#{pass}@db- nanika-001/nanika' • ม਺͸σϓϩΠ࣌ʹผͷετϨʔδ͔Βऔಘ • ͨͱ͑͹ etcenv Λ࢖͏৔߹ • https://github.com/eagletmt/hako-etcenv • user ͷ৔ॴʹ /hako/nanika/user ͷ஋͕ೖΔ

)BLPΛ࢖ͬͨ σϓϩΠ؀ڥ

)BLPΛ࢖ͬͨσϓϩΠ؀ڥ • ΞϓϦέʔγϣϯఆ͚ٛͩΛؚΉϦϙδτϦ hako_apps Λ༻ҙ • ؀ڥม਺ͳͲΛมߋ͍ͨ͠ͱ͖͸ɺ։ൃऀ͕ࣗ༝ʹ pull-requestɺmerge • ൿಗ஋Λ௥Ճ͍ͨ͠ͱ͖ͷΈΠϯϑϥʹґཔ

)BLPΛ࢖ͬͨσϓϩΠ؀ڥ • σϓϩΠίϚϯυͷ࣮ߦʹ͸ Rundeck Λར༻ • app αʔόͰ࣮ߦ͢ΔίϚϯυ͕ແ͍ͷͰɺ Capistrano Λ࢖͏ඞཁ͕ͳ͍ • Rundeck ্ͰδϣϒΛ࣮ߦ͢Δͱɺ • git clone git@repository:hako_apps.git • cd hako_apps && hako deploy nanika.yml

όονॲཧ • hako oneshot Ͱ service Λ࢖ΘͣʹɺRunTask Ͱ୯ ൃͷλεΫΛ࣮ߦͰ͖Δ • ࣾ಺Ͱ͸ kuroko2 ͷλεΫͱ࣮ͯ͠ߦͰ͖ΔΑ͏ʹ ੔උ • kuroko2 ʹ͍ͭͯ͸ http://techlife.cookpad.com/ entry/2015/12/07/195732

σϓϩΠ࣌ͷϑοΫ • Hako Ͱ͸ script ͱݺশ • σϓϩΠ࣌ʹ΍ͬͯΔ͜ͱ • Route53 ͷࣗಈઃఆ • nginx ίϯςφͷΞΫηε੍ݶͷઃఆ • σϓϩΠ͢ΔΠϝʔδͷϦϏδϣϯΛ Jenkins ͷ݁Ռ ͔Βܾఆ • ౳ʑ

Φʔτεέʔϧ • AutoScaling Group Λར༻͠ɺΫϥελ͝ͱʹΠϯ ελϯεΛ૿ݮͰ͖ΔΑ͏ʹ • ࠓͷͱ͜ΖεέʔϧΠϯ͸ͤͣɺσϓϩΠ࣌ (service ͷ࡞੒ɺߋ৽࣌) ʹΠϯελϯεͷΩϟύ γςΟ͕଍Γͯͳ͔ͬͨΒɺAutoScaling Group ͷ ΩϟύγςΟΛ্͛ͯεέʔϧΞ΢τ

ϩΪϯά • Docker ͷ log driver ΛࢦఆͰ͖Δ • fluentd log driver Λ࢖ͬͯɺϗετଆʹ fluentd Λཱͯ ͯɺ͔ͦ͜Β CloudWatch Logs ΁సૹ • service ໊΍ίϯςφ໊Λ log stream ͷ໊લʹؚΊΔ Α͏ʹ͍ͯ͠Δ • ద౰ͳΞϓϦͰαʔϏε͝ͱʹϩάΛӾཡͰ͖ΔΑ͏ʹ

)BLPʹͳͬͯΑ͔ͬͨ఺ • ΠϯελϯεΛखಈͰ࡞੒͠ͳͯ͘Α͍ • AutoScaling Group ͷΩϟύγςΟΛ૿΍͚ͩ͢ • ΞϓϦέʔγϣϯͷఆٛΛɺ։ൃऀ͕ࣗ༝ʹӾཡɾߋ৽ Ͱ͖ΔΑ͏ʹͳͬͨ • ґཔͰ͸ͳ͘ pull-request ʹ • Route 53 ͷઃఆͳͲɺయܕతͳ࡞ۀΛࣗಈԽͰ͖ͨ • ݁Ռɺؾܰʹࣾ಺ΞϓϦ͕࡞ΒΕΔΑ͏ʹͳͬͨ

ࠓޙͷల๬ • ϩΪϯάΛվળ͍ͨ͠ • ΞϓϦέʔγϣϯͷϩάΛଈ࠲ʹ֬ೝ͍ͨ͠ͱ͖ ͕͋Δ͕ɺݱঢ়Ͱ͖͍ͯͳ͍ • CloudWatch Logs ʹอଘ͢Δͷ͸ͦͷ··Ͱɺಉ ࣌ʹ଎ใΛϩʔΧϧϑΝΠϧʹॻ͘Α͏ʹ͢Δ?

ࠓޙͷల๬ • ECS Ҏ֎ͷαϙʔτ? • ͪΐ͏ͲઌिɺDocker 1.12 Ͱ orchestration ػೳ ͷڧԽ͕ൃද͞Εͨ • ͜ͷ service Λ Hako ͔Β੍ޚͰ͖ΔͱΑͦ͞͏ • https://blog.docker.com/2016/06/docker-1-12- built-in-orchestration/