Stretcher in 5 min 2015-8-5 @fujiwara

Stretcher is Կ github.com/fujiwara/stretcher A deployment tool with Consul / Serf event. Consul / Serf ͱ࿈ܞͯ͠ಈ͘σϓϩΠπʔϧ (࣮͸ Consul/Serf ͳ͠Ͱ΋ಈ͖·͢)

ͳͥ࡞ͬͨͷ͔ (Ҏલ) rsync ʹΑΔதԝϗετ͔Βͷdeploy ΦʔτεέʔϧʹରԠͰ͖ͳ͍ ֤ϗετ͔ΒrsyncͰpull? → buildதʹrsync͞ΕͨΒ… ୆਺͕ଟ͍ͱฒྻssh + rsync͕ͭΒ͍ AMI࡞Γ௚͠ & ΠϯελϯεೖΕସ͑͸଴ͯͳ͍ inspired by github.com/sorah/mamiya, AWS CodeDeploy

ઃܭࢥ૝ AWS͡Όͳͯ͘΋ಈ͘ GoͰॻ͘(ࣗ਎ͷdeployָ͕) rsync͔ͯ͠ΒίϚϯυ࣮ߦɺͱ͍͏ϑϩʔ͸౿ऻ Consul eventͰΠϕϯτ௨஌ Consul ͱ͸ૄ݁߹ (consul watch Ͱىಈ)

Architecture

Deployment process 1. ΞϓϦέʔγϣϯΛbuildͯ͠tar.gzʹ͢Δ 2. खॱॻ(manifest)Λॻ͘ 3. tar.gz, manifestΛS3(or httpd)ʹ্͛Δ 4. consul event Ͱ manifest URLΛ௨஌ consul event -name deploy s3://... ✄--------- ͜͜·ͰstretcherͰ͸ͳ͍ -------✄

consul watch -type event -name deploy stretcher ✄------------ stretcher͔͜͜Β ------------✄ 1. event͔Βmanifest URLΛऔಘ 2. tar.gzΛऔಘͯ͠TMPDIRʹల։ 3. rsync -av --deleteͰߋ৽ 4. post command࣮ߦ • ΞϓϦέʔγϣϯ࠶ىಈͳͲ

Manifest src: s3://example.com/app.tar.gz checksum: e0840daaa97cd2cf2175f9e5d133ffb3324a2b93 dest: /home/stretcher/app commands: pre: - echo 'staring deploy' post: - echo 'deploy done' success: - cat >> /path/to/success.log failure: - cat >> /path/to/failure.log excludes: - "*.pid" - "*.socket"

๭ຊ൪؀ڥ AWS ্ͷ๭αʔϏε (50ʙ100 instances) tar.gz ~200MB consul event ૹ৴͔Β໿10ඵͰ׬ྃ (rollback΋!) 2015/08/05 14:58:30 Starting up stretcher agent 2015/08/05 14:58:30 Waiting for consul events from STDIN... 2015/08/05 14:58:30 Executing manifest: s3://... 2015/08/05 14:58:33 Extract archive: /dev/shm/stretcher539962129 to /dev/shm/stretcher_src648982332 2015/08/05 14:58:36 rsync [-av --delete --exclude-from /dev/shm/stretcher_src648982332/conf/rsync_exclude.web /dev/shm/stretcher_src648982332/ /home/xxx/web/] 2015/08/05 14:58:36 sending incremental file list ... sent 787780 bytes received 5230 bytes 1586020.00 bytes/sec total size is 359702435 speedup is 453.59 2015/08/05 14:58:36 invoking command: /home/xxx/web/refresh_services.sh 2015/08/05 14:58:41 success. 2015/08/05 14:58:41 Deploy manifest succeeded.

׬ྃΛ଴ͭ Πϕϯτૹ৴ݩͰ͸શ୆ͷ࣮ߦ׬͕ྃ෼͔Βͳ͍ • post commandͰ deploy ID Λ KV ʹૹ৴ • Consul KV Dashboard Ͱશ୆ͷ ID ͕ἧ͏ͷΛ଴ͭ

ϩάΛݟΔ ୆਺͕ଟ͍ͱ… • success/failure command Ͱ࣮ߦϩάΛ KV ʹ • Consul KV Dashboard Ͱঢ়ଶ͕มԽͨ͠ΒτϦΨˠ௨஌

Consul KV Dashboard github.com/fujiwara/consul-kv-dashboard speakerdeck.com/fujiwara3/consul-kv-dashboard

Demo stretcher/examples

Φʔτεέʔϧ΁ͷରԠ Consul KV ʹ࠷৽ͷ manifest URL Λอଘ Πϯελϯεىಈޙ 1. consul join 2. stretcher ࣮ߦ 3. ࠷৽ͷmanifest URLΛKV͔Βऔಘ 4. ࣗ෼ࣗ਎ʹ event Λૹ৴

ΦʔτεέʔϧͰͷ஫ҙ఺ × AMIʹ࢒͍ͬͯΔݹ͍ΞϓϦ͕ىಈ ○ ࠷৽ͷ deploy IDͰͳ͍৔߹͸ىಈ͠ͳ͍ deploy࣌: unique ͳ ID Λൃߦ • ϑΝΠϧʹॻ͍ͯ tar ʹೖΕΔ • Consul KV ʹ΋ೖΕΔ ىಈ࣌:ϩʔΧϧϑΝΠϧͷ ID ͱ KV Λൺֱ • ৯͍ҧͬͯͨΒ sleep 10 && exit → daemontoolsͰ࠶ىಈ

Serf ରԠ (v0.1.0~) serf agent -event-handler="user:deploy=stretcher" serf event deploy "s3://..." ؀ڥม਺ SERF_USER_EVENT Ͱಈ࡞੾Γସ͑ ͳͷͰ… echo "s3://..." | SERF_USER_EVENT=deploy stretcher ࣮͸ Consul / Serf Ͱͳͯ͘΋ىಈՄೳ

એ఻ YAPC::Asia 2015Ͱ΋ͬͱ͍Ζ͍Ζ࿩͠·͢ 2015-8-21 13:10~