Share
Embed
Transcript
47 slides
-
Agenda • Blue-Green Deployment with AWS ECS • Docker (cgroup)
Statics • Monitoring Docker with Mackerel -
Immutable Infrastructure • Immutable server • Stateless / Disposable •
ઃఆΛมߋ͢Δࡍ࠶࡞͢Δ • ίʔυσϓϩΠؚΉ -
Blue-Green Deployment Host Host Host AWS Host Host Host Load
Balancer Dispose Old hosts Create New hosts -
Pattern 1 • 2 Autoscaling Group • as ECS Cluster
• Blue / Green • TaskΛ֤άϧʔϓʹσϓϩΠ -
Pattern 2 • 1 Autoscaling Group • 2 ECS Tasks
• Blue / Green Auto Scaling Group -
Pattern 2 • ৽͍͠ίʔυΛ৽Taskͱͯ͠σϓϩΠ • Delete/CreateLoadBalancerListeners • Ұॠɺ Listener ͕ͳ͘ͳΔ
{ "containerDefinitions": [ … "portMappings": [ { "containerPort": 80, "hostPort": 80 } … ϙʔτΛͣΒ͢ -
Pros/Cons of Pattern 1 • ҆શɾ࣮֬ͳΓସ͑ • Ұ࣌తʹ྆ํͷάϧʔϓʹଓ͢Δ • Ϧιʔεޮѱ͍
• Auto Scaling Group͕ཁஆؾ -
ϗετଆ͔ΒͷϞχλϦϯά • cgroupͷϨϙʔτΛར༻ • /sys/fs/cgroup/ {cpuacct,
memory} • ଞʹblkio, cpu, devicesͳͲͷα ϒγεςϜ͕͋Δ(֤छ੍ݶͳͲͷ੍ ޚ༻) VCVOUVͷ߹ LFSOFMHFOFSJDEPDLFS -
CPU༻ • /sys/fs/cgroup/cpuacct/docker/{ίϯςφID}/cpuacct.stat • user … ϢʔβʔϞʔυͷCPU ࣌ؒ • system
… γεςϜϞʔυͷCPU ࣌ؒ • CPU࣌ؒͷ୯Ґ: USER_HZ (σϑΥϧτ10ms) user 480 system 116 -
cpuacctͷଞͷࢦඪ • cpuacct.usage: ૯CPU࣌ؒ (φϊඵ) echo 0 > /cgroup/cpuacct/cpuacct.usage
ͰϦηοτ • cpuacct.usage_percpu: ֤CPUͷCPU ࣌ؒ (φϊඵ) -
ϝϞϦ༻ • /sys/fs/cgroup/memory/docker/{ίϯςφID}/memory.stat cache 761856 rss 25681920 rss_huge
0 mapped_file 16384 pgpgin 12569 … total_cache 761856 total_rss 25681920 … ͍ΘΏΔ344STT NBQQFE@pMF -
memory.statৄࡉ • cache • rss • mapped_file
• pgpgin • pgpgout • swap • active_anon • inactive_anon • active_file • inactive_file • unevictable • hierarchical_memory _limit • hierarchical_memsw • total_ࢠάϧʔϓΛ ؚΉ ৄࡉIUUQTXXXLFSOFMPSHEPD%PDVNFOUBUJPODHSPVQTNFNPSZUYU -
memoryؔ࿈(ͦͷଞ) memory.usage_in_bytes ϝϞϦ༻ྔ memory.memsw.usage_in_bytes ϝϞϦ༻ྔͱεϫοϓྖҬͷ memory.max_usage_in_bytes ϝϞϦ࠷େ༻ྔ memory.memsw.max_usage_in_bytes εϫοϓ࠷େ༻ྔ memory.limit_in_bytes
ϝϞϦͷ੍ݶ memory.memsw.limit_in_bytes ϝϞϦͱεϫοϓ੍ݶ memory.failcnt ϝϞϦ੍ݶʹୡͨ͠ճ memory.memsw.failcnt ϝϞϦ੍ݶʹୡͨ͠ճ -
ڞ௨Ϩϙʔτ • tasks ࣮ߦதͷϓϩηεͷҰཡ(PID) • cgroup.procs ࣮ߦதͷεϨουάϧʔϓͷҰཡ(TGID) % cat /sys/fs/cgroup/memory/docker/…./cgroup.procs
31218 31307 % ps aux | egrep '(31218|31307)' root 31218 0.0 0.0 4408 164 ? Ss 13:15 0:00 /bin/sh -‐ c rackup -‐p4567 root 31307 0.0 5.2 281364 26336 ? Sl 13:15 0:06 ruby1.9.1 /usr/local/bin/rackup -‐p4567 -
mackerel-agent • Go Ͱॻ͔ΕͨެࣜΤʔδΣϯτ • Linux / Windows αϙʔτ •
ϗετεϖοΫͱϝτϦοΫΛऩू mackerelio/mackerel-agent -
How to Use • ίϚϯυҰൃ • ඞཁͳύεɾAPIKEYΛ͢ docker run -‐h
`hostname` \ -‐v /var/run/docker.sock:/var/run/docker.sock \ -‐v /var/lib/mackerel-‐agent/:/var/lib/mackerel-‐agent/ \ -‐v /proc/mounts:/host/proc/mounts:ro \ -‐v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro -‐e 'apikey=<APIKEY>‘ \ mackerel/mackerel-‐agent -
DockerFile(ൈਮ) FROM debian:jessie # setup mackerel-‐agent RUN apt-‐get
update \ .. && apt-‐get -‐y install mackerel-‐agent \ .. # setup docker plugin COPY mackerel-‐plugin-‐docker.rb /mackerel-‐plugin-‐docker.rb RUN echo [plugin.metrics.docker] \ >> /etc/mackerel-‐agent/mackerel-‐agent.conf RUN echo command = \"ruby /mackerel-‐plugin-‐docker.rb\" \ >> /etc/mackerel-‐agent/mackerel-‐agent.conf # boot mackerel-‐agent CMD /startup.sh -
Memcachedͷྫ % docker run -‐d -‐P \
-‐-‐name memcached -‐p 11211:11211 \ sylvainlasnier/memcached )PTU NBDLFSFM BHFOU NFNDBDIFE -
linkΦϓγϣϯͰଓ % docker run -‐i -‐t -‐h `hostname` \
-‐v /var/run/docker.sock:/var/run/docker.sock \ -‐v /proc/mounts:/host/proc/mounts:ro \ -‐v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \ -‐e 'apikey=<APIKEY>' -‐-‐link memcached:memcached \ mackerel-‐agent /bin/bash root@dev:/# env HOSTNAME=develop04.host.h MEMCACHED_PORT_11211_TCP_PORT=11211 MEMCACHED_PORT_11211_TCP_PROTO=tcp MEMCACHED_NAME=/tender_curie/memcached MEMCACHED_PORT=tcp://172.17.0.88:11211 MEMCACHED_PORT_11211_TCP_ADDR=172.17.0.88 MEMCACHED_PORT_11211_TCP=tcp://172.17.0.88:11211 ݱঢ়Pattern2’ͰରԠ -
ڥมͰΞυϨεࢦఆ root@dev:/# /usr/local/bin/mackerel-‐plugin-‐memcached \ -‐host=$MEMCACHED_PORT_11211_TCP_ADDR memcached.connections.curr_connections
10 1427507503 memcached.cmd.cmd_get 0 1427507503 memcached.cmd.cmd_set 0 1427507503 -
ϓϥάΠϯઃఆΛࠐΉ % docker run -‐h `hostname` \ -‐v
/var/run/docker.sock:/var/run/docker.sock \ -‐v /var/lib/mackerel-‐agent/:/var/lib/mackerel-‐agent/ \ -‐v /proc/mounts:/host/proc/mounts:ro \ -‐v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro -‐e 'apikey=<APIKEY>‘ \ -‐-‐link memcached:memcached \ -‐v /etc/mackerel-‐agent/conf.d:/etc/mackerel-‐agent/conf.d:ro -‐e ‘include=/etc/mackerel-‐agent/conf.d/*' \ mackerel/mackerel-‐agent % cat /etc/mackerel-‐agent/conf.d/memcached.conf [plugin.metrics.memcached] command = "/usr/local/bin/mackerel-‐plugin-‐memcached -‐ host=$MEMCACHED_PORT_11211_TCP_ADDR" -
·ͱΊ • ECSͰBlue-Green Deployment • Patten 1, 2, 2’ •
Monitoring Docker with Mackerel