Mackerelʹ͓͚ΔCloud NativeͷऔΓΈͱνʔϜ༩͑ͨมԽCloudNative Days Tokyo 2020
View Slide
ࣗݾհ● ࠓҪ൏ਓ (id:hayajo_77 / @hayajo)● גࣜձࣾͯͳ● MackerelνʔϜ● SREhttps://developer.hatenastaff.com/entry/2019/06/10/120000
Mackerel?
Cloud Nativeͱ
“εέʔϥϒϧͳΞϓϦέʔγϣϯΛߏஙɾ࣮ߦ͢ΔͨΊͷೳྗΛ৫ʹͨΒ͢”https://github.com/cncf/toc/blob/master/DEFINITION.md
Cloud NativeΛ࣮ݱ͢Δʹ
2ͭͷΞϓϩʔν͕ඞཁ● εέʔϥϒϧͳΞϓϦέʔγϣϯͷߏங○ ճ෮ੑɺཧྗɺՄ؍ଌੑͷ͋ΔγεςϜͷߏங○ Cloud Native Trail Map, Cloud Native Landscape● εέʔϥϒϧͳΞϓϦέʔγϣϯͷӡ༻○ ΞϓϦέʔγϣϯͷ։ൃɺϦϦʔεɺӡ༻ϑϩʔͷ࣮ફ○ DevOpsϓϥΫςΟε
ࠓճͷ͓
Cloud NativeͷऔΓΈͱνʔϜʹ༩͑ͨӨڹ1. ίϯςφσβΠϯύλʔϯʹΑΔઃܭ2. ϞχλϦϯάαΠυΧʔͷ։ൃ3. OpenTelemetryϥΠϒϥϦͷ։ൃ4. CI/CDͷߏங5. Ҡߦ࣌ͷΧφϦΞϦϦʔε6. SLOΛఆٛ7. ϚωʔδυαʔϏεͷҠߦ8. Terraformͷ࠾༻9. োରԠԋश10. ίϯςφνΣοΫϦετͷ࡞ͦΕͧΕͷऔΓΈؔ࿈͍͋ͬͯ͠ΔͨΊɺհॱ࣮ࡍʹऔΓΜͩॱংΛө͍ͯ͠ΔͷͰ͋Γ·ͤΜɻ
1. ίϯςφσβΠϯύλʔϯʹΑΔઃܭ
1. ίϯςφσβΠϯύλʔϯʹΑΔઃܭ~ͬͨ͜ͱ~● ίϯςφԽͨ͜͠ͱͰΞϓϦέʔγϣϯઃܭ͕มԽ● ίϯςφσβΠϯύλʔϯʹΑΔઃܭΛߦ͏Α͏ʹͳͬͨ○ αΠυΧʔɺΞϯόαμɺΞμϓλͳͲ● ίϯςφ୯ҐͰؔ৺Λ͠ɺσβΠϯύλʔϯʹԊͬͨܗͰػೳՃػೳͷڞ௨Խɺૄ݁߹ԽΛਤͬͨ○ ࢹػೳΛՃ͢ΔΤʔδΣϯτϝοηʔδΩϡʔͱΓͱΓ͢ΔϓϩΩγͳͲ
1. ίϯςφσβΠϯύλʔϯʹΑΔઃܭ~νʔϜʹ༩͑ͨӨڹ~● ʮߏஙʯʹ༩͑ͨӨڹ○ ΞϓϦέʔγϣϯͷґଘػೳΛίϯςφʹΓग़͢͜ͱͰίʔυΛγϯϓϧʹͰ͖ͨ○ ґଘػೳΛ࠶ར༻Մೳͳίϯςφͱͯ͠ϞδϡʔϧԽͰ͖ͨ● ʮӡ༻ʯʹ༩͑ͨӨڹ○ ػೳ͝ͱʹϦιʔεΛͰ͖ɺΞϓϦέʔγϣϯಈ࡞ͷӨڹΛগͳ͘͢Δ͜ͱ͕Մೳͱͳͬͨ
2. ϞχλϦϯάαΠυΧʔͷ։ൃ
2. ϞχλϦϯάαΠυΧʔͷ։ൃ~ͬͨ͜ͱ~● ίϯςφͰಈ࡞͢ΔΞϓϦέʔγϣϯΛࢹ͢Δඞཁ͕͋Δ○ Ϧιʔεͷར༻ঢ়گ○ ΞϓϦέʔγϣϯґଘ͢ΔαʔϏεͷϝτϦοΫ● ઐ༻ͷϞχλϦϯάΤʔδΣϯτΛ։ൃ● αΠυΧʔίϯςφͱͯ͠σϓϩΠ
2. ϞχλϦϯάαΠυΧʔͷ։ൃ~νʔϜʹ༩͑ͨӨڹ~● ʮߏஙʯʹ༩͑ͨӨڹ○ αΠυΧʔʹ͢Δ͜ͱͰΞϓϦέʔγϣϯʹखΛೖΕΔ͜ͱͳ͘ࢹػೳΛՃ○ ӡ༻ΤϯδχΞʹΑͬͯσϓϩΠ͞Εͦͷઃఆʹด͍ͯͨ͡ࢹػೳ͕ɺ։ൃΤϯδχΞͰ؆୯ʹՃՄೳʹͳͬͨ● ʮӡ༻ʯʹ༩͑ͨӨڹ○ ࢹઃఆϑΝΠϧΛશʹɺઃఆ͢Δඞཁ͕ͳ͘ͳͬͨ○ σϓϩΠϑϩʔ͕ίϯςφͷσϓϩΠʹ౷Ұ͞Εͨ
3. OpenTelemetryϥΠϒϥϦͷ։ൃ
3. OpenTelemetryϥΠϒϥϦͷ։ൃ~ͬͨ͜ͱ~● ϞχλϦϯάαΠυΧʔΞϓϦέʔγϣϯ֎෦ ͔Βͷܭଌ● ΑΓৄ͍͠ܭଌΞϓϦέʔγϣϯ෦Ͱߦ͏ඞཁ͕͋Δ○ ΞϓϦέʔγϣϯΛվम͢Δ͜ͱʹͳΔ
3. OpenTelemetryϥΠϒϥϦͷ։ൃ~ͬͨ͜ͱ~● ʰmackerelexporter-goʱ○ https://github.com/mackerelio-labs/mackerelexporter-go● ΞϓϦέʔγϣϯ෦Ͱܭଌ͠ɺͦͷσʔλΛMackerelΤΫεϙʔτ͢ΔGoϥΠϒϥϦ
3. OpenTelemetryϥΠϒϥϦͷ։ൃ~νʔϜʹ༩͑ͨӨڹ~● ʮߏஙʯʹ༩͑ͨӨڹ○ ಛఆͷؔͷ࣮ߦ࣌ؒͳͲɺΞϓϦέʔγϣϯಈ࡞ʹؔΘΔ(APMͷΑ͏ͳ)ϝτϦοΫͷऔಘ͕༰қʹͳͬͨ● ʮӡ༻ʯʹ༩͑ͨӨڹ○ ΞϓϦέʔγϣϯ෦ͷڍಈʹৄ͘͠ͳ͍ӡ༻ΤϯδχΞͰΞϓϦέʔγϣϯͷৄࡉͳϞχλϦϯά͕࣮͘͢͠ͳͬͨ
4. CI/CDͷߏங
4. CI/CDͷߏங~ͬͨ͜ͱ~● ίϯςφӡ༻ʹ߹ΘͤͨCI/CDΛ࠶ߏங○ ैདྷͷCI/CDπʔϧͷӡ༻ɺڞ௨ج൫ͷσϓϩΠπʔϧ͔Βͷ٫తͱͯ͋ͬͨ͠● ίϯςφϦϙδτϦΛڥքʹBuildͱReleaseΛ͚ͯߟ͑Δ○ ૄ݁߹ʹ͢Δ͜ͱͰॊೈੑ্͕͕Γπʔϧͷબࢶ͕͕Δ
4. CI/CDͷߏங~ͬͨ͜ͱ~● Build Stage○ Ռ (ΞʔςΟϑΝΫτ) Λ࡞͢Δεςʔδ○ ςετɺίϯςφΠϝʔδͷϏϧυɺϦϙδτϦͷPUSH○ GitHub Actions (or Jenkins)● Release Stage○ ՌΛϦϦʔε͢Δεςʔδ○ PUSH͞ΕͨίϯςφΠϝʔδͷσϓϩΠ○ CodePipeline + CodeDeploy
GitHubGHAorJenkinsCodePipelineCodeDeployECSECRS3Build Stage Release Stage4. CI/CDͷߏங~ͬͨ͜ͱ~Test/BuildPush ImagePut Metadata Get MetadataStart Deployment DeployPull Image
4. CI/CDͷߏங~νʔϜʹ༩͑ͨӨڹ~● ʮӡ༻ʯʹ༩͑ͨӨڹ○ CI/CDπʔϧͷӡ༻ෛՙݮ○ Blue/GreenσϓϩΠͷ࠾༻○ ؆୯ͳϩʔϧόοΫ○ γϯϓϧͳϦϦʔεखॱͷ࣮ݱ
5. ҠߦظؒͷΧφϦΞϦϦʔε
5. ҠߦظؒͷΧφϦΞϦϦʔε~ͬͨ͜ͱ~● طଘγεςϜ͔ΒECSஈ֊తʹτϥϑΟοΫΛྲྀ༷ͯ͠ࢠΛݟͭͭҠߦ͍ͨ͠● ALBͷWeightedRoutingΛར༻࣮ͯ͠ݱ○ ҠߦରͷALBʹECS༻ͷλʔήοτάϧʔϓΛ࡞͠ɺECSλεΫWeightedRouting͢Δ● σϓϩΠ࣌ʹCodePipelineͷLambdaͰઃఆ
5. ҠߦظؒͷΧφϦΞϦϦʔε~ͬͨ͜ͱ~CodePipelineCodeDeployECSLambdaALBTGFor EC2TGFor ECSStart DeploymentDeployRunCollectTask IPsModifyListener RuleRegisterTask IPsWeighted Routing
5. ҠߦظؒͷΧφϦΞϦϦʔε~νʔϜʹ༩͑ͨӨڹ~● ʮӡ༻ʯʹ༩͑ͨӨڹ○ ECSҠߦظؒʹ͓͚ΔϦϦʔεͷϦεΫ͕ݮͬͨ■ ෆ۩߹͕͋ͬͨ߹Ͱૣ͍ஈ֊Ͱؾͮ͘͜ͱ͕Ͱ͖ͨ■ ΞϓϦέʔγϣϯઃఆΛΓସ͑Δඞཁͳ͘ɺWeightedRoutingͷൺΛઃఆͯ͠σϓϩΠ͚ͩ͢͠Ͱ؆୯ʹݩͷঢ়گʹ͢͜ͱ͕Ͱ͖ͨ
6. SLOΛఆٛ
6. SLOΛఆٛ~ͬͨ͜ͱ~● SLO(αʔϏεϨϕϧඪ)Λఆٛ○ MackerelνʔϜͰSREΛ࣮ફ͓ͯ͠ΓɺνʔϜͰSLOΛܾΊ͍ͯΔ● νʔϜͰ߹ҙͷ্ͰɺఆྔతͳஅʹΑͬͯ։ൃɾӡ༻ΛਐΊΔ͜ͱ͕Ͱ͖Δ○ Cloud NativeͷҠߦγεςϜͷมߋ͕͏○ ࣮ӡ༻͍ͯ͠ΔαʔϏεΛ҆৺ͯ͠Cloud NativeʹҠߦ͍ͯ͘͠
6. SLOΛఆٛ~νʔϜʹ༩͑ͨมԽ~● ʮӡ༻ʯʹ༩͑ͨӨڹ○ Ҡߦ࣌ͷޭϨΠςϯγʹΑΔΓ͠அ■ ΧφϦΞϦϦʔεͷൺΓ͠ͳͲ○ ϞχλϦϯά͍ͨ͠ϝτϦοΫ(SLI)͕໌֬ʹͳͬͨ■ ϞχλϦϯάαΠυΧʔOpenTelemetryͰՄࢹԽ○ ࠓ·Ͱհͨ͠औΓΈ༰ͱSREͷݪଇ͕૬ޓʹ࡞༻ͯ͠ӡ༻ΛճͤΔΑ͏ʹͳͬͨ
7. ϚωʔδυαʔϏεͷҠߦ
7. ϚωʔδυαʔϏεͷҠߦ~ͬͨ͜ͱ~● γεςϜશମΛεέʔϥϒϧʹ͢ΔͨΊʹΞϓϦέʔγϣϯ͕ґଘ͢Δίϯϙʔωϯτεέʔϥϒϧʹ͢Δඞཁ͕͋Δ● ࣗલͰߏஙɺӡ༻ཧ͍ͯͨ͠ίϯϙʔωϯτΛϚωʔδυαʔϏεҠߦͨ͠
7. ϚωʔδυαʔϏεͷҠߦ~ͬͨ͜ͱ~● ʮAWS Purpose-Built Databases WeekʯͰElastiCacheͷҠߦࣄྫΛհ● ʰMackerelͷ࣌ܥྻσʔλϕʔεʹ͓͚ΔRedis Clusterͷར༻ͱAmazon ElastiCacheͷҠߦʹ͍ͭͯʱ● https://speakerdeck.com/astj/aws-purpose-built-databases-week
7. ϚωʔδυαʔϏεͷҠߦʙ νʔϜʹ༩͑ͨมԽ ʙ● ʮߏஙʯʹ༩͑ͨӨڹ○ ෳࡶͳϓϩϏδϣχϯάπʔϧɺσϓϩΠπʔϧ͔Βͷ٫○ CloudFormation, TerraformͳͲͷϚωʔδυαʔϏεͱੑͷߴ͍πʔϧʹΑͬͯߏஙָ͕ʹͳͬͨ● ʮӡ༻ʯʹ༩͑ͨӨڹ○ εέʔϥϒϧͰߴՄ༻ͳγεςϜΛ࣮ݱ○ ཧʹඞཁͳखॱೳྗ͕গͳ͘ͳͬͨ○ ΩϟύγςΟϓϥϯχϯά͕Γ͘͢ͳͬͨ
8. Terraformͷ࠾༻
8. Terraformͷ࠾༻~ͬͨ͜ͱ~● ͜Ε·ͰෳͷπʔϧΛͬͯΠϯϑϥߏங͍ͯͨ͠○ CloudFormation, Chef, CapistranoͳͲ● ϚωʔδυαʔϏεҠߦ͢Δ͜ͱͰIaC͍͕͢͠Ͱ͖ͨ● ϚϧνΫϥυରԠΛߟྀͯ͠TerraformΛ࠾༻ͨ͠
8. Terraformͷ࠾༻~ͬͨ͜ͱ~● MackerelνʔϜͰͷTerraform׆༻ͷ༷ࢠʮHashiTalks: Japan2020ʯͰհ○ ʰTerraformʹΑΔϓϥΫςΟεͷͱվળʱ○ https://speakerdeck.com/heleeen/our-best-practice
8. Terraformͷ࠾༻~νʔϜʹ༩͑ͨมԽ~● ʮߏஙʯʹ༩͑ͨӨڹ○ σϓϩΠ·ΘΓϚωʔδυαʔϏεʹҠߦͨ͜͠ͱͰ։ൃΤϯδχΞʹΘ͔Γ͍͢ίʔυԽ͕ਐΜͩ○ ෳͷڥͰಉ͡Α͏ʹ࡞͢ΔϦιʔεఆٛΛϞδϡʔϧͱͯ͠ڞ௨Խ͢Δ͜ͱͰߏஙͷෛ୲͕ݮͬͨ● ʮӡ༻ʯʹ༩͑ͨӨڹ○ γεςϜߏΛϦϙδτϦཧͯ͠։ൃϑϩʔʹࡌͤΔ͜ͱͰɺ։ൃΤϯδχΞͰมߋ͘͢͠ͳͬͨ
9. োରԠԋश
9. োରԠԋश~ͬͨ͜ͱ~● ίϯςφӡ༻ϚωʔδυαʔϏεͷҠߦʹΑΓɺγεςϜͷཧɾӡ༻༷มΘΓ͖ͯͨ͠● ։ൃΤϯδχΞΛަ͑ͯোԋशΛ࣮ࢪ͠ɺΞʔΩςΫνϟͱӡ༻खॱɺোରԠखॱΛֶͿ͜ͱͰͦͷཧղΛଅਐͨ͠
9. োରԠԋश~ͬͨ͜ͱ~● ͜Ε·ͰͷোରԠԋशͷऔΓΈͪ͜Βͷࢿྉࢀর○ ʮϓϩμΫτΛ10ӡ༻͢Δ νʔϜΛͭ͘Δʯ○ https://speakerdeck.com/daiksy/devsumi2020?slide=39
9. োରԠԋश~νʔϜʹ༩͑ͨมԽ~● ʮӡ༻ʯʹ༩͑ͨӨڹ○ ίϯςφԽϚωʔδυαʔϏεͷར༻ͰɺϦιʔεͷ࠶ߏஙखॱಛఆͷϛυϧΣΞͷਂ͍ࣝͳͲɺҎલʹൺΔͱӡ༻ʹඞཁͳखॱೳྗ͕গͳ͘ͳͬͨͱ࣮ײͰ͖ͨ○ ଐਓੑ͕ഉআ͞ΕͯνʔϜʹ৽͘͠ՃΘͬͨϝϯόʔͷΦϯϘʔσΟϯά͕εϜʔζʹͳͬͨ
10. ίϯςφνΣοΫϦετͷ࡞
10. ίϯςφνΣοΫϦετͷ࡞~ͬͨ͜ͱ~● ଞνʔϜͰίϯςφӡ༻ͷҠߦ͕૿͖͑ͯͨ● ͯ͢ͷνʔϜʹίϯςφӡ༻ʹৄ͍͠ΤϯδχΞ͕͍ΔΘ͚Ͱͳ͍● ίϯςφӡ༻ͷUnknown unknownsΛͳͨ͘͢ΊɺଞνʔϜͷΤϯδχΞͱڠྗͯ͠ίϯςφӡ༻ͷݟΛνΣοΫϦετͱͯ͠·ͱΊͨ
10. ίϯςφνΣοΫϦετͷ࡞~ͬͨ͜ͱ~
10. ίϯςφνΣοΫϦετͷ࡞~͜ͷऔΓΈͰظ͢ΔมԽ~● ৽نίϯςφӡ༻ʹ͚ͨॳಈָ͕ʹͳΔ● طଘͷίϯςφӡ༻ʹ͓͚Δߟྀ࿙Ε࣮ݱϨϕϧΛ֬ೝͰ͖Δ● ৫શମͰCloud NativeʹऔΓΉʹɺϊϋࣄྫΛࣾͰڞ༗͢Δ͜ͱ͕ॏཁ
νʔϜ༩͑ͨมԽͷ·ͱΊ
ଐਓੑͷഉআ● औΓΈΛਐΊΔ͜ͱͰ։ൃΤϯδχΞɺӡ༻ΤϯδχΞͦΕͧΕͷྖҬ͕ࣗવʹ͍ۙͮͯΦʔόʔϥοϓ͢Δ෦͕૿͖͑ͯͨ○ ίϯςφσβΠϯύλʔϯʹΑΔઃܭՄ؍ଌੑͷͨΊͷϥΠϒϥϦͷ։ൃɺIaCͳͲɺίϯςφΛத৺ͱͨ͠Cloud NativeπʔϧͰ͜ΕΛ࣮ݱ͘͢͠ͳ͍ͬͯΔ● ։ൃ͕εέʔϥϒϧʹͳ͖͍ͬͯͯΔ
ߏஙɾӡ༻ෛՙͷݮ● ϚωδϝϯταʔϏεͷҠߦCloud Nativeπʔϧͷ׆༻ͰεέʔϥϒϧͳγεςϜͷߏங͕؆୯ʹͳͬͨ● ίϯςφΛϕʔεͱͨ͠ճ෮ੑͷ͋ΔߴՄ༻ͳγεςϜϦϦʔε࡞ۀͷվળʹΑΓɺӡ༻ෛՙ͕ݮͨ͠
Cloud NativeͱSREͷ૬ޓ࡞༻● SLOΛఆٛ͢Δ͜ͱͰఆྔతͳஅͰ҆৺ͯ͠Cloud NativeΛਐΊΔ͜ͱ͕Ͱ͖Δ● Cloud NativeΛਐΊΔ͜ͱͰՄ؍ଌੑ͕ߴ·ΓɺSLOͷఆٛΛ࢝Ίͱͨ͠SREΛ͞Βʹਪਐ͠ɺνʔϜΛεέʔϥϒϧʹͰ͖Δ● ͦΕͧΕ͕࡞༻͠߹͍ɺੵۃతʹมԽ͍ͯͨ͘͠Ίͷ͕Ͱ͖ͭͭ͋Δ
·ͱΊ
·ͱΊ● Cloud Nativeͷ࣮ݱʹ2ͭͷΞϓϩʔν͕ඞཁ○ εέʔϥϒϧͳΞϓϦέʔγϣϯͷߏங○ εέʔϥϒϧͳΞϓϦέʔγϣϯͷӡ༻● ྆ํΛ܁Γฦ͠ܧଓͯ͠औΓΉ͜ͱͰνʔϜʹมԽ͕ੜ·ΕΔ● มԽ͕ੵΈॏͳΔ͜ͱͰCloud NativeͳνʔϜʹͳΔ
curl -sIL mackerel.io | grep career
hayajo
inureyes
takasehideki
chumajin
niftycorp
nttcom
hygradme
masa5555
yuya4
fujiwara3
rilmayer
ufried
i35_267
jakevdp
paigeccino
lara
shpigford
bcantrill
kneath
danielanewman
keithpitt
sonatard
tanoku
rmw
eileencodes