©MIXI .*9*5&$)$0/'&3&/$& શੈքͰ੒௕͠ଓ͚ΔαʔϏεͷΠϯϑϥͱ։ൃͷཪଆ 7BOUBHFελδΦ ΈͯͶࣄۀ෦ 43&άϧʔϓ ਗ਼ਫ ܄

©MIXI "CPVU.F ਗ਼ਫ ܄ !JTBPTIJNJ[V ● ೥ʙ גࣜձࣾϛΫγΟʢݱ.*9*ʣ ● ೥ʙ 4/4ʮNJYJʯӡ༻ΤϯδχΞ ● ೥ʙ ϞϯελʔετϥΠΫ 43& ● ೥ʙ Ո଒ΞϧόϜ ΈͯͶ 43& ● ೥ʙ 43&άϧʔϓ Ϛωʔδϟʔ ि຤͸ࣾձਓਧ૗ָஂͰͷ׆ಈʢָஂ௕ɺτϩϯϘʔϯ໿೥ɺͨ·ʹࢦشऀʣɻ ΩϟϯϓͱΫϥϑτϏʔϧ͕޷͖ɻ 2

©MIXI Ո଒ΞϧόϜ ΈͯͶ 3

©MIXI Ո଒ΞϧόϜ ΈͯͶ 4 ύύɾϚϚ͕ࡱͬͨࢠͲ΋ͷࣸਅ΍ಈըΛɺ૆෕฼΍ ਌ੰͳͲট଴ͨ͠Ո଒͚ͩʹ؆୯ʹڞ༗Ͱ͖Δࣸਅɾ ಈըڞ༗ΞϓϦ ϑΥτϒοΫ ࣸਅϓϦϯτ ঎඼ͷྫ

©MIXI Ո଒ΞϧόϜ ΈͯͶ 5 ΈͯͶΈ·΋Γ(14 ΈͯͶग़ுࡱӨ ΈͯͶ೥լঢ় ΈͯͶίʔϧυΫλʔ

©MIXI Ո଒ΞϧόϜ ΈͯͶ ● ೥݄ϦϦʔε ● ݱࡏ͸ݴޠɾͷࠃͱ஍ҬͰα ʔϏεΛఏڙ ● ւ֎Ͱ͸ʮ'BNJMZ"MCVNʯͱ͍͏ ໊শͰల։த ● ೥݄೔ʹར༻ऀ਺͕ ສਓ˞ Λಥഁ ● ೔ຊࠃ಺Ͱ͸ϚϚ΍ύύͷ൒਺ͱͳ Δˋͷํ˞ ͕͝ར༻ 6 ˞J04ɾ"OESPJEä ΞϓϦొ࿥ऀ਺ɺϒϥ΢β൛ొ࿥ऀ਺ͷ߹ܭ ˞ʮΈͯͶʯొ࿥࣌ʹೖྗ͞Ε͓ͨࢠ͞·ͷ஀ੜ೔ͱްੜ࿑ಇলൃදʮਓޱಈଶ౷ܭʯ͔Βࢉग़ɻ೥݄࣌఺Ͱˋ

©MIXI γεςϜߏ੒ 7

©MIXI ΈͯͶΛࢧ͑ΔΫϥ΢υ 8 AWS EKS / EC2 / S3 / Aurora ElastiCache / SQS / SNS など Google Cloud BigQuery など େ൒͕"84Ͱߏ੒͞Ε͍ͯΔ

©MIXI "84ͷߏ੒ʢུ֓ਤʣ 9 Amazon EKS Amazon EC2 Amazon S3 Amazon Aurora Amazon ElastiCache Amazon CloudFront Application Load Balancer Amazon DynamoDB

©MIXI γεςϜߏ੒ ● "84ΞΧ΢ϯτ͸؀ڥ͝ͱʹ׬શʹ෼ׂ ● "84ͷϦιʔε͸5FSSBGPSNʹΑͬͯ؅ཧ ● "-#ɺ4ͷલ໘ʹ$MPVE'SPOUΛஔ͖ɺωοτϫʔΫܦ࿏ͷ࠷దԽͱΩϟογϡར༻ ● &,4Ϋϥελʔ͸൒೥ʹҰ౓Ξοϓσʔτ͢Δӡ༻ʢ৽چΫϥελʔೖΕସ͑ʣ ● ,VCFSOFUFTͷϦιʔε͸"SHP$%ͱ)FMN$IBSUʹΑͬͯ؅ཧ ● &$͸ׂҎ্͕εϙοτΠϯελϯεʢΦϯσϚϯυ͸͘͝Θ͔ͣʣ ● &$্Ͱ͸3BJMT"QQMJDBUJPOɺ4JEFLJR8PSLFS͕ಈ࡞ ● "84-PBE#BMBODFS$POUSPMMFSͱ5BSHFU(SPVQ#JOEJOHʹΑͬͯ"-#͔Β1PE·Ͱ ͷϧʔςΟϯάΛίϯτϩʔϧ ● "VSPSB͸(MPCBM%BUBCBTFΛར༻ʢৄࡉ͸ޙड़ʣ 10

©MIXI ϚϧνϦʔδϣϯߏ੒ 11

©MIXI ϚϧνϦʔδϣϯߏ੒ 12 "84 BQOPSUIFBTU "84 VTFBTU Ϣʔβʔ͔Β͍ۙϦʔδϣϯʹΞΫηε͢Δ͜ͱͰ ϨεϙϯελΠϜΛ୹͘͢Δ͜ͱ͕Ͱ͖Δ

©MIXI ϚϧνϦʔδϣϯߏ੒ʢσʔλϕʔεʣ 13 ap-northeast-1 us-east-1 Amazon EKS Amazon EC2 Amazon S3 Amazon Aurora Global Database Amazon Aurora Global Database Amazon EKS Amazon EC2 ௿ϨΠςϯγʢฏۉ ໿NTʣͰಉظ͞ΕΔ ಡΈऔΓΞΫηεͷΈରԠ ॻ͖ࠐΈɾಡΈऔΓΞΫηεରԠ

©MIXI ϚϧνϦʔδϣϯߏ੒ʹ͓͚Δ஫ҙ఺ ● ΈͯͶͰ͸Ϣʔβʔମݧͷ޲্Λ໨తͱͨ͠ϚϧνϦʔδϣϯ ○ ͍ΘΏΔ#$1ରࡦͰ͸ͳ͍ ● ֤ϦʔδϣϯͰಉ༷ͷ؂ࢹΛ͓͜ͳ͏ඞཁ͕͋Δ ○ *B$ʹΑͬͯखؒ͸͋·Γ͔͔Βͳ͍ ● Ϧʔδϣϯ΍";ʹΑͬͯఏڙ͞Εͳ͍αʔϏεɺΠϯελϯελ Πϓ͕͋Δ ● &$ͷΩϟύγςΟঢ়گ͸Ϧʔδϣϯɺ";ʹΑͬͯେ͖͘มΘΔ ● ϦʔδϣϯؒͰωοτϫʔΫো֐͕͋Δͱσʔλϕʔεͷಉظʹ େ͖ͳ஗Ԇ͕ى͖Δ 14

©MIXI αʔόʔΞϓϦέʔγϣϯͷ ։ൃ؀ڥ 15

©MIXI αʔόʔΞϓϦέʔγϣϯͷ։ൃ؀ڥ 16 Πϯλʔωοτ ։ൃऀ͝ͱʹ༻ҙ ͞Εͨ؀ڥ 開発者 44) ͱͯ΋γϯϓϧʹॻ͘ͱ͜Μͳײ͡ ͜͜Ͱ։ൃ

©MIXI αʔόʔΞϓϦέʔγϣϯͷ։ൃ؀ڥ 17 Πϯλʔωοτ ,VCFSOFUFT "͞Μͷ ։ൃ؀ڥ1PE #͞Μͷ ։ൃ؀ڥ1PE $͞Μͷ ։ൃ؀ڥ1PE &͞Μͷ ։ൃ؀ڥ1PE %͞Μͷ ։ൃ؀ڥ1PE Aさん Bさん Cさん Dさん Eさん 開発チーム

©MIXI ։ൃ؀ڥͷ࡞Γํ 18 εςοϓ̍ εςοϓ εςοϓ :".-ϑΝΠϧΛ (JU)VCʹϓογϡ ˞:".-ͷத਎͸44) ͷެ։伴 "SHP$%Ͱ4ZODૢ࡞Λ ͯ͠։ൃ؀ڥ͕ىಈ ˞਺ेඵޙʹ؀ڥ͕࡞੒ ͞ΕΔ 44)͢Δ ։ൃऀࣗ਎Ͱ࡞੒ɾ࡟আΛࣗ༝ʹߦ͏͜ͱ͕Մೳ

©MIXI खݩͷ.BD1$ ։ൃ؀ڥ΁ͷϩάΠϯपΓͷηΩϡϦςΟ 19 ,VCFSOFUFTΫϥελ ։ൃ؀ڥ 1PE ίϯςφ sshd kubectl exec netcat 22番ポート Πϯλʔωοτ ίϯςφ ConfigMap SSH公開鍵 ίϯςφ ίϯςφ ProxyCommand .ssh/config ssh 44)ͷϙʔτΛΠϯλʔωοτʹެ։͢Δඞཁ͸ͳ͍ ։ൃऀ ίϯςφ "84ೝূ

©MIXI ։ൃ؀ڥ͕৽͍͠ߏ੒ʹͳͬͯ ● Ҏલ͸ݸਓ͝ͱͷ&$ʹ44)ͯ͠%PDLFSΛར༻͍ͯͨ͠ ○ 44)ͷ伴؅ཧɺ5FSSBGPSNͷద༻ͱߏஙͷεςοϓ͕ଟ͔ͬͨ ■ :".-ͷ௥Ճɺ"SHP$%ͷ؅ཧը໘͔Βͷద༻ͱ؆୯ʹ ○ खॱ͕Θ͔ΓͮΒ͘43&ϝϯόʔͷαϙʔτ͕ඞཁʹͳͬͨΓ ■ ୭Ͱ΋؆୯ʹߏஙɾ࡟আͰ͖ΔΑ͏ʹ ○ ൪ϙʔτͷΠϯλʔωοτެ։ʢެ։伴ೝূͱ͸͍͑ةͳ͍ʣ ■ ϙʔτެ։ͳ͠Ͱɺ"84ͷೝূʢ440ʣͱެ։伴ೝূͷ૊Έ߹Θͤʹ ■ 74$PEF3FNPUF%FWFMPQNFOUʹ΋ରԠ ○ ϩά͕ཷ·Γ͗ͨ͢Γɺݕࡧͮ͠Β͔ͬͨΓ ■ (SBGBOB-PLJͰݕࡧ͠΍͘͢ 20

©MIXI ΦϒβʔόϏϦςΟͷ޻෉ 21

©MIXI ΦϒβʔόϏϦςΟ͸ͳͥඞཁʁ ● γεςϜͰԿ͕ى͖͍ͯΔͷ͔Λ೺Ѳ͢ΔͨΊ ● ,VCFSOFUFTͳͲίϯςφͷΦʔέετϨʔγϣϯ͕ར༻͞ΕΔ ͜ͱ͕૿͑ͨ݁ՌɺγεςϜ಺෦͕ෳࡶʹͳΓɺ֤ίϯϙʔωϯ τΛ؍ଌՄೳͳঢ়ଶʹ͢Δඞཁ͕͋Δ ● ϝτϦΫεɺϩάɺτϨʔεͳͲͷ৘ใ͸ଟ͋͘Γɺ໰୊ͷݪҼ Λௐࠪ͢ΔͨΊʹ͸ͦΕΒΛత֬ʹूܭ͠ɺՄࢹԽ͢Δ࢓૊Έ͕ ٻΊΒΕΔ 22

©MIXI ΦϒβʔόϏϦςΟͷ࣮ݱ 23 ϞόΠϧΞϓϦ New Relic Mobile Firebase Crashlytics αʔόʔΞϓϦ New Relic APM New Relic Browser ֤छϩά Grafana Loki Amazon Athena Google BigQuery ֤छϝτϦΫε Prometheus / Amazon Managed Service for Prometheus Amazon CloudWatch Grafana

©MIXI ΦϒβʔόϏϦςΟͷશମ૾ 24 Kubernetes Prometheus New Relic Rails newrelic_rpm Grafana Node Exporter Amazon CloudWatch Amazon Managed Service for Prometheus Grafana Loki Promtail Ϣʔβʔͷ ୺຤ New Relic Mobile ϞχλϦϯά؀ڥ Fluent Bit Amazon Kinesis Firehose Amazon S3 Amazon Athena GCS BigQuery ETLॲཧ σʔλͷྲྀΕ

©MIXI 43&Λ࣮ફ͢Δ্Ͱ΋ ΦϒβʔόϏϦςΟ͸ͱͯ΋େࣄ 25

©MIXI ·ͱΊ 26

©MIXI ·ͱΊ ● શੈքͰ ສਓҎ্ͷϢʔβʔʹར༻͞ΕΔαʔϏεͷγε ςϜߏ੒͕ࢀߟʹͳΕ͹޾͍Ͱ͢ɻ ● "VSPSB(MPCBM%BUBCBTFͷ͓͔͛ͰϚϧνϦʔδϣϯߏ੒Λ࡞ Δ͜ͱ͕Ͱ͖ɺੈքதͷϢʔβʔମݧΛվળͰ͖·ͨ͠ɻ ● ,VCFSOFUFTΫϥελ্ͷ։ൃ؀ڥ͸ηΩϡΞͰѻ͍΍͍͢؀ڥ ʹͳΓ·ͨ͠ɻ ● ΦϒβʔόϏϦςΟ͸ࠓ·ͰࢼߦࡨޡΛ܁Γฦ͠ɺ43&͚ͩͰͳ ͘։ൃऀશମͰར༻Ͱ͖Δ؀ڥΛ࡞Δ͜ͱ͕Ͱ͖·ͨ͠ɻ 27

©MIXI 8&"3&)*3*/( 28 ΈͯͶ ࠾༻ ݕࡧ ͞·͟·ͳϙδγϣϯͰืूதͰ͢ʂ