Upgrade to Pro — share decks privately, control downloads, hide ads and more …

全世界で成長し続けるサービスのインフラと開発の裏側 / MIXI TECH CONFERENCE 2023

全世界で成長し続けるサービスのインフラと開発の裏側 / MIXI TECH CONFERENCE 2023

MIXI TECH CONFERENCE 2023
2023年3月3日(金)
#mixitechcon

Isao Shimizu

March 03, 2023
Tweet

More Decks by Isao Shimizu

Other Decks in Technology

Transcript

  1. ©MIXI
    .*9*5&$)$0/'&3&/$&
    શੈքͰ੒௕͠ଓ͚ΔαʔϏεͷΠϯϑϥͱ։ൃͷཪଆ
    7BOUBHFελδΦ ΈͯͶࣄۀ෦ 43&άϧʔϓ
    ਗ਼ਫ ܄

    View Slide

  2. ©MIXI
    "CPVU.F
    ਗ਼ਫ ܄ !JTBPTIJNJ[V
    ● ೥ʙ גࣜձࣾϛΫγΟʢݱ.*9*ʣ
    ● ೥ʙ 4/4ʮNJYJʯӡ༻ΤϯδχΞ
    ● ೥ʙ ϞϯελʔετϥΠΫ 43&
    ● ೥ʙ Ո଒ΞϧόϜ ΈͯͶ 43&
    ● ೥ʙ 43&άϧʔϓ Ϛωʔδϟʔ
    ि຤͸ࣾձਓਧ૗ָஂͰͷ׆ಈʢָஂ௕ɺτϩϯϘʔϯ໿೥ɺͨ·ʹࢦشऀʣɻ
    ΩϟϯϓͱΫϥϑτϏʔϧ͕޷͖ɻ
    2

    View Slide

  3. ©MIXI
    Ո଒ΞϧόϜ ΈͯͶ
    3

    View Slide

  4. ©MIXI
    Ո଒ΞϧόϜ ΈͯͶ
    4
    ύύɾϚϚ͕ࡱͬͨࢠͲ΋ͷࣸਅ΍ಈըΛɺ૆෕฼΍
    ਌ੰͳͲট଴ͨ͠Ո଒͚ͩʹ؆୯ʹڞ༗Ͱ͖Δࣸਅɾ
    ಈըڞ༗ΞϓϦ
    ϑΥτϒοΫ
    ࣸਅϓϦϯτ
    ঎඼ͷྫ

    View Slide

  5. ©MIXI
    Ո଒ΞϧόϜ ΈͯͶ
    5
    ΈͯͶΈ·΋Γ(14 ΈͯͶग़ுࡱӨ
    ΈͯͶ೥լঢ় ΈͯͶίʔϧυΫλʔ

    View Slide

  6. ©MIXI
    Ո଒ΞϧόϜ ΈͯͶ
    ● ೥݄ϦϦʔε
    ● ݱࡏ͸ݴޠɾͷࠃͱ஍ҬͰα
    ʔϏεΛఏڙ
    ● ւ֎Ͱ͸ʮ'BNJMZ"MCVNʯͱ͍͏
    ໊শͰల։த
    ● ೥݄೔ʹར༻ऀ਺͕
    ສਓ˞
    Λಥഁ
    ● ೔ຊࠃ಺Ͱ͸ϚϚ΍ύύͷ൒਺ͱͳ
    Δˋͷํ˞
    ͕͝ར༻
    6
    ˞J04ɾ"OESPJEä ΞϓϦొ࿥ऀ਺ɺϒϥ΢β൛ొ࿥ऀ਺ͷ߹ܭ
    ˞ʮΈͯͶʯొ࿥࣌ʹೖྗ͞Ε͓ͨࢠ͞·ͷ஀ੜ೔ͱްੜ࿑ಇলൃදʮਓޱಈଶ౷ܭʯ͔Βࢉग़ɻ೥݄࣌఺Ͱˋ

    View Slide

  7. ©MIXI
    γεςϜߏ੒
    7

    View Slide

  8. ©MIXI
    ΈͯͶΛࢧ͑ΔΫϥ΢υ
    8
    AWS
    EKS / EC2 / S3 / Aurora
    ElastiCache / SQS / SNS など
    Google Cloud
    BigQuery など
    େ൒͕"84Ͱߏ੒͞Ε͍ͯΔ

    View Slide

  9. ©MIXI
    "84ͷߏ੒ʢུ֓ਤʣ
    9
    Amazon EKS
    Amazon EC2
    Amazon S3 Amazon Aurora
    Amazon
    ElastiCache
    Amazon
    CloudFront
    Application
    Load Balancer
    Amazon
    DynamoDB

    View Slide

  10. ©MIXI
    γεςϜߏ੒
    ● "84ΞΧ΢ϯτ͸؀ڥ͝ͱʹ׬શʹ෼ׂ
    ● "84ͷϦιʔε͸5FSSBGPSNʹΑͬͯ؅ཧ
    ● "-#ɺ4ͷલ໘ʹ$MPVE'SPOUΛஔ͖ɺωοτϫʔΫܦ࿏ͷ࠷దԽͱΩϟογϡར༻
    ● &,4Ϋϥελʔ͸൒೥ʹҰ౓Ξοϓσʔτ͢Δӡ༻ʢ৽چΫϥελʔೖΕସ͑ʣ
    ● ,VCFSOFUFTͷϦιʔε͸"SHP$%ͱ)FMN$IBSUʹΑͬͯ؅ཧ
    ● &$͸ׂҎ্͕εϙοτΠϯελϯεʢΦϯσϚϯυ͸͘͝Θ͔ͣʣ
    ● &$্Ͱ͸3BJMT"QQMJDBUJPOɺ4JEFLJR8PSLFS͕ಈ࡞
    ● "84-PBE#BMBODFS$POUSPMMFSͱ5BSHFU(SPVQ#JOEJOHʹΑͬͯ"-#͔Β1PE·Ͱ
    ͷϧʔςΟϯάΛίϯτϩʔϧ
    ● "VSPSB͸(MPCBM%BUBCBTFΛར༻ʢৄࡉ͸ޙड़ʣ
    10

    View Slide

  11. ©MIXI
    ϚϧνϦʔδϣϯߏ੒
    11

    View Slide

  12. ©MIXI
    ϚϧνϦʔδϣϯߏ੒
    12
    "84
    BQOPSUIFBTU
    "84
    VTFBTU
    Ϣʔβʔ͔Β͍ۙϦʔδϣϯʹΞΫηε͢Δ͜ͱͰ
    ϨεϙϯελΠϜΛ୹͘͢Δ͜ͱ͕Ͱ͖Δ

    View Slide

  13. ©MIXI
    ϚϧνϦʔδϣϯߏ੒ʢσʔλϕʔεʣ
    13
    ap-northeast-1 us-east-1
    Amazon EKS
    Amazon EC2
    Amazon S3
    Amazon Aurora
    Global Database
    Amazon Aurora
    Global Database
    Amazon EKS
    Amazon EC2
    ௿ϨΠςϯγʢฏۉ ໿NTʣͰಉظ͞ΕΔ
    ಡΈऔΓΞΫηεͷΈରԠ
    ॻ͖ࠐΈɾಡΈऔΓΞΫηεରԠ

    View Slide

  14. ©MIXI
    ϚϧνϦʔδϣϯߏ੒ʹ͓͚Δ஫ҙ఺
    ● ΈͯͶͰ͸Ϣʔβʔମݧͷ޲্Λ໨తͱͨ͠ϚϧνϦʔδϣϯ
    ○ ͍ΘΏΔ#$1ରࡦͰ͸ͳ͍
    ● ֤ϦʔδϣϯͰಉ༷ͷ؂ࢹΛ͓͜ͳ͏ඞཁ͕͋Δ
    ○ *B$ʹΑͬͯखؒ͸͋·Γ͔͔Βͳ͍
    ● Ϧʔδϣϯ΍";ʹΑͬͯఏڙ͞Εͳ͍αʔϏεɺΠϯελϯελ
    Πϓ͕͋Δ
    ● &$ͷΩϟύγςΟঢ়گ͸Ϧʔδϣϯɺ";ʹΑͬͯେ͖͘มΘΔ
    ● ϦʔδϣϯؒͰωοτϫʔΫো֐͕͋Δͱσʔλϕʔεͷಉظʹ
    େ͖ͳ஗Ԇ͕ى͖Δ
    14

    View Slide

  15. ©MIXI
    αʔόʔΞϓϦέʔγϣϯͷ
    ։ൃ؀ڥ
    15

    View Slide

  16. ©MIXI
    αʔόʔΞϓϦέʔγϣϯͷ։ൃ؀ڥ
    16
    Πϯλʔωοτ
    ։ൃऀ͝ͱʹ༻ҙ
    ͞Εͨ؀ڥ
    開発者
    44)
    ͱͯ΋γϯϓϧʹॻ͘ͱ͜Μͳײ͡
    ͜͜Ͱ։ൃ

    View Slide

  17. ©MIXI
    αʔόʔΞϓϦέʔγϣϯͷ։ൃ؀ڥ
    17
    Πϯλʔωοτ
    ,VCFSOFUFT
    "͞Μͷ
    ։ൃ؀ڥ1PE
    #͞Μͷ
    ։ൃ؀ڥ1PE
    $͞Μͷ
    ։ൃ؀ڥ1PE
    &͞Μͷ
    ։ൃ؀ڥ1PE
    %͞Μͷ
    ։ൃ؀ڥ1PE
    Aさん Bさん Cさん Dさん Eさん
    開発チーム

    View Slide

  18. ©MIXI
    ։ൃ؀ڥͷ࡞Γํ
    18
    εςοϓ̍ εςοϓ εςοϓ
    :".-ϑΝΠϧΛ
    (JU)VCʹϓογϡ
    ˞:".-ͷத਎͸44)
    ͷެ։伴
    "SHP$%Ͱ4ZODૢ࡞Λ
    ͯ͠։ൃ؀ڥ͕ىಈ
    ˞਺ेඵޙʹ؀ڥ͕࡞੒
    ͞ΕΔ
    44)͢Δ
    ։ൃऀࣗ਎Ͱ࡞੒ɾ࡟আΛࣗ༝ʹߦ͏͜ͱ͕Մೳ

    View Slide

  19. ©MIXI
    खݩͷ.BD1$
    ։ൃ؀ڥ΁ͷϩάΠϯपΓͷηΩϡϦςΟ
    19
    ,VCFSOFUFTΫϥελ
    ։ൃ؀ڥ 1PE
    ίϯςφ
    sshd
    kubectl exec netcat
    22番ポート
    Πϯλʔωοτ
    ίϯςφ
    ConfigMap
    SSH公開鍵
    ίϯςφ
    ίϯςφ
    ProxyCommand
    .ssh/config
    ssh
    44)ͷϙʔτΛΠϯλʔωοτʹެ։͢Δඞཁ͸ͳ͍
    ։ൃऀ
    ίϯςφ
    "84ೝূ

    View Slide

  20. ©MIXI
    ։ൃ؀ڥ͕৽͍͠ߏ੒ʹͳͬͯ
    ● Ҏલ͸ݸਓ͝ͱͷ&$ʹ44)ͯ͠%PDLFSΛར༻͍ͯͨ͠
    ○ 44)ͷ伴؅ཧɺ5FSSBGPSNͷద༻ͱߏஙͷεςοϓ͕ଟ͔ͬͨ
    ■ :".-ͷ௥Ճɺ"SHP$%ͷ؅ཧը໘͔Βͷద༻ͱ؆୯ʹ
    ○ खॱ͕Θ͔ΓͮΒ͘43&ϝϯόʔͷαϙʔτ͕ඞཁʹͳͬͨΓ
    ■ ୭Ͱ΋؆୯ʹߏஙɾ࡟আͰ͖ΔΑ͏ʹ
    ○ ൪ϙʔτͷΠϯλʔωοτެ։ʢެ։伴ೝূͱ͸͍͑ةͳ͍ʣ
    ■ ϙʔτެ։ͳ͠Ͱɺ"84ͷೝূʢ440ʣͱެ։伴ೝূͷ૊Έ߹Θͤʹ
    ■ 74$PEF3FNPUF%FWFMPQNFOUʹ΋ରԠ
    ○ ϩά͕ཷ·Γ͗ͨ͢Γɺݕࡧͮ͠Β͔ͬͨΓ
    ■ (SBGBOB-PLJͰݕࡧ͠΍͘͢
    20

    View Slide

  21. ©MIXI
    ΦϒβʔόϏϦςΟͷ޻෉
    21

    View Slide

  22. ©MIXI
    ΦϒβʔόϏϦςΟ͸ͳͥඞཁʁ
    ● γεςϜͰԿ͕ى͖͍ͯΔͷ͔Λ೺Ѳ͢ΔͨΊ
    ● ,VCFSOFUFTͳͲίϯςφͷΦʔέετϨʔγϣϯ͕ར༻͞ΕΔ
    ͜ͱ͕૿͑ͨ݁ՌɺγεςϜ಺෦͕ෳࡶʹͳΓɺ֤ίϯϙʔωϯ
    τΛ؍ଌՄೳͳঢ়ଶʹ͢Δඞཁ͕͋Δ
    ● ϝτϦΫεɺϩάɺτϨʔεͳͲͷ৘ใ͸ଟ͋͘Γɺ໰୊ͷݪҼ
    Λௐࠪ͢ΔͨΊʹ͸ͦΕΒΛత֬ʹूܭ͠ɺՄࢹԽ͢Δ࢓૊Έ͕
    ٻΊΒΕΔ
    22

    View Slide

  23. ©MIXI
    ΦϒβʔόϏϦςΟͷ࣮ݱ
    23
    ϞόΠϧΞϓϦ
    New Relic Mobile
    Firebase Crashlytics
    αʔόʔΞϓϦ
    New Relic APM
    New Relic Browser
    ֤छϩά
    Grafana Loki
    Amazon Athena
    Google BigQuery
    ֤छϝτϦΫε
    Prometheus / Amazon Managed
    Service for Prometheus
    Amazon CloudWatch
    Grafana

    View Slide

  24. ©MIXI
    ΦϒβʔόϏϦςΟͷશମ૾
    24
    Kubernetes
    Prometheus
    New Relic
    Rails
    newrelic_rpm
    Grafana
    Node Exporter
    Amazon CloudWatch
    Amazon
    Managed
    Service for
    Prometheus
    Grafana Loki
    Promtail
    Ϣʔβʔͷ
    ୺຤
    New
    Relic
    Mobile
    ϞχλϦϯά؀ڥ
    Fluent Bit Amazon Kinesis
    Firehose
    Amazon S3
    Amazon
    Athena
    GCS BigQuery
    ETLॲཧ
    σʔλͷྲྀΕ

    View Slide

  25. ©MIXI
    43&Λ࣮ફ͢Δ্Ͱ΋
    ΦϒβʔόϏϦςΟ͸ͱͯ΋େࣄ
    25

    View Slide

  26. ©MIXI
    ·ͱΊ
    26

    View Slide

  27. ©MIXI
    ·ͱΊ
    ● શੈքͰ ສਓҎ্ͷϢʔβʔʹར༻͞ΕΔαʔϏεͷγε
    ςϜߏ੒͕ࢀߟʹͳΕ͹޾͍Ͱ͢ɻ
    ● "VSPSB(MPCBM%BUBCBTFͷ͓͔͛ͰϚϧνϦʔδϣϯߏ੒Λ࡞
    Δ͜ͱ͕Ͱ͖ɺੈքதͷϢʔβʔମݧΛվળͰ͖·ͨ͠ɻ
    ● ,VCFSOFUFTΫϥελ্ͷ։ൃ؀ڥ͸ηΩϡΞͰѻ͍΍͍͢؀ڥ
    ʹͳΓ·ͨ͠ɻ
    ● ΦϒβʔόϏϦςΟ͸ࠓ·ͰࢼߦࡨޡΛ܁Γฦ͠ɺ43&͚ͩͰͳ
    ͘։ൃऀશମͰར༻Ͱ͖Δ؀ڥΛ࡞Δ͜ͱ͕Ͱ͖·ͨ͠ɻ
    27

    View Slide

  28. ©MIXI
    8&"3&)*3*/(
    28
    ΈͯͶ ࠾༻ ݕࡧ
    ͞·͟·ͳϙδγϣϯͰืूதͰ͢ʂ

    View Slide