Kubernetesの上に作る、統一されたマイクロサービス運用体験

SRE NEXT 2022 5/14(土) Room B 14:15 ~ 14:35 の発表資料です。

以下資料中で参照しているURL一覧です。

- https://cloud.google.com/config-connector/docs/overview
- https://cloud.google.com/blog/ja/products/containers-kubernetes/understanding-configuration-as-data-in-kubernetes
- https://cloud.google.com/spanner
- https://learning.oreilly.com/library/view/managing-kubernetes/9781492033905/ch03.html#fig0301
- https://github.com/mercari/spanner-autoscaler
- https://github.com/cloudspannerecosystem/autoscaler
- https://cloud.google.com/architecture/autoscaling-cloud-spanner
- https://raw.githubusercontent.com/cloudspannerecosystem/autoscaler/master/resources/architecture-per-project.png
- https://cloud.google.com/spanner/docs/backup
- https://speakerdeck.com/keisukeyamashita/spinnakerdeshi-jian-surumaikurosabisufalse-an-quan-naririsuhurotobesutopurakuteisu?slide=57

Transcript

1. ,VCFSOFUFTͷ্ʹ࡞Δɺ
   ౷Ұ͞ΕͨϚΠΫϩαʔϏεӡ༻ମݧ
   43&/&95
   4BU
   3PPN#
   !ULVDIJLJ
   

   View Slide

2. w!ULVDIJLJ
   wϝϧϖΠ43&
   wTQBOOFSBVUPTDBMFSͷϝϯ
   ςφ
   w(PQIFS
   8)0".*
   
   

   View Slide

3. "HFOEB
   wϝϧϖΠͷϚΠΫϩαʔϏεͷӡ༻
   w,VCFSOFUFTΧελϜίϯτϩʔϥʔΛ׆༻ͨ͠$MPVE
   4QBOOFSͷΦʔτεέʔϧ
   w43&ϚωʔδυίϯςφΠϝʔδΛ࢖༻ͨ͠$MPVE
   4QBOOFSͷόοΫΞοϓ
   wӡ༻ͷ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢ΔϝϦοτ
   w·ͱΊ
   

   View Slide

4. ϝϧϖΠͷϚΠΫϩαʔϏεͷӡ༻
   

   View Slide

5. wϚΠΫϩαʔϏεͰ͸εέʔϥϒϧͳ૊৫Λͭ͘ΔͨΊʹ֤νʔ
   ϜͰ։ൃ͔Βӡ༻·Ͱߦ͏
   wνʔϜ͝ͱʹӡ༻Λ೚ͤͯ͹Β͹Βͳ࢓૊ΈΛ࡞ͬͯ͠·͏͜ͱͰ
   ൃੜ͢Δ՝୊
   wπʔϧ։ൃίετͷ૿େ
   wڞ௨ͷӡ༻ϧʔϧΛద༻Ͱ͖ͳ͍
   w43&͕ϑΥϩʔͰ͖ͳ͍
   wڞ௨ͷϓϥοτϑΥʔϜ্ͰϚΠΫϩαʔϏεΛӡ༻͢Δ͜ͱͰղܾ
   ϚΠΫϩαʔϏεӡ༻ʹ͓͚Δ՝୊
   

   View Slide

6. ϝϧϖΠͷαʔϏεن໛
   .JDSPTFSWJDFT ,VCFSOFUFT1PET
   
   

   View Slide

7. ϝϧϖΠͷϚΠΫϩαʔϏεج൫
   w։ൃऀ͕ΦʔφʔγοϓΛ࣋ͬͯαʔϏεΛߏஙɾӡ
   ༻͢ΔͨΊͷϓϥοτϑΥʔϜ
   wϝϧΧϦͷ.JDSPTFSWJDFT1MBUGPSN5FBN͕ߏஙɾӡ༻
   w5FSSBGPSNͰ($1ϦιʔεΛ؅ཧ *B$
   
   w,VCFSOFUFT্ͷϦιʔεΛ:".-Ͱ؅ཧ $B%
   
   w$*ܦ༝ͰͷLVCFDUMBQQMZ
   w4QJOOBLFSͰσϓϩΠ
   *B$*OGSBTUSVDUVSFBT$PEF $B%$PO
   fi
   HVSBUJPOBT%BUB
   

   View Slide

8. $PO
   fi
   HVSBUJPOBT%BUB
   wΠϯϑϥΛ͋Δ΂͖ঢ়ଶΛએݴͨ͠σʔλͱ
   ͯ͠؅ཧ
   w,VCFSOFUFTͷ:".-ͳͲ
   w($1ͷϦιʔεΛ$B%Ͱ؅ཧ͢Δ,VCFSOFUFT
   ίϯτϩʔϥʔ΋͋Δ
   w$PO
   fi
   H$POOFDUPS
   IUUQTDMPVEHPPHMFDPNCMPHKBQSPEVDUTDPOUBJOFSTLVCFSOFUFTVOEFSTUBOEJOHDPO
   fi
   HVSBUJPOBTEBUBJOLVCFSOFUFT
   

   View Slide

9. ϝϧϖΠ43&ͷ໾ׂ
   wϚΠΫϩαʔϏεϓϥοτϑΥʔϜ্Ͱͷ
   ӡ༻αϙʔτ
   wϚΠΫϩαʔϏεڞ௨ͷΠϯϑϥͷӡ༻
   wӡ༻ʹ΋$B%ͷΑ͏ͳ࢓૊ΈΛఏڙ
   wຊηογϣϯͰ͸ͭͷࣄྫΛ঺հ
   $B%$PO
   fi
   HVSBUJPOBT%BUB
   

   View Slide

10. ,VCFSOFUFTΧελϜίϯτϩʔϥʔ
    Λ׆༻ͨ͠$MPVE4QBOOFSͷ
    Φʔτεέʔϧ
    

    View Slide

11. $MPVE4QBOOFS
    w($1ͷϑϧϚωʔδυϦϨʔγϣφϧɾ
    σʔλϕʔε
    wμ΢ϯλΠϜͳ͠Ͱ/PEFΛ૿ݮՄೳ
    wϝϧϖΠͷඪ४తͳσʔλϕʔεͱͯ͠ར༻
    IUUQTDMPVEHPPHMFDPNTQBOOFS
    

    View Slide

12. /PEFΛ૿ݮͤ͞Δӡ༻ϑϩʔ
    %FWFMPQFS
    1VMM3FRVFTU
    $MPVE4QBOOFS
    /PEFT
    UFSSBGPSNBQQMZPO$*
    

    View Slide

13. /PEFΛ૿ݮͤ͞Δӡ༻ϑϩʔ
    %FWFMPQFS
    1VMM3FRVFTU
    $MPVE4QBOOFS
    /PEFT
    UFSSBGPSNBQQMZPO$*
    8BJUGPS$*
    8BJUGPS$*
    5BLFBMPOHUJNF
    

    View Slide

14. $MPVE4QBOOFSͷӡ༻ʹ͓͚Δ՝୊
    w$MPVE4QBOOFSͷ$16࢖༻཰ߴಅʹΑΓϨΠςϯγ
    ͕૿Ճͯ͠΋/PEF௥Ճʹ͕͔͔࣌ؒΔ
    wͲͷ͘Β͍/PEF௥Ճ͢Δඞཁ͕͋Δͷ͔൑அ
    w1VMM3FRVFTUͷ࡞੒
    wίʔυϨϏϡʔͱ$*଴ͪ
    $16࢖༻཰ʹԠͯ͡Φʔτεέʔϧ͍ͤͨ͞
    

    View Slide

15. Φʔτεέʔϧͷݕ౼
    w($1ϓϩμΫτΛ૊Έ߹ΘͤΔ
    w$MPVE4DIFEVMFS
    w$MPVE'VODUJPOT
    w$MPVE.POJUPSJOH
    

    View Slide

16. Φʔτεέʔϧͷݕ౼
    w͜ͷ৔߹ɺͭͷӡ༻ํ๏͕ߟ͑ΒΕΔ
    w֤ϚΠΫϩαʔϏεͰӡ༻
    w($1ϓϩδΣΫτ͝ͱʹՔಇ͢Δ
    w($1ϓϩδΣΫτͰՔಇ͠ɺෳ਺ͷ($1ϓϩ
    δΣΫτͷ4QBOOFSΛ؅ཧ͢Δ
    w தԝूݖతʹ43&͕ӡ༻
    

    View Slide

17. Φʔτεέʔϧͷݕ౼
    w($1ϓϩμΫτͷ૊Έ߹ΘͤͰ΋΍Γ͍ͨ͜ͱ͸࣮
    ݱͰ͖Δ͕ʜ
    w֤ϚΠΫϩαʔϏεͰӡ༻
    w։ൃऀ͕ӡ༻͢Δ΋ͷ͕૿͑Δ
    wதԝूݖతͳӡ༻
    w43&ͷෛ୲͕େ͖͍
    w։ൃऀͷΦʔφʔγοϓΛଛͳ͏
    

    View Slide

18. Φʔτεέʔϧͷݕ౼
    w,VCFSOFUFTϦιʔεͱͯ͠؅ཧͰ͖Ε͹
    طଘͷӡ༻ϑϩʔʹ৐ͤΒΕΔ
    wϦιʔεͷϨϏϡʔɾσϓϩΠΛϚΠΫϩ
    αʔϏεͷνʔϜ಺Ͱ׬݁Ͱ͖Δ
    ,VCFSOFUFTΧελϜίϯτϩʔϥʔΛ࣮૷
    

    View Slide

19. ,VCFSOFUFTΧελϜίϯτϩʔϥʔ
    wϢʔβ͕ಠࣗʹఆٛͨ͠ϦιʔεΧελϜ
    Ϧιʔεʹର͢ΔॲཧΛߦ͏ίϯτϩʔϥʔ
    w3FDPODJMJBUJPOMPPQʹΑΓɺએݴͨ͠ঢ়ଶ
    ͱ࣮ࡍͷঢ়ଶΛൺֱ͠ɺ͋Δ΂͖ঢ়ଶʹಉ
    ظ͠ଓ͚Δ
    

    View Slide

20. 3FDPODJMJBUJPOMPPQ
    IUUQTMFBSOJOHPSFJMMZDPNMJCSBSZWJFXNBOBHJOHLVCFSOFUFTDIIUNM
    fi
    H
    

    View Slide

21. 3FDPODJMJBUJPOMPPQ
    wݱࡏͷ/PEF਺Λ͋Δ΂͖/PEF਺ʹಉظ
    ͢Δɺͱߟ͑Δͱ૬ੑ͕ྑͦ͞͏
    w)1"ʹࣅ͍ͯΔ
    )1")PSJ[POUBM1PE"VUPTDBMFS
    

    View Slide

22. TQBOOFSBVUPTDBMFS
    wIUUQTHJUIVCDPNNFSDBSJTQBOOFS
    BVUPTDBMFS
    w)1"ͷΑ͏ʹ$MPVE4QBOOFSͷ/PEFΛΦʔτ
    εέʔϧ͢Δ,VCFSOFUFTΧελϜίϯτϩʔ
    ϥʔ
    w೥݄ϦϦʔε
    )1")PSJ[POUBM1PE"VUPTDBMFS
    

    View Slide

23. TQBOOFSBVUPTDBMFSΛ׆༻ͨ͠ӡ༻
    43& %FWFMPQFS
    .BOBHF
    ,VCFSOFUFTΧελϜϦιʔε
    4QBOOFS"VUPTDBMFS
    ΧελϜϦιʔε
    LVCFDUMBQQMZ
    TQBOOFSBVUPTDBMFS
    $MPVE4QBOOFS
    4DBMFVQEPXO
    "VUPTDBMF
    DPO
    fi
    HVSBUJPO
    'PDVTPODPOUSPMMFSNBOBHFNFOU 0OMZNBOBHF:".-
    

    View Slide

24. ͜͜·Ͱͷ·ͱΊ
    w$MPVE4QBOOFSͷ/PEFΛ૿ݮͤ͞Δӡ༻ϑ
    ϩʔʹ՝୊͕͋ΓΦʔτεέʔϧͷඞཁੑΛ࣮ײ
    w,VCFSOFUFTΧελϜίϯτϩʔϥʔͷ
    3FDPODJMJBUJPOMPPQͷ࢓૊Έʹண໨
    w)1"ͷΑ͏ʹ$MPVE4QBOOFSΛΦʔτεέʔϧ
    ͢Δ,VCFSOFUFTΧελϜίϯτϩʔϥʔΛ࣮૷
    

    View Slide

25. DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS
    wIUUQTHJUIVCDPN
    DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS
    w($1ϓϩμΫτΛ૊Έ߹ΘͤͯΦʔτεέʔϧ
    w೥݄ࠒϦϦʔε
    w$16λεΫ༏ઌ౓ߴҎ֎ͷϝτϦΫεͰεέʔ
    ϦϯάՄೳ
    https://cloud.google.com/architecture/autoscaling-cloud-spanner
    

    View Slide

26. DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS
    https://raw.githubusercontent.com/cloudspannerecosystem/autoscaler/master/resources/architecture-per-project.png
    

    View Slide

27. DMPVETQBOOFSFDPTZTUFNBVUPTDBMFS
    wҎԼͷཧ༝ʹΑΓ࠾༻ͤͣ
    w($1ϓϩμΫτΛ૊Έ߹ΘͤΔ࣮૷Ͱڍ͛ͨ
    ໰୊͕ಉ༷ʹൃੜͦ͠͏
    w,VCFSOFUFTϦιʔεͱͯ͠Φʔτεέʔϧ
    ͷઃఆΛ؅ཧͰ͖Δํ͕ӡ༻ମݧ͕ྑ͍
    

    View Slide

28. 43&ϚωʔδυίϯςφΠϝʔδΛ
    ར༻ͨ͠$MPVE4QBOOFSͷ
    όοΫΞοϓ
    

    View Slide

29. $MPVE4QBOOFSͷόοΫΞοϓ
    w$MPVE4QBOOFSʹόοΫΞοϓΛࣗಈͰఆظతʹ࣮ߦ͢Δ
    ػೳ͸ͳ͍
    w$MPVE4DIFEVMFS͔Β"1*ϦΫΤετΛૹΔ͜ͱͰఆظతʹ
    ࣮ߦՄೳ
    wͭͷόοΫΞοϓํ๏ ύϑΥʔϚϯε
    Өڹ
    ϙʔλϏϦςΟ อ࣋ظؒ
    όοΫΞοϓ ͳ͠
    Πϯελϯεʹ
    ඥ͖ͮҠಈෆՄ
    ࠷௕೥ؒ
    ΤΫεϙʔτ ௿
    ೚ҙͷετϨʔδ্
    Ͱอ؅
    ࡟আ͢Δ·Ͱ
    https://cloud.google.com/spanner/docs/backup
    

    View Slide

30. $MPVE4QBOOFSͷόοΫΞοϓ
    w$MPVE4DIFEVMFS୯ମͰ΋όοΫΞοϓͷऔ
    ಘ͸Ͱ͖Δ͕
    wΤΫεϙʔτδϣϒࣦഊ࣌ʹ4MBDL௨஌͍ͨ͠
    wΤΫεϙʔτδϣϒͷ࣮ߦ࣌ؒΛܭଌ͍ͨ͠
    πʔϧͷ࣮૷Λݕ౼
    

    View Slide

31. πʔϧΛ࣮૷͢Δͱ͖ͷબ୒ࢶ
    w"QQ&OHJOF$SPOαʔϏε
    w$MPVE4DIFEVMFS$MPVE'VODUJPOT
    w$MPVE4DIFEVMFS$MPVE3VO
    

    View Slide

32. ӡ༻ޙʹൃੜͨ͠໰୊
    w։ൃऀ͸"QQ&OHJOFΛૢ࡞͢ΔݖݶΛ࣋
    ͨͳ͍ͷͰ43&͕ઃఆ
    wઃఆΛߋ৽͍ͨ͠৔߹΋43&ʹґཔ
    

    View Slide

33. 43&
    ӡ༻ޙʹൃੜͨ͠໰୊
    "QQ&OHJOF
    "QQ&OHJOF
    
    
    
    %FWFMPQFS %FQMPZ
    
    
    
    #PUUMFOFDL
    )FBWZXPSLMPBE
    3FRVFTU
    
    
    
    

    View Slide

34. ӡ༻ޙʹൃੜͨ͠໰୊ͷվળ
    w"QQ&OHJOFͷσϓϩΠύΠϓϥΠϯΛߏங
    ͢Δ͜ͱͰ໰୊ΛղܾͰ͖ͦ͏Ͱ͸͋Δ͕
    wόοΫΞοϓͷͨΊ͚ͩʹ࢓૊ΈΛ௥Ճ͢Δ
    ͷ͸େม
    w؅ཧ͢Δ΋ͷ͕෼ࢄ͢Δ
    

    View Slide

35. ӡ༻ޙʹൃੜͨ͠໰୊ͷվળ
    w4QJOOBLFS1JQFMJOFΛ:".-Ͱ࡞੒͢Δ࢓૊
    Έ͕͋Δ
    w,VCFSOFUFT$SPO+PCͰ࣮ߦ͢Ε͹ɺ1JQFMJOF
    ͷߏஙͱ+PCͷઃఆΛ:".-Ͱ׬݁Ͱ͖Δ
    wΞϓϦέʔγϣϯͱಉ͡Α͏ʹ؅ཧͰ͖Δ
    IUUQTTQFBLFSEFDLDPNLFJTVLFZBNBTIJUBTQJOOBLFSEFTIJKJBOTVSVNBJLVSPTBCJTVGBMTFBORVBOOBSJSJTVIVSPUPCFTVUPQVSBLVUFJTV TMJEF
    

    View Slide

36. ӡ༻ޙʹൃੜͨ͠໰୊ͷվળ
    w,VCFSOFUFT$SPO+PCͰ࣮ߦ͢ΔͨΊʹ͸
    ίϯςφΠϝʔδ͕ඞཁ
    wϚΠΫϩαʔϏε͝ͱʹΠϝʔδΛ࡞੒
    wઐ༻ͷ%PDLFS
    fi
    MFΛ࡞੒ͯ͠ΠϝʔδΛϏϧυ
    w43&͕࡞੒ͨ͠ΠϝʔδΛ֤ϚΠΫϩαʔϏ
    εͰ࢖༻
    

    View Slide

37. 43&ϚωʔδυͳΠϝʔδΛ࢖༻ͨ͠ӡ༻
    $POUBJOFS3FHJTUSZ
    %FWFMPQFS
    .BOBHF
    ,VCFSOFUFT$SPO+PC
    3VO$%1JQFMJOF
    %FQMPZ
    6TF*NBHF
    43&
    :".-
    'FUDI
    'PDVTPOJNBHFNBOBHFNFOU 0OMZNBOBHF:".-
    

    View Slide

38. ͜͜·Ͱͷ·ͱΊ
    w$MPVE4QBOOFSͷόοΫΞοϓΛ։ൃऀ͕
    ΦʔφʔγοϓΛ࣋ͬͯઃఆͰ͖ͳ͔ͬͨ
    w43&ͷෛՙ΋ߴ͔ͬͨ
    w։ൃऀࣗ਎ͰઃఆͰ͖ΔΑ͏ʹ؀ڥΛ੔උ
    w։ൃऀͷΦʔφʔγοϓΛଛͳΘͣɺ43&ͷෛ
    ՙΛܰݮ
    

    View Slide

39. ӡ༻ͷ࢓૊ΈΛ,VCFSOFUFT্ʹ
    ౷Ұ͢ΔϝϦοτ
    

    View Slide

40. ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢ΔϝϦοτ
    w:".-Λॻ͍ͯϦιʔεΛσϓϩΠ͢Δͩ
    ͚ͱ͍͏खܰ͞
    w։ൃऀ͕؅ཧ͢Δͷ͸:".-͚ͩ
    wπʔϧ͝ͱʹΩϟονΞοϓ͢΂͖͜ͱ͕গ
    ͳ͍ͷͰӡ༻ͷෛ୲͕ݮΔ
    

    View Slide

41. ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢ΔϝϦοτ
    wઃఆ࡞ۀ͕νʔϜ಺Ͱ׬݁͢ΔͨΊΦʔ
    φʔγοϓΛଛͳΘͳ͍
    wӡ༻ͷ࢓૊ΈΛઃܭ͢Δ্Ͱॏཁ
    wεέʔϥϒϧͳ૊৫Λͭ͘ΔͨΊʹ͸ඞཁ
    ෆՄܽ
    

    View Slide

42. ·ͱΊ
    

    View Slide

43. ຊηογϣϯͷ·ͱΊ
    wӡ༻ମݧ޲্ͷͨΊʹӡ༻ʹ$B%ͷ࢓૊ΈΛ
    औΓೖΕͨࣄྫΛ঺հ
    wӡ༻ͷ࢓૊ΈΛ,VCFSOFUFT্ʹ౷Ұ͢Δ͜ͱ
    ͰҎԼΛ࣮ݱ
    w:".-Λॻ͍ͯσϓϩΠ͢Δ͚ͩͷखܰ͞
    w։ൃऀͷΦʔφʔγοϓΛଛͳΘͳ͍ӡ༻؀ڥ
    $B%$PO
    fi
    HVSBUJPOBT%BUB
    

    View Slide