Kubernetes拡張を利用した自作Autoscalerで実現するストレスフリーな運用の世界

Transcript

1. ,VCFSOFUFT֦ுΛར༻ͨࣗ͠࡞"VUPTDBMFSͰ ࣮ݱ͢ΔετϨεϑϦʔͳӡ༻ͷੈք .BZV
   :BNBNPUP
   c
   $ZCFSBHFOU
   *OD $MPVE
   /BUJWF
   %BZT
   5PLZP
   +VMZ
   
   
2. c
   1SPpMF ‣ ࢁຊ ·Ώ(@mayusy) ‣ גࣜձࣾCyberagent ΞυςΫຊ෦(2018.06~) ‣ Software Engineer ओʹಈը޿ࠂͷ഑৴ϓϥοτϑΥʔϜ౳Λ։ൃ ‣ Go/ C#/ Kubernetes/ alcohol /PX d
   $ZCFSBHFOU
   ΞυςΫຊ෦ʹ+PJO d
   ৽ଔͰ๭4ࣾʹ+PJO 
   ࣗ࡞"VUPTDBMFS։ൃ
3. w ฎνʔϜͷΞʔΩςΫνϟ্ɺ͔ͳΓ($1ͷٕज़ʹدͬͨӡ༻ࣄྫʹ ͳΓ·͢
   w ($1ͷར༻͸ฎνʔϜʹ͓͚Δ࿩Ͱ͋Γɺ$ZCFSBHFOUʹ͸($1Ҏ֎ ͷํ๏ͰΠϯϑϥΛߏங͍ͯ͠Δ૊৫΋ଘࡏ͠·͢ c
   ͸͡Ίʹ
4. ‣νʔϜͷϓϩμΫτͷ͝঺հ
   ‣ࣗ࡞"VUPTDBMFS͕ඞཁʹͳͬͨഎܠ
   ‣(FNJOJͷ֓ཁͱ࣮૷
   ‣࣮ࡍͷӡ༻݁Ռ
   ‣·ͱΊ c
   "HFOEB
5. νʔϜͷϓϩμΫτ ͷ͝঺հ
6. ಈը޿ࠂͷ ഑৴ϓϥοτϑΥʔϜ ޿ࠂ΁ͷಋઢ ഑৴׬ྃϖʔδ c
   ϓϩμΫτ֓ཁ
7. ‣$MPVE
   1VC4VC
   w ($1Ͱఏڙ͞Ε͍ͯΔϝοηʔδϯάαʔϏε
   w 5PQJDɿ1VCMJTIFS͔ΒͷϝοηʔδΛड৴
   w 4VCTDSJQUJPOɿ4VCTDSJCFS΁ϝοηʔδΛૹ৴
8. ‣$MPVE
   #JHUBCMF
   w($1ͷετϨʔδαʔϏεͷҰͭɽ
   wεέʔϥϒϧͰߴ͍εϧʔϓοτΛތΔ,74
   w/PEF୯ҐͰεέʔϧ͍ͯ͘͠ɽ ,VCFSOFUFTͷ/PEFͱ͸ผ෺
   w"VUPTDBMFػೳ͸༻ҙ͞Ε͍ͯͳ͍
9. Cloud Pub/Sub Publisher Cloud Pub/SubΛ࢖ͬͯඇಉظ ʹϝοηʔδͷॲཧΛߦ͏. Serviceؒͷίϛϡχέʔγϣϯ ͸gRPCͰߦ͏ c
   ΞʔΩςΫνϟ֓ཁ Service Subscriber Cloud Bigtable Subscriber͸Bigtableʹσʔλͷ ಡΈॻ͖΋ߦͳ͍ͬͯΔ
10. ࣗ࡞Autoscaler͕ ඞཁʹͳͬͨഎܠ
11. ͢Έ·ͤΜɼ͋Γ͕ͱ͏͍͟͝·͢ɽ ͳΜ͔4VCTDSJCFS͕ґଘ͍ͯ͠Δ#JHUBCMFͷ஗ԆͰϝοηʔδ͕
    ٧·ͬͯΔΈ͍ͨͰ͢Ͷɽ
    #JHUBCMFͷ/PEF਺·Ͱ૿΍͠·ͨ͠ɽ #JHUBCMFͷ஗ԆʹΑΓ4VCTDSJCFSͷMBUFODZ্͕͕ͬͨࢫͷΤϥʔ͕ग़Δ
12. ͢Έ·ͤΜɼ͋Γ͕ͱ͏͍͟͝·͢ɽ εέʔϧͯ͠΋/PEF਺଍Γͯͳ͍Έ͍ͨͳͷͰɼ
    ߋʹʹ/PEF਺૿΍͠·ͨ͠ɽ #JHUBCMFͷεέʔϧޙ΋4VCTDSJCFSͷMBUFODZ͸ࢭ·Βͣ
13. #JHUBCMFͷ஗Ԇ΋ऩ·ͬͨͷͰ/PEF਺ΛݮΒ͓͖ͯ͠·ͨ͠ɽ εέʔϧͰ͖ͳ͍͜ͱ͚ͩ͡Όͳͯ͘ɼ
    /PEF਺͕ଟ͗͢Δࡍʹແବͳίετ͕͔͔ͬͯ͠·͏͜ͱ΋໰୊Ͱ͢Ͷɽ #JHUBCMFͷ஗Ԇ͕ऩଋͨ͠ޙ
14. HPA Subscriber ᶃpublisherΛ௨ͯ͠Pub/Sub ʹϝοηʔδ͕དྷΔ Subscriber Subscriber ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ Publisher c
    ͜ͷͱ͖ϓϩμΫτͰى͖͍ͯͨ໰୊ $MPVE
    1VC4VC
15. HPA Subscriber Subscriber Subscriber ᶄSubscriberʹେྔͷ ϝοηʔδ͕ૹ৴͞ΕΔ Publisher $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ c
    ͜ͷͱ͖ϓϩμΫτͰى͖͍ͯͨ໰୊
16. ᶅBigtable΁ͷϦΫΤετ ΋େྔʹൃੜ $MPVE
    #JHUBCMF HPA Subscriber Subscriber Subscriber Publisher $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ c
    ͜ͷͱ͖ϓϩμΫτͰى͖͍ͯͨ໰୊
17. $MPVE
    #JHUBCMF ᶆSubscriber͔ΒͷϦΫΤετ૿Ճ ʹΑΓBigtableͷෛՙ͕૿Ճ HPA Subscriber Subscriber Subscriber Publisher $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ c
    ͜ͷͱ͖ϓϩμΫτͰى͖͍ͯͨ໰୊
18. ᶇSubscriberͷະॲཧͷ ϝοηʔδ͕૿Ճ $MPVE
    #JHUBCMF HPA Subscriber Subscriber Subscriber Publisher $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ c
    ͜ͷͱ͖ϓϩμΫτͰى͖͍ͯͨ໰୊
19. ᶇHuman Bigtable Scaler (ਓྗ)ʹΑͬͯ
 NodeΛखಈͰ௥Ճ $MPVE
    #JHUBCMF HPA Subscriber Subscriber Subscriber Publisher $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ c
    ͜ͷͱ͖ϓϩμΫτͰى͖͍ͯͨ໰୊
20. ᶇNodeΛखಈͰ௥Ճ $MPVE
    #JHUBCMF HPA Subscriber Subscriber Subscriber Publisher $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ c
    ͜ͷͱ͖ϓϩμΫτͰى͖͍ͯͨ໰୊ ਓྗͷ࡞ۀ͕සൟʹൃੜ͢ΔΑ͏ʹͳͬͯ͠·ͬͨͨΊɼ
    ΍͸Γ#JHUBCMFͷ"VUPTDBMFS͸ඞཁ
21. c
    ϓϩάϥϜ͔Β#JHUBCMFΛTDBMF͢Δํ๏ͷެࣜ৘ใ ϓϩάϥϜ্͔Β4DBMF͢Δํ๏͸͋Δ͕ɼBVUPTDBMFػೳ͸ࣗલͰ࣮૷͢Δඞཁ͕͋Δ
 ࢀߟɿIUUQTDMPVEHPPHMFDPNCJHUBCMFEPDTTDBMJOH
22. w#JHUBCMFͷ$16࢖༻཰ΛτϦΨʔͱͯ͠#JHUBCMFΛBVUPTDBMF͢Δɽ
    w,VCFSOFUFTͷϦιʔεঢ়گΛτϦΨʔͱͯ͠ɼ#JHUBCMFΛBVUPTDBMF͢Δɽ c
    #JHUBCMFͷࣗ࡞"VUPTDBMFSͰ࣮ݱ͍ͨ͜͠ͱ ӡ༻ෛՙͷܰݮ
    ˍ
    ༨෼ͳϦιʔεͷ࢖༻ʹΑΔແବͳίετͷ࡟ݮ
23. Bigtableʹ͸Ϧιʔεత ͳ༨༟͕͋Δ $MPVE
    #JHUBCMF HPA Subscriber Subscriber Subscriber Publisher $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ Subscriberෆ଍ʹΑΓ ϝοηʔδͷॲཧ͕஗Ԇ c
    ೚ҙͷϝτϦΫεىҼͰ1PEΛεέʔϧ͍ͨ͠ CPU/RAMෛՙ͕ߴ͍Θ͚Ͱ͸ͳ͍ͷͰ σϑΥϧτͰ͸Φʔτεέʔϧ͠ͳ͍
24. Bigtableʹ͸Ϧιʔεత ͳ༨༟͕͋Δ $MPVE
    #JHUBCMF HPA Subscriber Subscriber Publisher Subscriber $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ Human Pod Autoscaler (ਓྗ)ʹΑͬͯ PodΛεέʔϧ c
    ೚ҙͷϝτϦΫεىҼͰ1PEΛεέʔϧ͍ͨ͠ Subscriber ϦΫΤετ਺౳ΛτϦΨʔ ͱͯ͠TDBMF͍ͨ͠
25. Bigtableʹ͸Ϧιʔεత ͳ༨༟͕͋Δ $MPVE
    #JHUBCMF HPA Subscriber Subscriber Publisher Subscriber $MPVE
    1VC4VC ‣֎෦ʹϦΫΤετΛૹ৴͢ΔॲཧΛ൐͏Subscriberʹͯ Human Pod Autoscaler (HPA)ʹΑͬͯ PodΛεέʔϧ c
    ೚ҙͷϝτϦΫεىҼͰ1PEΛεέʔϧ͍ͨ͠ Subscriber ͜͜Ͱ΋ਓྗͷ࡞ۀ͕සൟʹൃੜ͢ΔΑ͏ʹͳͬͯ͠·ͬͨͨΊɼ
    ೚ҙͷϝτϦΫεͰͷ1PEͷ"VUPTDBMFS͕ඞཁͱͳͬͨ
26. Geminiͷ ֓ཁͱ։ൃͷৄࡉ
27. KubernetesͷCRDͱ͍͏ػೳΛར༻ͨ͠Custom Autoscaler ‣ػೳ • ೚ҙͷϝτϦΫεىҼͰͷBigtableͷautoscale • ೚ҙͷϝτϦΫεىҼͰͷPod autoscale c
    (FNJOJͱ͸
28. ,VCFSOFUFTͷ"1*ΛΧελϚΠζͰ͖Δػೳͷ͏ͪͷҰͭɽ
    ‣"1*
    &YUFOTJPO
    w $VTUPN
    3FTPVSDF
    %FpOJUJPO
    $VTUPN
    $POUSPMMFS
    w "VUIFOUJDBUJPO 5PLFO
    3FWJFX
    "1*
    w "VUIPSJ[BUJPO 4VCKFDU
    "DDFTT
    3FWJFX
    "1*
    w "ENJTTJPO
    $POUSPM c
    $3%ͷ֓ཁ
29. w Ϣʔβʔಠࣗͷ3FTPVSDF
    5ZQFΛఆٛͰ͖Δػೳ
    w ಠࣗͷϚχϑΣετϑΝΠϧΛѻ͏͜ͱ͕ՄೳʹͳΔɽ c
    $VTUPN
    3FTPVSDF
    %FpOJUJPO
30. w Ϣʔβʔ͕ಠࣗʹߏஙͨ͠$POUSPMMFSɽ
    w ಛఆͷλεΫΛࣗಈԽͰ͖Δɽ
    w $VTUPN
    3FTPVSDFͱ૊Έ߹Θͤͯɼଞ$POUSPMMFSͱಉ༷ʹ$POUSPM
    MPPQ
    3FDPODJMJBUJPO
    -PPQ Λ࣮૷͢Δ͜ͱ͕Մೳɽ c
    $VTUPN
    $POUSPMMFS
31. Gemini Resource 1 Gemini … Gemini Resource 2 Gemini Resource 3 ‣TDBMFઃఆͷऔಘ
    w$VTUPN
    $POUSPMMFSͷ$POUSPM
    MPPQͷதͰܧଓతʹऔಘ
    w֤(FNJOJ
    3FTPVSDF͝ͱʹεέʔϧର৅΍NJO
    NBY஋ͷઃఆ͕Մೳ
32. ‣ϝτϦΫεͷ࣌ܥྻσʔλͷऩू
    w4UBDLESJWFS
    .POJUPSJOHͱ࿈ܞ
    wΞϥʔτϙϦγʔ͕ࢀর͢ΔϝτϦΫεͷ࣌ܥྻσʔλΛऔಘ
    w ྫɿ#JHUBCMF
    DQV
    VTBHF
    #JHUBCMFͷϝτϦΫε
    w ྫɿະॲཧͷϝοηʔδ਺ 1VC4VCͷϝτϦΫε
    wεέʔϧ൑ఆʹ࢖༻͢ΔΞϥʔτϙϦγʔ͸ઃఆϑΝΠϧͰࢦఆՄೳ
33. ‣ $VTUPN
    SFTPVSDF΁ͷओͳઃఆ߲໨ #JHUBCMF
    TDBMFฤ
    w ؂ࢹର৅ͷJOTUBODF
    OBNF
    DMVTUFS
    OBNF
    w /PEF਺ͷNJO
    NBY஋
    w TDBMFͷτϦΨʔͱ͍ͨ͠ΞϥʔτͷϙϦγʔ*% BQJ7FSTJPO
    BVUPTDBMFSDZCFSBHFOUDPKQWCFUB
    LJOE
    HFNJOJ
    
    DVTUPN
    SFTPVSDF໊
    NFUBEBUB
    
    
    MBCFMT
    
    
    
    
    DPOUSPMMFSUPPMTLTJP
    
    
    
    OBNF
    HFNJOJTBNQMF
    TQFD
    DMVTUFS/BNF
    YYYYYYYYYY
    
    
    JOTUBODF/BNF
    YYYYYYYYYY
    
    
    NBY/PEFT
    
    
    
    NJO/PEFT
    
    
    
    QPMJDZ*E
    YYYYYYYYYY
    
    TUBDLESJWFS্ͰϙϦγʔ͝ͱʹৼΒΕΔ*%
    ઃఆྫ
34. ‣ $VTUPN
    SFTPVSDF΁ͷओͳઃఆ߲໨ 1PE
    TDBMFฤ
    w ؂ࢹର৅ͷEFQMPZNFOU
    OBNF
    w 3FQMJDB਺ͷNJO
    NBY஋
    w TDBMFͷτϦΨʔͱ͍ͨ͠ΞϥʔτͷϙϦγʔ*% BQJ7FSTJPO
    BVUPTDBMFSDZCFSBHFOUDPKQWCFUB
    LJOE
    HFNJOJ
    
    DVTUPN
    SFTPVSDF໊
    NFUBEBUB
    
    
    MBCFMT
    
    
    
    
    DPOUSPMMFSUPPMTLTJP
    
    
    
    OBNF
    HFNJOJTBNQMF
    TQFD
    
    
    EFQMPZNFOU/BNF
    YYYYYYYYYY
    
    
    NBY3FQMJDBT
    
    
    
    NJO3FQMJDBT
    
    
    
    QPMJDZ*E
    YYYYYYYYYY
    
    
    TUBDLESJWFS্ͰϙϦγʔ͝ͱʹৼΒΕΔ*%
    ઃఆྫ
35. ‣εέʔϧϩδοΫ
    w աڈͷ࣌ܥྻσʔλ͕ܧଓͯ͠ᮢ஋ ΞϥʔτϙϦγʔʹઃఆ͞Ε ͍ͯΔᮢ஋ Λ௒͑Δ͔൱͔Ͱ൑ఆ
    w εέʔϧΞ΢τ͸ૣ͘ɼεέʔϧΠϯ͸Ώͬ͘Γͱߦ͏
    w εύΠΫ൑ఆϩδοΫɼલճεέʔϧ͔ΒҰఆ࣌ؒ͸εέʔϧ͠ͳ ͍Α͏ʹ͢ΔϩδοΫ౳΋ಠࣗͰಋೖ #JHUBCMFΛTDBMFͨ͠ࡍͷ/PEF਺ͷมԽਤ εέʔϧΠϯ εέʔϧΞ΢τ
36. ‣࣌ܥྻσʔλͷղੳ
    w ੜͷ࣌ܥྻσʔλͰ͸ϊΠζ͕ଟ͘ɼεέʔϧͷޡ൑ఆ͕ൃੜ͠΍͘͢ͳΔ ͨΊɼҠಈฏۉ 4." Λ͔͚ͯεϜʔδϯάͨ͠σʔλΛར༻ɽ
    w 4."ͷXJOEPX
    TJ[F
    ϓϩοτִؒ
    աڈԿ෼ͷσʔλΛ൑ఆʹ࢖༻͢Δ͔౳ ͷύϥϝʔλ͸ࡉ͔͍ௐ੔͕ඞཁɽ εύΠΫ෦෼ͷϊΠζ͕ ܰݮ͞Ε͍ͯΔ
37. GKE Gemini ᶃϝτϦΫεͷ࣌ܥྻσʔλͱ ొ࿥ࡁΈͷΞϥʔτϙϦγʔΛऔಘ #JHUBCMF ᶄϝτϦΫε͕ϙϦγʔͷᮢ஋Λ Ұఆ࣌ؒ࣋ଓͯ͠௒͑Δ͔൱͔Λ൑ఆ ᶅBigtableͷAPIΛୟ͍ͯ
 Node਺Λߋ৽ c
    "VUPTDBMF࣌ͷॲཧϑϩʔ #JHUBCMFฤ
38. kube-apiserver ᶃϝτϦΫεͷ࣌ܥྻσʔλͱ ొ࿥ࡁΈͷΞϥʔτϙϦγʔΛऔಘ Node Pod Pod Gemini c
    "VUPTDBMF࣌ͷॲཧϑϩʔ 1PEฤ
39. ᶄϝτϦΫε͕ϙϦγʔͷᮢ஋Λ Ұఆ࣌ؒ࣋ଓͯ͠௒͑Δ͔൱͔Λ൑ఆ c
    "VUPTDBMF࣌ͷॲཧϑϩʔ 1PEฤ Node Pod Pod Gemini kube-apiserver
40. ᶅscale out/ inͷ൑ఆΛͨ͠৔߹͸ kube-apiserver্ͷdeploymentΛߋ৽ c
    "VUPTDBMF࣌ͷॲཧϑϩʔ 1PEฤ Node Pod Pod Gemini kube-apiserver
41. ᶆScheduler͕Deploymentͷ৘ใ ʹ߹ΘͤͯPodΛNodeʹׂΓ౰ͯ c
    "VUPTDBMF࣌ͷॲཧϑϩʔ 1PEฤ Node Pod Pod Gemini kube-apiserver
42. ᶇPod͕৽ͨʹ࡞ΒΕΔ c
    "VUPTDBMF࣌ͷॲཧϑϩʔ 1PEฤ Node Pod New Pod Gemini Pod kube-apiserver
43. w $VTUPN
    $POUSPMMFS͸LVCFCVJMEFS౳ͷ4%,Λ࢖༻ͤͣϑϧεΫϥονͰߏஙɽ
    w 4%,ͷ৘ใ͕গͳ͘ɼ·ͩར༻ࣄྫ΋গͳ͍࣌ظʹ࡞ͬͨͨΊɽ ࣮૷౰࣌
    
    w (PͰ࣮૷ɽ
    w ओʹ࢖༻ͨ͠ϥΠϒϥϦ
    w ࣌ܥྻσʔλ΍ϙϦγʔऔಘɿ4UBDLESJWFS
    .POJUPSJOH
    "1*
    $MJFOU
    -JCSBSJFT
    
    w EFQMPZNFOUߋ৽ɿLVCFSOFUFTDMJFOUHP c
    ։ൃͷৄࡉ
44. 1PEͷ4DBMFʹؔͯ͠͸ɼ౰ॳҎԼͷ࣮૷ํ๏΋ݕ౼͍ͯͨ͠ɽ
    ‣HPAͷCustom Metrics • Prometheus΍StackdriverͳͲͷϞχλϦϯάπʔϧ͔Β೚ҙͷϝτϦΫεΛऔಘͯ͠ɺ ᮢ஋ΛઃఆͰ͖Δɻ • ͭ·Γ೚ҙͷϝτϦΫεىҼͰΦʔτεέʔϧͰ͖ΔΑ͏ʹͳΔɻˠGeminiͱಉ͡ػೳ c
    ٕज़બఆ
45. ‣ԿނCustom MetricsͰ͸ͳ͘CRDΛ࠾༻ͨ͠ͷ͔ • BigtableͷNodeΛεέʔϧ͍ͤͨ͞ཉٻ΋͋ͬͨͷͰɺPodͷεέʔϧτϦΨʔ Λ૿΍ͤΔ͚ͩͰ͸ෆे෼ɽ • ୯७ʹCRDΛ࢖ͬͯΈ͔ͨͬͨɾCRDͷӡ༻ࣄྫΛ࡞Γ͔ͨͬͨɽ
46. ࣮ࡍͷӡ༻݁Ռ
47. 4DBMF
    PVU ͷᮢ஋ ˞ॾࣄ৘ʹΑΓ1PEͷTDBMFػೳͷํͷӡ༻݁Ռͷσʔλ͕͓ݟͤͰ͖ͳ͍ͷͰɼ
 ࠓճ͸CJHUBCMFͷTDBMFͷํͷӡ༻݁ՌͷΈͷ͝঺հͱͳΓ·͢ɽ
50. Human ** Scalerͷग़ಈ͸ ௥Ճ Pod ௥Ճ
51. Human ** Scalerͷग़ಈ͸ ௥Ճ Pod ௥Ճ ΄΅θϩʹ
52. ‣ ྑ͔ͬͨ఺
    w εύΠΫ൑ఆ౳ɼಠࣗͷεέʔϧ൑ఆج४ͷಋೖ͕Մೳɽ
    w εέʔϧτϦΨʔΛॊೈʹ௥ՃͰ͖Δɽ
    w ྫɿؔ࿈͢ΔଞαʔϏεͷϦΫΤετ૿ՃΛࣗαʔϏεͷTDBMF
    PVUτϦΨʔʹ͢Δɽ
    ‣ ۤ࿑ͨ͠఺
    w τϦΨʔΛ૿΍͢͜ͱͰɼ$POUSPMMFSͷεέʔϧϩδοΫ͕ෳࡶʹͳΔɽ
    w ҆ఆͨ͠εέʔϧϩδοΫʹ͢ΔͨΊʹௐ੔ͱվળΛ܁Γฦ͢ඞཁ͕͋ΔͨΊɼ҆ఆՔಈ Ͱ͖ΔϨϕϧʹͳΔ·Ͱʹ͸खؒͱ͕͔͔࣌ؒΔɽ c
    ࣗ࡞"VUPTDBMFSͷ։ൃΛऴ͑ͯ
53. ·ͱΊ
54. • ӡ༻ෆՄܰݮͱίετ࡟ݮͷͨΊʹɼCRDΛ࢖ͬͯPod΍BigtableΛscaleͰ͖ ΔAutoscalerΛࣗ࡞ͨ͠ɽ • ͜Ε·ͰखಈͰߦ͍͍͔ͬͯͨͭ͘ͷӡ༻͕΄΅ࣗಈԽͰ͖ΔΑ͏ʹͳͬͨɽ • AutoscalerΛࣗ࡞͢ΔͱεέʔϧϩδοΫ΍τϦΨʔʹॊೈੑΛ࣋ͨͤΒΕΔ ҰํͰɼ҆ఆՔಈ·Ͱʹ࣌ؒͱख͕͔͔ؒΔɽ • ࠓޙOSSԽ౳΋ࢹ໺ʹ͍ΕͯܧଓతͳվળΛଓ͚͍͖͍ͯͨɽ c
    ·ͱΊ
55. Thank you!