クラウドネイティブを創る技術

話し手の所属するGMOペパボでは、First Containerアーキテクチャを用いたクラウド・ホスティングサービスであるロリポップ！マネージドクラウドを提供しています。サービスを開発するに当たり、mrubyやGolangで開発した独自のミドルウェアなど多くのコンポーネントを生み出してきました。それらとKubernetesを比較しながら、コンテナのオーケストレーションにおけるアーキテクチャについて論じ、実装の詳細に迫り、コンテナをこれまでのLinuxOSにおけるプロセスと捉えた場合に、例えばログの扱いや、モニタリングをどうしていくべきなのか、クラウドネイティブ時代におけるベストプラクティスを考える時間にしたいと考えています。

Transcript

1. Ϋϥ΢υωΠςΟϒΛ૑Δٕज़
   ʙݰքṗͰͱΕͨਅୄʹΦʔΨχοΫΦΠϧΛఴ͑ͯʙ
   
   @pyama86
   

   View Slide

2. ࢁԼ࿨඙!QZBNB
   (.0ϖύϘٕज़ج൫νʔϜ
   γχΞɾϓϦϯγύϧ
   UFOTOBQPODPN
   QZBNBGVO
   TUOTKQ
   

   View Slide

3. 45/4
   -JOVY/444FSWFS
   TUOTKQ
   

   View Slide

4. 45/4
   

   View Slide

5. झຯ: ࠗ׆
   

   View Slide

6. “Έͳ͞Μ͸ɺ
   CloudNativeͰ͔͢ʁ”
   

   View Slide

7. View Slide

8. Ϋϥ΢υωΠςΟϒΛ
   ૑Δٕज़
   

   View Slide

9. ࠓ೔࿩͢͜ͱ
   1.ϚωʔδυΫϥ΢υʹ͍ͭͯ
   
   2.KubernetesͱϚωʔδυΫϥ΢υͷൺֱ
   
   3.ίϯςφΦʔέετϨʔγϣϯγεςϜͷߏஙɺӡ༻ʹ͍ͭͯ
   
   4.๻ͷ൒ੜͱ༮গظʹ͍͔ʹՄѪ͔͔ͬͨΛٞ࿦
   

   View Slide

10. –Johnny Appleseed
    “Ϋϥ΢υωΠςΟϒٕज़͸ɺύϒϦοΫΫϥ΢υɺϓϥΠϕʔτΫϥ
    ΢υɺϋΠϒϦουΫϥ΢υͳͲͷۙ୅తͰμΠφϛοΫͳ؀ڥʹ͓͍
    ͯɺεέʔϥϒϧͳΞϓϦέʔγϣϯΛߏங͓Αͼ࣮ߦ͢ΔͨΊͷ
    ೳྗΛ૊৫ʹ΋ͨΒ͠·͢ɻ ͜ͷΞϓϩʔνͷ୅දྫʹɺ

    ίϯςφɺαʔϏεϝογϡɺϚΠΫϩαʔϏεɺΠϛϡʔμϒϧΠ
    ϯϑϥετϥΫνϟɺ͓ΑͼએݴܕAPI͕͋Γ·͢ɻ
    ͜ΕΒͷख๏ʹΑΓɺճ෮ੑɺ؅ཧྗɺ͓ΑͼՄ؍ଌੑͷ͋Δૄ݁߹
    γεςϜ͕࣮ݱ͠·͢ɻ ͜ΕΒΛݎ࿚ͳࣗಈԽͱ૊Έ߹ΘͤΔ͜ͱͰɺ
    ΤϯδχΞ͸ΠϯύΫτͷ͋ΔมߋΛ࠷খݶͷ࿑ྗͰසൟ͔ͭ༧ଌͲ͓
    Γʹߦ͏͜ͱ͕Ͱ͖·͢ɻ”
    

    View Slide

11. ϨϯλϧαʔόͱΫϥ΢υͷ͍͍ͱ͜ͲΓ
    ϩϦϙοϓʂϚωʔδυΫϥ΢υ
    IUUQTNDMPMJQPQKQ
    

    View Slide

12. ϩϦϙοϓʂϚωʔδυΫϥ΢υͷັྗ
    • ؆୯ͳΦʔτεέʔϧ
    • ίϯςφςϯϓϨʔτΛݩʹɺ͙͢ʹ։ൃ͕࢝ΊΒΕΔ
    
    • ϨϯαόͷΑ͏ʹDB͕෇ଳ͠ɺ͞ΒʹΠϯϑϥͷӡ༻͸

    ϩϦϙοϓʹ͓೚ͤ
    

    View Slide

13. ϩϦϙοϓʂϚωʔδυΫϥ΢υͷཁૉٕज़
    • FastContainerΞʔΩςΫνϟ
    
    • haconiwa
    

    View Slide

14. FastContainer
    Webαʔό্Ͱͷϓϩηε࣮ߦํࣜͷҰͭͰ͋ΔFastCGI͔Β
    @matsumotory ͕ண૝ͨ͠ΞʔΩςΫνϟ
    FastCGI
    Client
    Web
    Server
    CGI
    CGI
    ϦΫΤετ
    ىಈཁٻ
    ىಈ
    ܭࢉ
    Ϩεϙϯε
    FastContainer
    Client
    Web
    Server
    Container
    Container
    ϦΫΤετ
    ىಈཁٻ
    ىಈ
    ܭࢉ
    Ϩεϙϯε
    

    View Slide

15. haconiwa
    • @udzura ࡞ͷmruby੡ίϯςφϥϯλΠϜ
    
    • RubyͰίϯςφΛఆٛ
    
    • ίϯςφىಈͷ֤εςοϓ΍Ұఆִؒ͝ͱ
    ϑοΫϙΠϯτ͕͋Δ
    
    • CRIUରԠ
    

    View Slide

16. ϚωʔδυΫϥ΢υ
    proxy ngx_mruby
    compute
    ngx_mruby
    haconiwa

    backend coreapi CMD
    B
    compute
    ngx_mruby
    haconiwa

    compute
    ngx_mruby
    haconiwa

    ngx_mruby͕σΟεύονϟͱͳΓɺτϥϑΟοΫϧʔςΟϯ
    ά΍ίϯςφىಈΛߦ͏
    NFS Storage
    

    View Slide

17. ϚωʔδυΫϥ΢υ
    proxy ngx_mruby
    compute
    ngx_mruby
    haconiwa

    backend coreapi
    Data

    Store
    compute
    ngx_mruby
    haconiwa

    compute
    ngx_mruby
    haconiwa

    ϦΫΤετ୯ҐͷΦʔτεέʔϧ
    NFS Storage
    Ϣʔβʔσʔλ͸
    ͢΂ͯετϨʔδʹ͋Δ
    

    View Slide

18. ϚωʔδυΫϥ΢υ
    ʙίϯςφىಈʙ
    

    View Slide

19. ϚωʔδυΫϥ΢υ - ίϯςφىಈ
    CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTSFBEZ
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    SFBEZ଴ػத
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    8FCQSPYZ http://www.kaasan.info/archives/3733
    

    View Slide

20. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTCPPUJOH
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    ίϯςφϨίʔυΛ
    ىಈத΁
    CPPUJOHىಈத
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    ໨తͷίϯςφͷ*1͕
    -*45&/ͯ͠ͳ͍
    ϚωʔδυΫϥ΢υ - ίϯςφىಈ
    ϦΫΤετܖػͰ

    ৘ใऔಘ
    

    View Slide

21. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    ίϯςφϨίʔυΛ
    Քಇத΁
    FOBCMFEՔಇத
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    ίϯςφىಈ
    8FCQSPYZ
    DPOUBJOFS
    ίϯςφͷ

    εςʔλεมߋ
    ϚωʔδυΫϥ΢υ - ίϯςφىಈ
    

    View Slide

22. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    DPOUBJOFS
    ࣍ͷϦΫΤετ͸طʹ

    ίϯςφىಈࡁΈͳͷͰ௚઀ϓϩΩγ
    ϚωʔδυΫϥ΢υ - ίϯςφىಈ
    

    View Slide

23. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTSFBEZ
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    DPOUBJOFS
    ίϯςφͷण໋͕དྷͨΒ
    ίϯςφΛఀࢭ͢Δ
    ϚωʔδυΫϥ΢υ - ίϯςφఀࢭ
    SFBEZ଴ػத΁
    ͳʹ͔ॲཧதͷ৔߹͸ॲཧ͕
    ऴΘΔ·Ͱ଴ͭ
    

    View Slide

24. ϚωʔδυΫϥ΢υ
    ʙΦʔτεέʔϧʙ
    

    View Slide

25. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    ʘෛՙ΍͹͍ʗ
    8FCQSPYZ
    DPOUBJOFS
    ίϯςφ௥Ճཁٻ
    ϚωʔδυΫϥ΢υ - Φʔτεέʔϧ
    DPOUBJOFS
    TUBUVTSFBEZ
    IPTUDPNQVUF
    ίϯςφ৘ใΛ
    ௥Ճ
    

    View Slide

26. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    DPOUBJOFS
    ϦΫΤετܖػͰίϯςφىಈ
    ϚωʔδυΫϥ΢υ - Φʔτεέʔϧ
    ίϯςφ৘ใΛ
    ༗ޮԽ
    DPOUBJOFS
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    

    View Slide

27. ओཁίϯϙʔωϯτ
    

    View Slide

28. OpenStack
    Baremetal
    

    View Slide

29. ओཁίϯϙʔωϯτ - ngx_mruby
    NSVCZ@TTM@IBOETIBLF@IBOEMFS@DPEF
    TTMOHJOY44-OFX
    DFSUJpDBUF@EBUB'JMFSFBE QBUIUP\TTMTFSWFSOBNF^DSU
    
    LFZ@EBUB'JMFSFBE QBUIUP\TTMTFSWFSOBNF^LFZ
    
    TTMDFSUJpDBUF@EBUBDFSUJpDBUF@EBUB
    TTMDFSUJpDBUF@LFZ@EBUBLFZ@EBUB
    
    OHJOYͷ͋ΒΏΔΠϕϯτʹϑοΫͯ͠ɺNSVCZͷίʔυΛ࣮ߦ͠ɺ
    ϓϥΨϒϧʹτϥϑΟοΫίϯτϩʔϧग़དྷΔ
    

    View Slide

30. ओཁίϯϙʔωϯτ - ngx_mruby
    • CoreAPIʹϦΫΤετΛ࣮ߦ͠ɺίϯςφϧʔςΟϯάɺίϯςφىಈΛ
    ߦ͏
    
    • ϦΫΤετ୯ҐͰɺίϯςφʹϔϧενΣοΫΛߦ͏
    
    • ϊϯϒϩοΩϯάʹ৘ใ͕औಘͰ͖ΔΑ͏ngx_mruby v2ʹͯඇಉظεϦʔ
    ϓɺαϒϦΫΤετΛ࣮૷
    
    • HTTP͸L7ϓϩΩγɺͦΕҎ֎͸L4ϓϩΩγ
    

    View Slide

31. ओཁίϯϙʔωϯτ - ngx_mruby
    SFRVFTU
    SFRVFTU
    SFRVFTU
    SVODPOUBJOFS
    SVODPOUBJOFS
    SVODPOUBJOFS
    SFTQPOTF
    SFTQPOTF
    SFTQPOTF
    XPSLFS
    XPSLFS͋ͨΓͷNSVCZ࣮ߦ ίϯςφىಈ
    Ͱ

    ඇಉظ*0͕ϒϩοΫ͞ΕΔ
    ίϯςφىಈ
    ଴ͪ࣌ؒ
    

    View Slide

32. ओཁίϯϙʔωϯτ - ngx_mruby
    /HJOY"TZODTMFFQΛར༻͠ɺίϯςφىಈ଴ͪ࣌ؒΛ
    ඇಉظʹ4MFFQ͢Δ
    SFRVFTU
    SFRVFTU NSVCZ
    SFTQPOTF
    SFTQPOTF
    8PSLFS
    MPPQEP
    CSFBLJGDPOUBJOFSCPPU
    /HJOY"TZODTMFFQ
    FOE
    NSVCZ
    TMFFQ
    ίϯςφىಈޙ
    ඇಉظSleep
    

    View Slide

33. !
    ओཁίϯϙʔωϯτ - ngx_mruby
    ngx_mruby
    Container
    • mruby-fast-remote-checkΛ༻͍ͯɺ

    3way ϋϯυγΣʔΫޙʹclose(2)
    
    • Ԡ౴͕ͳ͚Ε͹ίϯςφΛىಈ
    ϔϧενΣοΫ͸ϦΫΤετͷλΠϛϯάͰߦ͏
    

    View Slide

34. !
    ओཁίϯϙʔωϯτ - haconiwa
    3VCZͷ4ZOUBYͰίϯςφΛఆٛ
    )BDPOJXBEFpOFEPcDPOpHc
    DPOpHOBNFOFXIBDPOJXBUPCFIPTUOBNF
    DPOpHDHSPVQ<DQVTIBSFT>
    DPOpHDISPPU@UPWBSZPVS@SPPUGT
    DPOpHOBNFTQBDFVOTIBSFNPVOU
    DPOpHDBQBCJMJUJFTBMMPXBMM
    DPOpHDBQBCJMJUJFTESPQDBQ@TZT@BENJO
    FOE
    

    View Slide

35. ओཁίϯϙʔωϯτ - CoreAPI(Jardin)
    • GolangΛ༻͍ͯɺlabstack/echoɺjinzhu/gormΛར༻͠։ൃ
    
    • GETܥ͸ݪଇRedisʹΩϟογϡ͠ɺԠ౴ੑೳΛ޲্
    
    • Consul EventΛར༻֤ͯ͠ϩʔϧʹϝοηʔδϯάΛૹΔΑ͏ͳ͜ͱ΋
    ΍͍ͬͯΔ
    

    View Slide

36. ओཁίϯϙʔωϯτ - usermetrics
    • PrometheusΛ࠾༻͠ɺϢʔβʔίϯςφͷ৘ใΛ͢΂ͯू໿
    
    • ίϯςφͷϦιʔεΛgoogle/cadvisorΛར༻͠ɺexporterܦ༝Ͱऩू
    
    • Prometheusܦ༝ͰAlertManager͔ΒWebhookͰΦʔτεέʔϧ
    
    • CPUɺMemoryɺNetworkෛՙͰΞϥʔτΛτϦΨʔ
    

    View Slide

37. ओཁίϯϙʔωϯτ - Scheduler(kangetsu)
    • ϓϩϏδϣχϯάɾδϣϒεέδϡʔϥʔΛGolangͰrallison/go-workersΛར༻͠։ൃ
    
    • ετϨʔδྖҬͷ࡞੒ɺquotaͷઃఆ
    
    • DB࡞੒
    
    • WordPressͷϓϨΠϯετʔϧ
    

    View Slide

38. View Slide

39. ओཁίϯϙʔωϯτ - Cleaner(skmr)
    • ֤ίϯϙʔωϯτͷσʔλෆ੔߹ΛࣗಈͰݕ஌͠मਖ਼
    
    • κϯϏϓϩηεͷຣࡴ
    
    • ίϯςφঢ়ଶͱCMDBͷࠩ෼Λऩଋ
    

    View Slide

40. ͓·͚ - Hashicorp Software
    • αʔϏεσΟεΧόϦʹConsulɺConsul DNSΛϑϧ׆༻
    
    • DBʹ֨ೲ͢Δൿಗ৘ใ͸VaultΛར༻͠҉߸Խ
    
    • OpenStackVMͷϓϩϏδϣχϯά͸TerraformΛར༻
    

    View Slide

41. ͳͥKubernetesͰ͸ͳ͍ͷ͔ʁ
    

    View Slide

42. ͳͥKubernetesͰ͸ͳ͍ͷ͔ʁ
    1.ϖύϘ͸ϗεςΟϯάࣄۀΛఏڙ
    
    2.FastContainerΞʔΩςΫνϟ
    
    3.࣌୅తͳཁҼ
    
    4.ࣄۀࠩผԽ
    

    View Slide

43. KubernetesͱϚωʔδυΫϥ΢υͷҧ͍
    1.UI/UX
    
    2.ΞʔΩςΫνϟ
    
    3.ίϯςφىಈ
    
    4.Φʔτεέʔϧ
    

    View Slide

44. UI/UX
    ϚωʔδυΫϥ΢υ Kubernetes
    •Ϣʔβʔ͸όοΫΤϯυΛҙࣝ͢Δ͜ͱͳ
    ͘ɺෛՙʹԠͯ͡Ϧιʔε͕૿ݮ͢Δ
    •ςϯϓϨʔτΛબ୒͠ɺscp΋͘͠͸gitpush

    ͰΞϓϦέʔγϣϯΛσϓϩΠ
    •YAMLͰهड़Ͱ͖ΔϦιʔεͰॊೈʹ

    Ϧιʔε͕ఆٛՄೳ
    •೚ҙͷίϯςφΠϝʔδ͕ར༻ՄೳͰ

    σϓϩΠ͸ίϯςφbuild΋͘͠͸mount
    

    View Slide

45. ΞʔΩςΫνϟ
    

    View Slide

46. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    ΞʔΩςΫνϟ
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics
    

    View Slide

47. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    σʔλετΞ
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics
    

    View Slide

48. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    API&Scheduler
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics
    

    View Slide

49. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    Node compornents
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics
    

    View Slide

50. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    Container Runtime
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics
    

    View Slide

51. ίϯςφىಈ
    

    View Slide

52. Kubernetes - ίϯςφىಈ
    • YAMLΛAPIʹ౤ೖ͢Δ͜ͱͰίϯςφىಈ
    
    • ϔϧενΣοΫ͸ProbeͰ࣮ߦ(HTTP/TCP/ExecCommand)
    
    • ىಈ࣌ʹߦ͏ͷ͸readinessProbeɺىಈޙ͸livenessProbe
    
    • ॳظԽॲཧ͸InitContainerɺPostStartͰॳظԽॲཧΛϑοΫՄೳ
    
    • NඵִؒͰϑοΫ͢Δͱ͔ɺ༗ޮظݶ͕དྷͨΒίϯςφΛఀࢭ͢Δʹ͸ί
    ϯτϩʔϥʔͰಠ࣮ࣗ૷͕ඞཁ
    

    View Slide

53. ίϯςφىಈϊʔυ͸ͲͷΑ͏ʹܾ·Δʁ
    ϚωʔδυΫϥ΢υ Kubernetes
    •ੜଘ͍ͯ͠Δϊʔυͷϥ΢ϯυϩϏϯ
    
    •ϊʔυ͕ఀࢭ͍ͯ͠Δ৔߹͸࣍ͷϦΫΤετ
    ͔ΒੜଘϊʔυͰίϯςφىಈ
    •kube-scheduler͕ϊʔυΛબ୒͢Δ
    
    •ϑΟϧλϦϯά(ϥϕϧɺϦιʔεetc)
    
    •είΞϦϯά(෼ࢄɺϦιʔεetc)
    

    View Slide

54. Φʔτεέʔϧ
    

    View Slide

55. Φʔτεέʔϧ͸ͲͪΒ΋metricsϕʔε
    ϚωʔδυΫϥ΢υ Kubernetes
    •usermetrics(Prometheus)ʹઃఆ͞ΕͨΞϥʔ
    τ͔ΒWebhookͰCoreAPI͕࣮ߦ͞Εɺίϯ
    ςφͷ༧໿࿮͕૿͑Δ
    
    •εέʔϧΠϯʹ͍ͭͯ͸ɺෛՙ͕Լ͕ͬͨΒ
    ༧໿࿮͕࡟আ͞Εͯɺίϯςφͷ༗ޮظݶ͕
    ౸དྷͨ͠ΒࣗಈͰεέʔϧΠϯ
    •metric-sserverΛΠϯετʔϧ͢Δͱɺ
    HorizontalPodAutoscalerΛར༻ͯ͠Ϧιʔε
    ʹԠͯ͡ΦʔτεέʔϧɾεέʔϧΠϯΛ࣮
    ݱͰ͖Δ
    
    •ࢦఆͨ͠ෛՙʹͳΔΑ͏ʹϙου͕ௐ੔͞Ε
    Δ
    

    View Slide

56. ͜͜·Ͱͷ·ͱΊ
    • ϚωʔδυΫϥ΢υ͸Kubernetes΄ͲॊೈͰ͸ͳ͍͕ɺΠϯϑϥʹৄ͘͠
    ͳͯ͘΋࢖͑Δɺ͓खܰΫϥ΢υͰ͋Δ
    
    • ΞʔΩςΫνϟ΍ΦʔτεέʔϧͷڍಈͳͲྨࣅ఺΋ଟ͋͘Δ
    

    View Slide

57. ϚωʔδυΫϥ΢υ΋
    Ұ෦Kubernetesར༻
    

    View Slide

58. View Slide

59. View Slide

60. ͳͥKubernetesͳͷ͔ʁ
    • ϚτϦϣʔγΧ΋ݕ౼͕ͨ͠ɺγεςϜμ΢ϯ࣌ͷӨڹ͕શஅʹͳΔͨΊ
    ϦεΫߴ͍
    
    • KubernetesΛࣗ෼͕ͨͪӡ༻͠ɺ஌Δ͜ͱͰΑΓΑ͍Ϋϥ΢υɾωΠςΟ
    ϒ΁(Ṗ
    

    View Slide

61. Ϋϥ΢υαʔϏεΛఏڙ͠ͳ͕Βɺ
    KubernetesΛӡ༻
    

    View Slide

62. ίϯςφΦʔέετϨʔγϣϯγεςϜ
    Scheduler
    compute-node
    Container Container Container
    compute-node
    Container Container Container
    εέδϡʔϥʔ͕ϊʔυʹϓϩηεΛׂΓ౰ͯɺ
    ·ͨͦΕΒͷ৘ใ͕ϦΞϧλΠϜʹऩू͞Εɺ
    ࣗ཯తʹಈ͘γεςϜ
    

    View Slide

63. ݟͨ͜ͱ͋Δͳʁʁʁ
    

    View Slide

64. View Slide

65. View Slide

66. ίϯςφΦʔέετϨʔγϣϯγεςϜ͸OSͬΆ͍
    • εέδϡʔϥʔ͕ϓϩηεΛεέδϡʔϦϯά͠ɺͦΕΒͷϑΟʔυόο
    ΫΛड͚ͳ͕Βಈ͖ଓ͚Α͏ͱ͢Δڍಈ͸OSͱࣅ͍ͯΔ఺͕ଟ͍
    
    • OSͱྨࣅ఺͕ଟ͍ͳΒ͹OSʹ͓͚ΔάουϓϥΫςΟε͕ྑ͍࡞༻Λੜ
    ΉՄೳੑ͕͋Δ
    

    View Slide

67. ϩάͷӬଓԽ
    

    View Slide

68. OS
    γϯάϧϊʔυͷ৔߹
    container
    log
    collector
    

    View Slide

69. ΦʔέετϨʔγϣϯγεςϜ
    Scheduler
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector
    

    View Slide

70. ΦʔέετϨʔγϣϯγεςϜ
    Scheduler
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector
    

    View Slide

71. ϩάʹ͓͚Δ՝୊
    • ैདྷͷΑ͏ʹ1ϩʔϧ1αʔόͰ͸ͳ͍͜ͱ͔Βɺ1ͭͷαʔόͰෳ਺ͷछ
    ྨͷϩά͕ग़ྗ͞ΕΔ
    
    • ίϯςφͷूੵ཰ΛߴΊΔͱɺچདྷΑΓϩάͷྲྀྔ͸࣮֬ʹ૿͑Δ
    
    • ෳ਺ͷछྨͷϩάΛ෼ղɺϧʔςΟϯά͢Δ͜ͱͷෳࡶ͞
    

    View Slide

72. KafkaͰόοϑΝϦϯά͢Δ
    compute-node
    Container Container Container
    log
    collector
    log log Kafka cluster
    

    View Slide

73. ,BGLBͷಛ௃
    wߴ଎ॲཧ͕ՄೳͳϝοηʔδϯάγεςϜ
    w෼ࢄॻ͖ࠐΈɺඇಉظϨϓϦέʔγϣϯ˞ɺ෼ࢄಡΈࠐΈ
    wॳظ͸-JOLFEJOʹͯ։ൃɺݱঢ়͸"QBDIFࡒஂʹͯ؅ཧɺ
    -JOLFEJO͔ΒεϐϯΞ΢τͨ͠$POqVFOU͕։ൃΛओମతʹߦ͏
    wݴޠ͸+BWBɺ4DBMBͰ։ൃ
    wΫϥελϦϯά͕ՄೳͰ଱ো֐ੑɺૹୡอূɺݎ࿚ੑ͕ߴ͍
    

    View Slide

74. LBGLBDMVTUFS
    ඵͰཧղ͢Δ,BGLB
    producer
    (ૹΓख)
    kafka-broker-1
    topic
    partition-0
    kafka-broker-2
    topic
    partition-1
    DPOTVNFSHSPVQ
    consumer
    (ड͚ख)
    consumer
    (ड͚ख)
    consumer
    (ड͚ख)
    ૹΓखͱड͚खΛ/ͱ͢ΔόοϑΝʔ
    

    View Slide

75. ϩάͷग़ྗ࣌ؒ < Ξοϓϩʔυ࣌ؒ
    compute-node
    Container Container Container
    log
    collector
    log log Kafka cluster
    όϦૣ͍ʂ
    όϦ஗͍ʂʂʂ
    

    View Slide

76. KafkaΛڬΉϝϦοτ
    wίϯςφىಈϊʔυͷϦιʔεΛ༗ޮར༻͢ΔͨΊʹɺϝΠϯΞϓϦ
    Ҏ֎ͷ࢓ࣄΛͤ͞ͳ͍
    w,BGLBΛڬΉ͜ͱͰσʔλ͕ू໿͞ΕΔ͜ͱͰɺଟ͘ͷίϯςφ͔Β
    ͷ৘ใΛҰՕॴͰѻ͑ΔΑ͏ʹͳΔ
    w,BGLBDPOOFDUΛར༻͢Δ͜ͱͰଟ͘ͷ044ɺϚωʔδυɾαʔϏε
    ͱ࿈ܞͰ͖ΔͷͰ͓ಘ
    

    View Slide

77. ϖύϘ ϩάج൫(௨শେ࿨ా)
    • ϓϩμΫγϣϯͰར༻͍ͯ͠ΔKubernetes͸͢΂ͯKafkaʹϩάΛू໿
    ͠ɺS3΁ͷϩάอશɺGryalogͰͷݕࡧ͕Մೳʹͳ͍ͬͯΔ
    
    • ΞϓϦέʔγϣϯ։ൃऀ͸ΞϓϦΛ։ൃ͢Δ͜ͱ͚ͩʹूதͰ͖Δ
    
    • ϚωʔδυΫϥ΢υ΋େ࿨ాԽ͍ͨ͠
    

    View Slide

78. Ϋϥ΢υωΠςΟϒΛ૑Δʹ͸
    ·ͣ͸ࣗ෼͕Ϋϥ΢υωΠςΟϒʹͳΔ
    ඞཁ͕͋Δ
    

    View Slide

79. ࣗ෼ͨͪͰੈքʹͨͩҰͭͷ
    Ϋϥ΢υαʔϏεΛ૑Γ
    ӡ༻͢Δ͜ͱ͸ָ͍͠
    

    View Slide

80. Ϋϥ΢υαʔϏεΛ૑Γ
    Ϋϥ΢υαʔϏεͱ૊Έ߹Θͤ
    ϢʔβʔΛϋοϐʔʹʂʂʂ
    

    View Slide

81. ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU
    Զͱ͜ͷϏοά΢Σʔϒʹ৐ͬͯ͘Εʂʂ̍
    

    View Slide