Upgrade to Pro — share decks privately, control downloads, hide ads and more …

クラウドネイティブを創る技術

 クラウドネイティブを創る技術

話し手の所属するGMOペパボでは、First Containerアーキテクチャを用いたクラウド・ホスティングサービスであるロリポップ!マネージドクラウドを提供しています。サービスを開発するに当たり、mrubyやGolangで開発した独自のミドルウェアなど多くのコンポーネントを生み出してきました。それらとKubernetesを比較しながら、コンテナのオーケストレーションにおけるアーキテクチャについて論じ、実装の詳細に迫り、コンテナをこれまでのLinuxOSにおけるプロセスと捉えた場合に、例えばログの扱いや、モニタリングをどうしていくべきなのか、クラウドネイティブ時代におけるベストプラクティスを考える時間にしたいと考えています。

Kazuhiko Yamashita

July 23, 2019
Tweet

More Decks by Kazuhiko Yamashita

Other Decks in Programming

Transcript

  1. Ϋϥ΢υωΠςΟϒΛ૑Δٕज़
    ʙݰքṗͰͱΕͨਅୄʹΦʔΨχοΫΦΠϧΛఴ͑ͯʙ

    @pyama86

    View full-size slide

  2. ࢁԼ࿨඙!QZBNB
    (.0ϖύϘٕज़ج൫νʔϜ
    γχΞɾϓϦϯγύϧ
    UFOTOBQPODPN
    QZBNBGVO
    TUOTKQ

    View full-size slide

  3. 45/4
    -JOVY/444FSWFS
    TUOTKQ

    View full-size slide

  4. झຯ: ࠗ׆

    View full-size slide

  5. “Έͳ͞Μ͸ɺ
    CloudNativeͰ͔͢ʁ”

    View full-size slide

  6. Ϋϥ΢υωΠςΟϒΛ
    ૑Δٕज़

    View full-size slide

  7. ࠓ೔࿩͢͜ͱ
    1.ϚωʔδυΫϥ΢υʹ͍ͭͯ

    2.KubernetesͱϚωʔδυΫϥ΢υͷൺֱ

    3.ίϯςφΦʔέετϨʔγϣϯγεςϜͷߏஙɺӡ༻ʹ͍ͭͯ

    4.๻ͷ൒ੜͱ༮গظʹ͍͔ʹՄѪ͔͔ͬͨΛٞ࿦

    View full-size slide

  8. –Johnny Appleseed
    “Ϋϥ΢υωΠςΟϒٕज़͸ɺύϒϦοΫΫϥ΢υɺϓϥΠϕʔτΫϥ
    ΢υɺϋΠϒϦουΫϥ΢υͳͲͷۙ୅తͰμΠφϛοΫͳ؀ڥʹ͓͍
    ͯɺεέʔϥϒϧͳΞϓϦέʔγϣϯΛߏங͓Αͼ࣮ߦ͢ΔͨΊͷ
    ೳྗΛ૊৫ʹ΋ͨΒ͠·͢ɻ ͜ͷΞϓϩʔνͷ୅දྫʹɺ

    ίϯςφɺαʔϏεϝογϡɺϚΠΫϩαʔϏεɺΠϛϡʔμϒϧΠ
    ϯϑϥετϥΫνϟɺ͓ΑͼએݴܕAPI͕͋Γ·͢ɻ
    ͜ΕΒͷख๏ʹΑΓɺճ෮ੑɺ؅ཧྗɺ͓ΑͼՄ؍ଌੑͷ͋Δૄ݁߹
    γεςϜ͕࣮ݱ͠·͢ɻ ͜ΕΒΛݎ࿚ͳࣗಈԽͱ૊Έ߹ΘͤΔ͜ͱͰɺ
    ΤϯδχΞ͸ΠϯύΫτͷ͋ΔมߋΛ࠷খݶͷ࿑ྗͰසൟ͔ͭ༧ଌͲ͓
    Γʹߦ͏͜ͱ͕Ͱ͖·͢ɻ”

    View full-size slide

  9. ϨϯλϧαʔόͱΫϥ΢υͷ͍͍ͱ͜ͲΓ
    ϩϦϙοϓʂϚωʔδυΫϥ΢υ
    IUUQTNDMPMJQPQKQ

    View full-size slide

  10. ϩϦϙοϓʂϚωʔδυΫϥ΢υͷັྗ
    • ؆୯ͳΦʔτεέʔϧ
    • ίϯςφςϯϓϨʔτΛݩʹɺ͙͢ʹ։ൃ͕࢝ΊΒΕΔ

    • ϨϯαόͷΑ͏ʹDB͕෇ଳ͠ɺ͞ΒʹΠϯϑϥͷӡ༻͸

    ϩϦϙοϓʹ͓೚ͤ

    View full-size slide

  11. ϩϦϙοϓʂϚωʔδυΫϥ΢υͷཁૉٕज़
    • FastContainerΞʔΩςΫνϟ

    • haconiwa

    View full-size slide

  12. FastContainer
    Webαʔό্Ͱͷϓϩηε࣮ߦํࣜͷҰͭͰ͋ΔFastCGI͔Β
    @matsumotory ͕ண૝ͨ͠ΞʔΩςΫνϟ
    FastCGI
    Client
    Web
    Server
    CGI
    CGI
    ϦΫΤετ
    ىಈཁٻ
    ىಈ
    ܭࢉ
    Ϩεϙϯε
    FastContainer
    Client
    Web
    Server
    Container
    Container
    ϦΫΤετ
    ىಈཁٻ
    ىಈ
    ܭࢉ
    Ϩεϙϯε

    View full-size slide

  13. haconiwa
    • @udzura ࡞ͷmruby੡ίϯςφϥϯλΠϜ

    • RubyͰίϯςφΛఆٛ

    • ίϯςφىಈͷ֤εςοϓ΍Ұఆִؒ͝ͱ
    ϑοΫϙΠϯτ͕͋Δ

    • CRIUରԠ

    View full-size slide

  14. ϚωʔδυΫϥ΢υ
    proxy ngx_mruby
    compute
    ngx_mruby
    haconiwa

    backend coreapi CMD
    B
    compute
    ngx_mruby
    haconiwa

    compute
    ngx_mruby
    haconiwa

    ngx_mruby͕σΟεύονϟͱͳΓɺτϥϑΟοΫϧʔςΟϯ
    ά΍ίϯςφىಈΛߦ͏
    NFS Storage

    View full-size slide

  15. ϚωʔδυΫϥ΢υ
    proxy ngx_mruby
    compute
    ngx_mruby
    haconiwa

    backend coreapi
    Data

    Store
    compute
    ngx_mruby
    haconiwa

    compute
    ngx_mruby
    haconiwa

    ϦΫΤετ୯ҐͷΦʔτεέʔϧ
    NFS Storage
    Ϣʔβʔσʔλ͸
    ͢΂ͯετϨʔδʹ͋Δ

    View full-size slide

  16. ϚωʔδυΫϥ΢υ
    ʙίϯςφىಈʙ

    View full-size slide

  17. ϚωʔδυΫϥ΢υ - ίϯςφىಈ
    CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTSFBEZ
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    SFBEZ଴ػத
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    8FCQSPYZ http://www.kaasan.info/archives/3733

    View full-size slide

  18. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTCPPUJOH
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    ίϯςφϨίʔυΛ
    ىಈத΁
    CPPUJOHىಈத
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    ໨తͷίϯςφͷ*1͕
    -*45&/ͯ͠ͳ͍
    ϚωʔδυΫϥ΢υ - ίϯςφىಈ
    ϦΫΤετܖػͰ

    ৘ใऔಘ

    View full-size slide

  19. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    ίϯςφϨίʔυΛ
    Քಇத΁
    FOBCMFEՔಇத
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    ίϯςφىಈ
    8FCQSPYZ
    DPOUBJOFS
    ίϯςφͷ

    εςʔλεมߋ
    ϚωʔδυΫϥ΢υ - ίϯςφىಈ

    View full-size slide

  20. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    DPOUBJOFS
    ࣍ͷϦΫΤετ͸طʹ

    ίϯςφىಈࡁΈͳͷͰ௚઀ϓϩΩγ
    ϚωʔδυΫϥ΢υ - ίϯςφىಈ

    View full-size slide

  21. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTSFBEZ
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    DPOUBJOFS
    ίϯςφͷण໋͕དྷͨΒ
    ίϯςφΛఀࢭ͢Δ
    ϚωʔδυΫϥ΢υ - ίϯςφఀࢭ
    SFBEZ଴ػத΁
    ͳʹ͔ॲཧதͷ৔߹͸ॲཧ͕
    ऴΘΔ·Ͱ଴ͭ

    View full-size slide

  22. ϚωʔδυΫϥ΢υ
    ʙΦʔτεέʔϧʙ

    View full-size slide

  23. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    ʘෛՙ΍͹͍ʗ
    8FCQSPYZ
    DPOUBJOFS
    ίϯςφ௥Ճཁٻ
    ϚωʔδυΫϥ΢υ - Φʔτεέʔϧ
    DPOUBJOFS
    TUBUVTSFBEZ
    IPTUDPNQVUF
    ίϯςφ৘ใΛ
    ௥Ճ

    View full-size slide

  24. CMDB
    $PSF"1*
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF
    ίϯςφͷ৘ใ
    DPNQVUF
    $MJFOU
    TUPSBHF
    DPNQVUF
    OGT
    OGT
    http://www.kaasan.info/archives/3733
    8FCQSPYZ
    DPOUBJOFS
    ϦΫΤετܖػͰίϯςφىಈ
    ϚωʔδυΫϥ΢υ - Φʔτεέʔϧ
    ίϯςφ৘ใΛ
    ༗ޮԽ
    DPOUBJOFS
    DPOUBJOFS
    TUBUVTFOBCMFE
    IPTUDPNQVUF

    View full-size slide

  25. ओཁίϯϙʔωϯτ

    View full-size slide

  26. OpenStack
    Baremetal

    View full-size slide

  27. ओཁίϯϙʔωϯτ - ngx_mruby
    NSVCZ@TTM@IBOETIBLF@IBOEMFS@DPEF
    TTMOHJOY44-OFX
    DFSUJpDBUF@EBUB'JMFSFBE QBUIUP\TTMTFSWFSOBNF^DSU

    LFZ@EBUB'JMFSFBE QBUIUP\TTMTFSWFSOBNF^LFZ

    TTMDFSUJpDBUF@EBUBDFSUJpDBUF@EBUB
    TTMDFSUJpDBUF@LFZ@EBUBLFZ@EBUB

    OHJOYͷ͋ΒΏΔΠϕϯτʹϑοΫͯ͠ɺNSVCZͷίʔυΛ࣮ߦ͠ɺ
    ϓϥΨϒϧʹτϥϑΟοΫίϯτϩʔϧग़དྷΔ

    View full-size slide

  28. ओཁίϯϙʔωϯτ - ngx_mruby
    • CoreAPIʹϦΫΤετΛ࣮ߦ͠ɺίϯςφϧʔςΟϯάɺίϯςφىಈΛ
    ߦ͏

    • ϦΫΤετ୯ҐͰɺίϯςφʹϔϧενΣοΫΛߦ͏

    • ϊϯϒϩοΩϯάʹ৘ใ͕औಘͰ͖ΔΑ͏ngx_mruby v2ʹͯඇಉظεϦʔ
    ϓɺαϒϦΫΤετΛ࣮૷

    • HTTP͸L7ϓϩΩγɺͦΕҎ֎͸L4ϓϩΩγ

    View full-size slide

  29. ओཁίϯϙʔωϯτ - ngx_mruby
    SFRVFTU
    SFRVFTU
    SFRVFTU
    SVODPOUBJOFS
    SVODPOUBJOFS
    SVODPOUBJOFS
    SFTQPOTF
    SFTQPOTF
    SFTQPOTF
    XPSLFS
    XPSLFS͋ͨΓͷNSVCZ࣮ߦ ίϯςφىಈ
    Ͱ

    ඇಉظ*0͕ϒϩοΫ͞ΕΔ
    ίϯςφىಈ
    ଴ͪ࣌ؒ

    View full-size slide

  30. ओཁίϯϙʔωϯτ - ngx_mruby
    /HJOY"TZODTMFFQΛར༻͠ɺίϯςφىಈ଴ͪ࣌ؒΛ
    ඇಉظʹ4MFFQ͢Δ
    SFRVFTU
    SFRVFTU NSVCZ
    SFTQPOTF
    SFTQPOTF
    8PSLFS
    MPPQEP
    CSFBLJGDPOUBJOFSCPPU
    /HJOY"TZODTMFFQ
    FOE
    NSVCZ
    TMFFQ
    ίϯςφىಈޙ
    ඇಉظSleep

    View full-size slide

  31. !
    ओཁίϯϙʔωϯτ - ngx_mruby
    ngx_mruby
    Container
    • mruby-fast-remote-checkΛ༻͍ͯɺ

    3way ϋϯυγΣʔΫޙʹclose(2)

    • Ԡ౴͕ͳ͚Ε͹ίϯςφΛىಈ
    ϔϧενΣοΫ͸ϦΫΤετͷλΠϛϯάͰߦ͏

    View full-size slide

  32. !
    ओཁίϯϙʔωϯτ - haconiwa
    3VCZͷ4ZOUBYͰίϯςφΛఆٛ
    )BDPOJXBEFpOFEPcDPOpHc
    DPOpHOBNFOFXIBDPOJXBUPCFIPTUOBNF
    DPOpHDHSPVQ<DQVTIBSFT>
    DPOpHDISPPU@UPWBSZPVS@SPPUGT
    DPOpHOBNFTQBDFVOTIBSFNPVOU
    DPOpHDBQBCJMJUJFTBMMPXBMM
    DPOpHDBQBCJMJUJFTESPQDBQ@TZT@BENJO
    FOE

    View full-size slide

  33. ओཁίϯϙʔωϯτ - CoreAPI(Jardin)
    • GolangΛ༻͍ͯɺlabstack/echoɺjinzhu/gormΛར༻͠։ൃ

    • GETܥ͸ݪଇRedisʹΩϟογϡ͠ɺԠ౴ੑೳΛ޲্

    • Consul EventΛར༻֤ͯ͠ϩʔϧʹϝοηʔδϯάΛૹΔΑ͏ͳ͜ͱ΋
    ΍͍ͬͯΔ

    View full-size slide

  34. ओཁίϯϙʔωϯτ - usermetrics
    • PrometheusΛ࠾༻͠ɺϢʔβʔίϯςφͷ৘ใΛ͢΂ͯू໿

    • ίϯςφͷϦιʔεΛgoogle/cadvisorΛར༻͠ɺexporterܦ༝Ͱऩू

    • Prometheusܦ༝ͰAlertManager͔ΒWebhookͰΦʔτεέʔϧ

    • CPUɺMemoryɺNetworkෛՙͰΞϥʔτΛτϦΨʔ

    View full-size slide

  35. ओཁίϯϙʔωϯτ - Scheduler(kangetsu)
    • ϓϩϏδϣχϯάɾδϣϒεέδϡʔϥʔΛGolangͰrallison/go-workersΛར༻͠։ൃ

    • ετϨʔδྖҬͷ࡞੒ɺquotaͷઃఆ

    • DB࡞੒

    • WordPressͷϓϨΠϯετʔϧ

    View full-size slide

  36. ओཁίϯϙʔωϯτ - Cleaner(skmr)
    • ֤ίϯϙʔωϯτͷσʔλෆ੔߹ΛࣗಈͰݕ஌͠मਖ਼

    • κϯϏϓϩηεͷຣࡴ

    • ίϯςφঢ়ଶͱCMDBͷࠩ෼Λऩଋ

    View full-size slide

  37. ͓·͚ - Hashicorp Software
    • αʔϏεσΟεΧόϦʹConsulɺConsul DNSΛϑϧ׆༻

    • DBʹ֨ೲ͢Δൿಗ৘ใ͸VaultΛར༻͠҉߸Խ

    • OpenStackVMͷϓϩϏδϣχϯά͸TerraformΛར༻

    View full-size slide

  38. ͳͥKubernetesͰ͸ͳ͍ͷ͔ʁ

    View full-size slide

  39. ͳͥKubernetesͰ͸ͳ͍ͷ͔ʁ
    1.ϖύϘ͸ϗεςΟϯάࣄۀΛఏڙ

    2.FastContainerΞʔΩςΫνϟ

    3.࣌୅తͳཁҼ

    4.ࣄۀࠩผԽ

    View full-size slide

  40. KubernetesͱϚωʔδυΫϥ΢υͷҧ͍
    1.UI/UX

    2.ΞʔΩςΫνϟ

    3.ίϯςφىಈ

    4.Φʔτεέʔϧ

    View full-size slide

  41. UI/UX
    ϚωʔδυΫϥ΢υ Kubernetes
    •Ϣʔβʔ͸όοΫΤϯυΛҙࣝ͢Δ͜ͱͳ
    ͘ɺෛՙʹԠͯ͡Ϧιʔε͕૿ݮ͢Δ
    •ςϯϓϨʔτΛબ୒͠ɺscp΋͘͠͸gitpush

    ͰΞϓϦέʔγϣϯΛσϓϩΠ
    •YAMLͰهड़Ͱ͖ΔϦιʔεͰॊೈʹ

    Ϧιʔε͕ఆٛՄೳ
    •೚ҙͷίϯςφΠϝʔδ͕ར༻ՄೳͰ

    σϓϩΠ͸ίϯςφbuild΋͘͠͸mount

    View full-size slide

  42. ΞʔΩςΫνϟ

    View full-size slide

  43. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    ΞʔΩςΫνϟ
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics

    View full-size slide

  44. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    σʔλετΞ
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics

    View full-size slide

  45. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    API&Scheduler
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics

    View full-size slide

  46. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    Node compornents
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics

    View full-size slide

  47. etcd CMDB
    kube-api-server
    Jardin
    kube-scheduler
    kube-proxy
    kubelet
    container runtime haconiwa
    Container Runtime
    kubernetes ϚωʔδυΫϥ΢υ
    ngx_mruby
    skmr
    usermetrics

    View full-size slide

  48. ίϯςφىಈ

    View full-size slide

  49. Kubernetes - ίϯςφىಈ
    • YAMLΛAPIʹ౤ೖ͢Δ͜ͱͰίϯςφىಈ

    • ϔϧενΣοΫ͸ProbeͰ࣮ߦ(HTTP/TCP/ExecCommand)

    • ىಈ࣌ʹߦ͏ͷ͸readinessProbeɺىಈޙ͸livenessProbe

    • ॳظԽॲཧ͸InitContainerɺPostStartͰॳظԽॲཧΛϑοΫՄೳ

    • NඵִؒͰϑοΫ͢Δͱ͔ɺ༗ޮظݶ͕དྷͨΒίϯςφΛఀࢭ͢Δʹ͸ί
    ϯτϩʔϥʔͰಠ࣮ࣗ૷͕ඞཁ

    View full-size slide

  50. ίϯςφىಈϊʔυ͸ͲͷΑ͏ʹܾ·Δʁ
    ϚωʔδυΫϥ΢υ Kubernetes
    •ੜଘ͍ͯ͠Δϊʔυͷϥ΢ϯυϩϏϯ

    •ϊʔυ͕ఀࢭ͍ͯ͠Δ৔߹͸࣍ͷϦΫΤετ
    ͔ΒੜଘϊʔυͰίϯςφىಈ
    •kube-scheduler͕ϊʔυΛબ୒͢Δ

    •ϑΟϧλϦϯά(ϥϕϧɺϦιʔεetc)

    •είΞϦϯά(෼ࢄɺϦιʔεetc)

    View full-size slide

  51. Φʔτεέʔϧ

    View full-size slide

  52. Φʔτεέʔϧ͸ͲͪΒ΋metricsϕʔε
    ϚωʔδυΫϥ΢υ Kubernetes
    •usermetrics(Prometheus)ʹઃఆ͞ΕͨΞϥʔ
    τ͔ΒWebhookͰCoreAPI͕࣮ߦ͞Εɺίϯ
    ςφͷ༧໿࿮͕૿͑Δ

    •εέʔϧΠϯʹ͍ͭͯ͸ɺෛՙ͕Լ͕ͬͨΒ
    ༧໿࿮͕࡟আ͞Εͯɺίϯςφͷ༗ޮظݶ͕
    ౸དྷͨ͠ΒࣗಈͰεέʔϧΠϯ
    •metric-sserverΛΠϯετʔϧ͢Δͱɺ
    HorizontalPodAutoscalerΛར༻ͯ͠Ϧιʔε
    ʹԠͯ͡ΦʔτεέʔϧɾεέʔϧΠϯΛ࣮
    ݱͰ͖Δ

    •ࢦఆͨ͠ෛՙʹͳΔΑ͏ʹϙου͕ௐ੔͞Ε
    Δ

    View full-size slide

  53. ͜͜·Ͱͷ·ͱΊ
    • ϚωʔδυΫϥ΢υ͸Kubernetes΄ͲॊೈͰ͸ͳ͍͕ɺΠϯϑϥʹৄ͘͠
    ͳͯ͘΋࢖͑Δɺ͓खܰΫϥ΢υͰ͋Δ

    • ΞʔΩςΫνϟ΍ΦʔτεέʔϧͷڍಈͳͲྨࣅ఺΋ଟ͋͘Δ

    View full-size slide

  54. ϚωʔδυΫϥ΢υ΋
    Ұ෦Kubernetesར༻

    View full-size slide

  55. ͳͥKubernetesͳͷ͔ʁ
    • ϚτϦϣʔγΧ΋ݕ౼͕ͨ͠ɺγεςϜμ΢ϯ࣌ͷӨڹ͕શஅʹͳΔͨΊ
    ϦεΫߴ͍

    • KubernetesΛࣗ෼͕ͨͪӡ༻͠ɺ஌Δ͜ͱͰΑΓΑ͍Ϋϥ΢υɾωΠςΟ
    ϒ΁(Ṗ

    View full-size slide

  56. Ϋϥ΢υαʔϏεΛఏڙ͠ͳ͕Βɺ
    KubernetesΛӡ༻

    View full-size slide

  57. ίϯςφΦʔέετϨʔγϣϯγεςϜ
    Scheduler
    compute-node
    Container Container Container
    compute-node
    Container Container Container
    εέδϡʔϥʔ͕ϊʔυʹϓϩηεΛׂΓ౰ͯɺ
    ·ͨͦΕΒͷ৘ใ͕ϦΞϧλΠϜʹऩू͞Εɺ
    ࣗ཯తʹಈ͘γεςϜ

    View full-size slide

  58. ݟͨ͜ͱ͋Δͳʁʁʁ

    View full-size slide

  59. ίϯςφΦʔέετϨʔγϣϯγεςϜ͸OSͬΆ͍
    • εέδϡʔϥʔ͕ϓϩηεΛεέδϡʔϦϯά͠ɺͦΕΒͷϑΟʔυόο
    ΫΛड͚ͳ͕Βಈ͖ଓ͚Α͏ͱ͢Δڍಈ͸OSͱࣅ͍ͯΔ఺͕ଟ͍

    • OSͱྨࣅ఺͕ଟ͍ͳΒ͹OSʹ͓͚ΔάουϓϥΫςΟε͕ྑ͍࡞༻Λੜ
    ΉՄೳੑ͕͋Δ

    View full-size slide

  60. ϩάͷӬଓԽ

    View full-size slide

  61. OS
    γϯάϧϊʔυͷ৔߹
    container
    log
    collector

    View full-size slide

  62. ΦʔέετϨʔγϣϯγεςϜ
    Scheduler
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector

    View full-size slide

  63. ΦʔέετϨʔγϣϯγεςϜ
    Scheduler
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector
    compute-node
    Container Container Container
    log
    collector
    log
    collector
    log
    collector

    View full-size slide

  64. ϩάʹ͓͚Δ՝୊
    • ैདྷͷΑ͏ʹ1ϩʔϧ1αʔόͰ͸ͳ͍͜ͱ͔Βɺ1ͭͷαʔόͰෳ਺ͷछ
    ྨͷϩά͕ग़ྗ͞ΕΔ

    • ίϯςφͷूੵ཰ΛߴΊΔͱɺچདྷΑΓϩάͷྲྀྔ͸࣮֬ʹ૿͑Δ

    • ෳ਺ͷछྨͷϩάΛ෼ղɺϧʔςΟϯά͢Δ͜ͱͷෳࡶ͞

    View full-size slide

  65. KafkaͰόοϑΝϦϯά͢Δ
    compute-node
    Container Container Container
    log
    collector
    log log Kafka cluster

    View full-size slide

  66. ,BGLBͷಛ௃
    wߴ଎ॲཧ͕ՄೳͳϝοηʔδϯάγεςϜ
    w෼ࢄॻ͖ࠐΈɺඇಉظϨϓϦέʔγϣϯ˞ɺ෼ࢄಡΈࠐΈ
    wॳظ͸-JOLFEJOʹͯ։ൃɺݱঢ়͸"QBDIFࡒஂʹͯ؅ཧɺ
    -JOLFEJO͔ΒεϐϯΞ΢τͨ͠$POqVFOU͕։ൃΛओମతʹߦ͏
    wݴޠ͸+BWBɺ4DBMBͰ։ൃ
    wΫϥελϦϯά͕ՄೳͰ଱ো֐ੑɺૹୡอূɺݎ࿚ੑ͕ߴ͍

    View full-size slide

  67. LBGLBDMVTUFS
    ඵͰཧղ͢Δ,BGLB
    producer
    (ૹΓख)
    kafka-broker-1
    topic
    partition-0
    kafka-broker-2
    topic
    partition-1
    DPOTVNFSHSPVQ
    consumer
    (ड͚ख)
    consumer
    (ड͚ख)
    consumer
    (ड͚ख)
    ૹΓखͱड͚खΛ/ͱ͢ΔόοϑΝʔ

    View full-size slide

  68. ϩάͷग़ྗ࣌ؒ < Ξοϓϩʔυ࣌ؒ
    compute-node
    Container Container Container
    log
    collector
    log log Kafka cluster
    όϦૣ͍ʂ
    όϦ஗͍ʂʂʂ

    View full-size slide

  69. KafkaΛڬΉϝϦοτ
    wίϯςφىಈϊʔυͷϦιʔεΛ༗ޮར༻͢ΔͨΊʹɺϝΠϯΞϓϦ
    Ҏ֎ͷ࢓ࣄΛͤ͞ͳ͍
    w,BGLBΛڬΉ͜ͱͰσʔλ͕ू໿͞ΕΔ͜ͱͰɺଟ͘ͷίϯςφ͔Β
    ͷ৘ใΛҰՕॴͰѻ͑ΔΑ͏ʹͳΔ
    w,BGLBDPOOFDUΛར༻͢Δ͜ͱͰଟ͘ͷ044ɺϚωʔδυɾαʔϏε
    ͱ࿈ܞͰ͖ΔͷͰ͓ಘ

    View full-size slide

  70. ϖύϘ ϩάج൫(௨শେ࿨ా)
    • ϓϩμΫγϣϯͰར༻͍ͯ͠ΔKubernetes͸͢΂ͯKafkaʹϩάΛू໿
    ͠ɺS3΁ͷϩάอશɺGryalogͰͷݕࡧ͕Մೳʹͳ͍ͬͯΔ

    • ΞϓϦέʔγϣϯ։ൃऀ͸ΞϓϦΛ։ൃ͢Δ͜ͱ͚ͩʹूதͰ͖Δ

    • ϚωʔδυΫϥ΢υ΋େ࿨ాԽ͍ͨ͠

    View full-size slide

  71. Ϋϥ΢υωΠςΟϒΛ૑Δʹ͸
    ·ͣ͸ࣗ෼͕Ϋϥ΢υωΠςΟϒʹͳΔ
    ඞཁ͕͋Δ

    View full-size slide

  72. ࣗ෼ͨͪͰੈքʹͨͩҰͭͷ
    Ϋϥ΢υαʔϏεΛ૑Γ
    ӡ༻͢Δ͜ͱ͸ָ͍͠

    View full-size slide

  73. Ϋϥ΢υαʔϏεΛ૑Γ
    Ϋϥ΢υαʔϏεͱ૊Έ߹Θͤ
    ϢʔβʔΛϋοϐʔʹʂʂʂ

    View full-size slide

  74. ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU
    Զͱ͜ͷϏοά΢Σʔϒʹ৐ͬͯ͘Εʂʂ̍

    View full-size slide