Upgrade to Pro — share decks privately, control downloads, hide ads and more …

サーバモニタリング向け時系列データベースの探究 / The study of time-series database for server monitoring

サーバモニタリング向け時系列データベースの探究 / The study of time-series database for server monitoring

第9回インターネットと運用技術シンポジウム(IOTS2016)での招待講演スライド。

ITシステムの高度化に伴い、サーバモニタリングの需要が高まっている。
その背景には、システムの大規模化およびサーバ仮想化技術の普及によるサーバ台数の増加など運用技術のパラダイムの変遷がある。
その結果、サーバモニタリングを支える基盤となる時系列データベースには、大量のメトリックの書き込みと高精度かつ長期間のデータ保存が求められている。
しかし、これらの要件を満たしつつ、 運用しやすい既存の時系列データベースがないという課題がある。
そこで、サーバ監視サービスMackerelでは、これらの問題を解決するための時系列データベースを開発している。
本講演では、低コストで、24時間365日稼働し、スケールする時系列データベースの実現に向けた講演者の探究を紹介する。

Yuuki Tsubouchi (yuuk1)

December 01, 2016
Tweet

More Decks by Yuuki Tsubouchi (yuuk1)

Other Decks in Technology

Transcript

  1. αʔόϞχλϦϯά޲͚
    ࣌ܥྻσʔλϕʔεͷ୳ڀ
    גࣜձࣾ͸ͯͳ
    ΢ΣϒΦϖϨʔγϣϯΤϯδχΞ
    TSUBOUCHI, Yuuki
    @y_uuk1
    2016/12/01 ୈ9ճΠϯλʔωοτͱӡ༻ٕज़γϯϙδ΢ϜʢIOTS2016ʣট଴ߨԋ

    View full-size slide

  2. ʮӡ༻ͰΧόʔ͢Δʯ
    ʮӡ༻୲౰ऀͷਓபͷ্ʹ੒ཱ͢Δ
    લ࣌୅తͰ໺൮ͳߏ଄෺ʯ
    ʮෆշͳ൰͍͠࢓ࣄΛ΍Δඞཁ͕ͳ͘ͳΔͷ
    ͸ɺਓؒʹͱͬͯͻ͡ΐ͏ͳ෱ࢱ͔΋͠Εͳ
    ͍͕ɺ͋Δ͍͸ͦ͏Ͱͳ͍͔΋͠Εͳ͍ɻʯ
    ୈ 9 ճΠϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ ։࠵Ҋ಺
    http://www.iot.ipsj.or.jp/iots/2016/announcement

    View full-size slide

  3. We define toil as mundane,
    repetitive operational work
    providing no enduring value,
    which scales linearly with service
    growth.
    Betsy Beyer, Chris Jones, Jennifer Petoff, etc. Site Reliability Engineering. Oreilly & Associates Inc.
    April, 2016.

    View full-size slide

  4. Software Engineering
    System Engineering

    View full-size slide

  5. ࣌ܥྻDBͷ։ൃΛྫʹͱΓ
    WebαʔϏεӡ༻ٕज़ͷ
    ΤοηϯεΛ఻͑Δ

    View full-size slide

  6. ͸ͯͳͷ୅දతWebαʔϏε

    View full-size slide

  7. 10೥੒௕͢Δ
    WebαʔϏε

    View full-size slide

  8. 2001೥ தݹPC1୆
    https://www.hatena.ne.jp/company/hatena10th/history

    View full-size slide

  9. 2006೥ αʔό350୆
    http://www.slideshare.net/stanaka/how-to-use-virtualization-technology-in-hatena

    View full-size slide

  10. 2008೥ XenʹΑΔԾ૝ԽͰ୆਺૿Ճ
    http://www.slideshare.net/stanaka/how-to-use-virtualization-technology-in-hatena

    View full-size slide

  11. 2010೥ 1000୆ಥഁ
    http://www.slideshare.net/stanaka/how-to-use-virtualization-technology-in-hatena

    View full-size slide

  12. ݱࡏͰ͸਺ઍ୆ن໛

    View full-size slide

  13. αʔϏε਺50+
    (಺෦޲ؚ͚Ή)

    View full-size slide

  14. αʔϏεಉ͕࢜ෳࡶʹ࿈ܞ͢Δ

    View full-size slide

  15. γεςϜ͕ෳࡶԽͨ݁͠Ռ
    ͳʹ͕ى͖͍ͯΔ͔Θ͔Βͳ͘ͳΔ

    View full-size slide

  16. ຊ೔͸ֶੜ͞Μ΋ͨ͘͞Μ͍Βͬ͠ΌΔͷͰɺWebαʔϏεاۀͷऔΓ૊ΈΛ஌ͬͯ΋Βͬͯɺؔ৺Λ
    ΋͍͚ͬͯͨͩΕ͹ͱࢥ͍·͢ɻ

    ෳࡶԽ͢ΔγεςϜΛલʹ
    ਓʑͷٙ໰͸ਚ͖ͳ͍

    View full-size slide

  17. ؂ࢹΛࣗಈԽ͢Δ
    େਓ਺ͷΦϖϨʔλΛޏ༻͠
    ਓྗͰղܾ

    View full-size slide

  18. ҟৗͷݕ஌ͱ௨஌

    View full-size slide

  19. ҟৗΛൃݟͨ࣌͠ʹ
    γεςϜͷաڈΛ஌Γ͍ͨ

    View full-size slide

  20. CPUར༻཰ͷάϥϑ - 6࣌ؒલʙݱࡏ

    View full-size slide

  21. CPUར༻཰ͷάϥϑ - Ұिؒલʙݱࡏ

    View full-size slide

  22. ࣌ܥྻσʔλΛ௕ظؒอଘ͢Δඞཁ
    ͕͋Δ

    View full-size slide

  23. ࣌ܥྻσʔλͷอଘ
    TSDB
    αʔό
    (ܥྻ໊, λΠϜελϯϓ, ஋)
    (ܥྻ໊, λΠϜελϯϓ, ஋)
    (ܥྻ໊, λΠϜελϯϓ, ஋)

    ؅ཧαʔό
    αʔό
    αʔό
    ΤʔδΣϯτ
    ΤʔδΣϯτ
    ΤʔδΣϯτ

    View full-size slide

  24. ࣌ܥྻσʔλͷن໛ײ
    TSDB
    αʔό
    (ܥྻ໊, λΠϜελϯϓ, ஋)
    (ܥྻ໊, λΠϜελϯϓ, ஋)
    (ܥྻ໊, λΠϜελϯϓ, ஋)

    ؅ཧαʔό
    αʔό
    αʔό
    ΤʔδΣϯτ
    ΤʔδΣϯτ
    ΤʔδΣϯτ
    1000୆
    100 datapoints/min/host
    100,000 writes/min

    View full-size slide

  25. ແࢹͰ͖ͳ͍
    ؂ࢹγεςϜͷӡ༻ίετ

    View full-size slide

  26. Mackerel
    https://mackerel.io

    View full-size slide

  27. https://mackerel.io

    View full-size slide

  28. αʔόͷϝτϦοΫՄࢹԽ

    View full-size slide

  29. MackerelͷΞʔΩςΫνϟ

    View full-size slide

  30. Mackerelͷ࣌ܥྻσʔλͷن໛ײ
    • ΤʔδΣϯτ͕Ϣʔβ͞Μͷϗετ͔Βຖ෼ϝτϦοΫ
    ౤ߘ
    • 2016/01࣌఺ͰΞΫςΟϒΤʔδΣϯτ਺ 10,000+
    • 1ΤʔδΣϯτ͋ͨΓͷϝτϦοΫ਺͸࠷େ200
    • ԾʹฏۉϝτϦοΫ਺Λ100 datapoints/hostͱ͢Δͱ
    ߹ܭૹ৴ϝτϦοΫ਺ 1,000,000 datapoints/min+
    • ϝτϦοΫͷେྔॻ͖ࠐΈʹ଱͑ΒΕΔσʔλϕʔε͕
    ඞཁ

    View full-size slide

  31. MackerelΛࢧ͑ΔTSDB
    Λ୳ڀ͢Δ

    View full-size slide

  32. TSDBͱͷࢥ͍ग़

    View full-size slide

  33. 2012೥ ֶੜ࣌୅ʹ͸ͯͳͰΞϧόΠτ
    • Mackerelͷલ਎ͱͳΔΞϓϦέʔγϣϯ։ൃʹܞΘΔ
    • RRDtoolͱ͍͏2000೥୅ॳ಄ͷTSDBͰάϥϑඳը
    • ͦ΋ͦ΋ωοτϫʔΫσʔϞϯͰͳ͍ͨΊɺίϚϯυ
    Λ͍ͨͨͯ஋ͷߋ৽΍ࢀরΛ͍ͯͨ͠
    • CPANϞδϡʔϧͷ࡞੒ https://metacpan.org/pod/RRDTool::Rawish
    • ϝτϦοΫͷΫϩʔϦϯάͱRRDtoolϕʔεͰάϥϑද
    ࣔͰ͖Δπʔϧͷ࡞੒ https://github.com/yuuki/
    Monitorel

    View full-size slide

  34. http://shoichimasuhara.hatenablog.com/entry/2014/04/10/003617

    View full-size slide

  35. 2014೥ ͸ͯͳೖࣾ
    • Mackerelͷ։ൃͱ͋ΘͤͯGraphiteͷݕূΛ։࢝
    • RRDtool͸εέʔϥϏϦςΟ΍৴པੑͳͲ͞·͟·ͳ
    ఺Ͱ໰୊͕͋ͬͨ
    • Graphite͸γϯϓϧͰ࣮૷Λཧղ͠΍͘͢ɺӡ༻ʹख
    ؒ͸͔͔Δ͕ͦΕͳΓʹΑ͘εέʔϧ͍ͯ͠Δ
    • ϐʔΫ࣌Ͱඵؒ8ສ write IOPS (͍҆SSD ਺ेຕ෼)
    • ϑΝΠϧγεςϜ΍ετϨʔδͷಛੑʹ͋Θͤͯνϡʔ
    χϯά͠ଓ͚͍ͯΔ

    View full-size slide

  36. http://blog.yuuk.io/entry/high-performance-graphite

    View full-size slide

  37. https://speakerdeck.com/yuukit/performance-improvement-of-tsdb-in-mackerel

    View full-size slide

  38. ۙ೥Ͱ͸
    ߴ౓ͳཁٻΛٻΊΒΕΔΑ͏ʹ

    View full-size slide

  39. https://speakerdeck.com/sugiyama88/mackerel-meetup-number-8

    View full-size slide

  40. ߴ଎ͳ਺ઍݸͷܥྻͷදࣔ

    View full-size slide

  41. 1෼ղ૾౓ͷσʔλΛ௕ظؒอ࣋
    https://mackerel.io/ja/docs/entry/overview

    View full-size slide

  42. 1෼ҎԼͷղ૾౓ͷରԠ
    https://mackerel.io/ja/docs/entry/overview

    View full-size slide

  43. ΞʔΩςΫνϟͷ
    ࡮৽͕ඞཁ

    View full-size slide

  44. ݱߦͷΞʔΩςΫνϟ(Graphite)ͷ՝୊
    • εέʔϧΞ΢τʹେ͖ͳਓख͕͔͔Δ
    • ݱࡏ͸ioDriveͷΑ͏ͳߴ଎ͳϑϥογϡετϨʔδΛ༻͍
    ͯεέʔϧΞοϓ
    • ߴ଎ͳετϨʔδ͕ඞཁͳͨΊɺGB୯Ձ͕ඇৗʹߴ͍
    • ϥ΢ϯυϩϏϯσʔλϕʔεͰ͋ΔͨΊɺݹ͍σʔλΛ্ॻ
    ͖͢Δ
    • ͕ͨͬͯ͠ɺݹ͍σʔλΛผͷετϨʔδ΁ୀආͤ͞ΔΑ͏
    ͳ࣮૷ΛՃ͑Δ͜ͱ͕೉͍͠
    • ΦϯϝϞϦͳϦϨʔϓϩΩγΛॻ͖ࠐΈཁٻΛதܧ͢ΔͨΊɺ
    ϦϨʔϓϩΩγ͕མͪΔͱσʔλ͚͕ܽى͖ΔՄೳੑ͕͋Δ

    View full-size slide

  45. https://docs.google.com/spreadsheets/d/1sMQe9oOKhMhIVw9WmuCEWdPtAoccJ4a-IuZv4fXDHxM/pubhtml
    20ݸͷOSSͷTSDB࣮૷

    View full-size slide

  46. TSDBʹؔ͢Δֶज़ݚڀ࿦จʢGorillaʣ
    Tuomas Pelkonen etc. Gorilla: a fast, scalable, in-memory time series database. In Proceedings of the 41st International
    Conference on Very Large Data Bases, Kohala Coast, Hawaii. Volume 8 Issue 12, August 2015. Pages 1816-1827
    • Facebook͕։ൃ͍ͯ͠ΔTSDB
    • ௚ۙ26࣌ؒͷΈΠϯϝϞϦͰ֨ೲ
    • ࣌ܥྻσʔλͷղ૾౓͸15ඵ
    • ϝϞϦʹͷͤΔͨΊʹಠࣗͷѹॖํࣜΛఏҊ
    • ුಈখ਺఺਺͸ಉ͡ܥྻʹ͓͍ͯಉ͡஋͕࿈ଓ͢Δ
    έʔε͕ଟ͍͜ͱʹண໨ͨࠩ͠෼ූ߸Խ
    • ݹ͍σʔλ͸HBaseʹಀ͕͢

    View full-size slide

  47. TSDBʹؔ͢Δֶज़ݚڀ࿦จʢBTrDBʣ
    Michael P Andersen and David E. Culler, University of California, Berkeley. BTrDB: Optimizing Storage System Design for
    Timeseries Processing. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST 16). Feb
    2016. Pages 39-52.
    • IoT޲͚ͷTSDB
    • nanoඵʹରԠ
    • σʔλߏ଄͸copy-on-write treeϕʔεͷ΋ͷͰI/Oޮ཰
    ͕ඇৗʹߴ͍
    • 16.7 million writes/s and 19.8 million reads/s (EC2
    c3.8xlarge)
    • GoݴޠʹΑΔ࣮૷͕ެ։͞Ε͍ͯΔ
    • https://github.com/SoftwareDefinedBuildings/btrdb

    View full-size slide

  48. https://speakerdeck.com/rrreeeyyy/time-series-database-world

    View full-size slide

  49. ͦͷଞͷTSDBͷ՝୊
    • InfluxDB: RaftϓϩτίϧΛ༻͍ͨ৑௕ԽػߏΛ঎༻൛ͷΈͰα
    ϙʔτ
    • ӡ༻͕೉͍͠෼ࢄDB্ʹߏங͞Ε͍ͯΔ
    • OpenTSDB͸HBase, KairosDB͸Cassandra্ʹߏஙͳͲ
    • 1೥ͷϨϙʔτද͕ࣔ஗͍
    • (ະ֬ೝ͕ͩ) OpenTSDBɺInfluxDBͳͲ
    • ޙड़͢Δ௿ղ૾౓ςʔϒϧ࡞੒ʹ૬౰͢Δ࣮૷͕ඞཁ
    • ϓϩμΫγϣϯʹར༻Մೳͳ඼࣭ͷιϑτ΢ΣΞͰ͸ͳ͍
    • BTrDB͸development version
    • ίϛϡχςΟͰͷ࣮੷͕ͳ͍

    View full-size slide

  50. ࣮͸·ܾͩఆ൛͕ͳ͍

    View full-size slide

  51. TSDBʹٻΊΒΕΔ΋ͷ͸
    ͳʹ͔

    View full-size slide

  52. TSDBʹٻΊΒΕΔੑ࣭
    • TSDBͱͯ͠ͷجຊతͳػೳ
    • ৴པੑ
    • εέʔϥϏϦςΟ
    • ߴ଎ͳϨεϙϯελΠϜ
    • ௿ίετ

    View full-size slide

  53. ࠓޙͷTSDBʹٻΊΒΕΔػೳ
    • ߴղ૾౓σʔλͷ௕ظอଘ
    • ௒ߴղ૾౓σʔλͷରԠ
    • େྔͷܥྻͷߴ଎දࣔ
    • (ܥྻʹର͢Δϝλσʔλඥ෇͚)
    • (ػցֶशΛ༻͍ͨҟৗݕ஌΍ྨࣅϝτϦοΫ
    ݕࡧ)

    View full-size slide

  54. ͜ΕΒΛ͢΂ͯຬͨ͢ʹ͸
    Ͳ͏͢Δ͔

    View full-size slide

  55. ύϒϦοΫΫϥ΢υͷ
    ϚωʔδυαʔϏεΛར༻

    View full-size slide

  56. ݹ͍σʔλΛS3ͷΑ͏ͳ
    ͍҆σʔλετϨʔδʹୀආ

    View full-size slide

  57. TSDBʹٻΊΒΕΔੑ࣭
    • TSDBͱͯ͠ͷجຊతͳػೳ
    • ৴པੑ
    • εέʔϥϏϦςΟ
    • ߴ଎ͳϨεϙϯελΠϜ
    • ௿ίετ
    ← Graphiteϕʔεͷ
    ΠϯλϑΣʔε
    ← ϚωʔδυαʔϏεͰ୲อ
    ← ϚωʔδυαʔϏεͰ୲อ
    ← σʔλߏ଄΍
    ɹσʔλ഑ஔͷ޻෉
    ← ݹ͍σʔλΛ͍҆ετϨʔδ΁ୀආ
    ← ͦͷଞɺෳ਺ͷ࠷దԽΛ࣮ࢪ

    View full-size slide

  58. ৽TSDBΞʔΩςΫνϟ

    View full-size slide

  59. write
    datapoints
    ϝοηʔδΩϡʔ ίϯγϡʔϚ
    ΠϯϝϞϦ
    DB
    Webαʔό
    ΦϯσΟεΫ
    DB
    େ༰ྔ
    ετϨʔδ
    read
    datapoints
    subscribe
    flush &
    rollup
    put
    purge table
    read
    read
    read

    View full-size slide

  60. write
    datapoints
    ϝοηʔδΩϡʔ ίϯγϡʔϚ
    ΠϯϝϞϦ
    DB
    Webαʔό
    ΦϯσΟεΫ
    DB
    େ༰ྔ
    ετϨʔδ
    read
    datapoints
    subscribe
    flush &
    rollup
    put
    purge table
    read
    read
    read

    View full-size slide

  61. write
    datapoints
    ϝοηʔδΩϡʔ ίϯγϡʔϚ
    ΠϯϝϞϦ
    DB
    Webαʔό
    ΦϯσΟεΫ
    DB
    େ༰ྔ
    ετϨʔδ
    read
    datapoints
    subscribe
    flush &
    rollup
    put
    purge table
    read
    read
    read
    3छͷDB
    ௚ۙ1࣌ؒ
    ίʔϧυσʔλ
    ϗοτσʔλ

    View full-size slide

  62. write
    datapoints
    ϝοηʔδΩϡʔ ίϯγϡʔϚ
    ΠϯϝϞϦ
    DB
    Webαʔό
    ΦϯσΟεΫ
    DB
    େ༰ྔ
    ετϨʔδ
    read
    datapoints
    subscribe
    flush &
    rollup
    put
    purge table
    read
    read
    read
    ΩϡʔͷલޙͰܥΛ෼ׂ

    View full-size slide

  63. write
    datapoints
    ϝοηʔδΩϡʔ ίϯγϡʔϚ
    ΠϯϝϞϦ
    DB
    Webαʔό
    ΦϯσΟεΫ
    DB
    େ༰ྔ
    ετϨʔδ
    read
    datapoints
    subscribe
    flush &
    rollup
    put
    purge table
    read
    read
    read
    3छDB͔ΒಡΈ෼͚Δ
    WebΞϓϦέʔγϣϯ
    Graphiteޓ׵ͷΠϯλϑΣʔε

    View full-size slide

  64. write
    datapoints
    ϝοηʔδΩϡʔ ίϯγϡʔϚ
    ΠϯϝϞϦ
    DB
    Webαʔό
    ΦϯσΟεΫ
    DB
    େ༰ྔ
    ετϨʔδ
    read
    datapoints
    subscribe
    put &
    rollup
    put
    purge table
    read
    read
    read
    ,JOFTJT -BNCEB
    3FEJT$MVTUFS
    %ZOBNP%#
    4
    ϚωʔδυαʔϏεͳͷͰ
    جຊతʹεέʔϧΞ΢τ΍

    ৑௕ੑ͸อূ͞Ε͍ͯΔ

    View full-size slide

  65. ৽TSDBͷσʔλߏ଄

    View full-size slide

  66. ΦϯσΟεΫDB (DynamoDB)ͷεΩʔϚઃܭ
    ߦΩʔ ྻσʔλ
    ϝτϦοΫ໊ [ timestamp(int32):value(float64), … ]
    TFSWFSMPBEBWH < ʜ>
    TFSWFSMPBEBWH < ʜ>
    ಉ͡ܥྻͷϝτϦοΫͷϦετ

    View full-size slide

  67. Time-based range partitioning (DynamoDB)
    ߦΩʔ ྻσʔλ
    ϝτ
    ϦοΫ

    [ timestamp(int
    32):value(float
    64), … ]
    TFSWFS
    MPBEB
    WH
    <ʜ>
    2016/11/30 2016/11/29 2016/11/22
    ߦΩʔ ྻσʔλ
    ϝτ
    ϦοΫ

    [ timestamp(int
    32):value(float
    64), … ]
    TFSWFS
    MPBEB
    WH
    <ʜ>
    ߦΩʔ ྻσʔλ
    ϝτ
    ϦοΫ

    [ timestamp(int
    32):value(float
    64), … ]
    TFSWFS
    MPBEB
    WH
    <ʜ>
    ……
    େྔϨίʔυ࡟আॲཧ͸ෛՙ͕ߴ͍ͨΊ
    ݹ͍೔෇ςʔϒϧΛυϩοϓ͢ΔΑ͏ʹ
    େ༰ྔ
    ετϨʔδ
    ྻσʔλΛҰఆαΠζʹऩΊΔ෭࣍తޮՌ

    View full-size slide

  68. Write-through cache (Redis +DynamoDB)
    Redis ߦΩʔ ྻσʔλ
    ϝτϦο
    Ϋ໊
    [ timestamp(int32):va
    lue(float64), … ]
    TFSWFSMP
    BEBWH
    <ʜ>
    DynamoDB
    ίϯγϡʔϚ
    ҰఆҎ্ಉ͡ܥྻͷσʔλϙΠϯτ͕
    ͨ·ͬͨΒϑϥογϡ
    ౸ணͨ͠
    σʔλϙΠϯτΛ
    ΠϯϝϞϦDBʹ֨ೲ

    View full-size slide

  69. Graphiteޓ׵WebαʔόͷI/OଟॏԽ
    Redis
    DynamoDB
    S3





    BatchGetItem API (100items)
    εϨου
    εϨουʹΑΓ
    I/OଟॏԽ͠
    ϨεϙϯελΠϜ
    Λߴ଎Խ

    View full-size slide

  70. ৽TSDBʹ͓͚Δͦͷଞͷ࠷దԽΞΠσΞ
    • ௿ղ૾౓ςʔϒϧͷߏஙʹΑΔɺ௕ظϨϙʔ
    τදࣔͷϨεϙϯεߴ଎Խ
    • Amazon AthenaΛ༻͍ͨS3ͷσʔλ഑ஔ࣮
    ૷ͷ؆қԽ
    • SIMDԋࢉʹΑΔ࣌ܥྻσʔλू໿ԋࢉͷߴ
    ଎Խ

    View full-size slide

  71. ৽TSDBͷܽ఺

    View full-size slide

  72. ৽TSDBΞʔΩςΫνϟͷܽ఺
    • ΦϯϓϨϛε؀ڥͰ͸࣮ݱͰ͖ͳ͍
    • ͨͩ͠ɺӡ༻ίετΛແࢹ͢Ε͹ࣅͨΑ͏ͳOSS࣮
    ૷ʹஔ͖׵͑͸Մೳ
    • Kinesis=>KafkaɺDynamoDB=>CassandraͳͲ
    • ίϯϙʔωϯτ͕ଟ͍ͨΊɺҰݟෳࡶͰ͋Δ
    • ൚༻DBʹ͍ͭͯੑೳ΍ίετΛ࠷దԽ͠Α͏ͱ͢Δ
    ͱɺͲ͏ͯ͠΋ෳ਺ͷಛੑΛ΋ͭDBΛ૊Έ߹ΘͤΔ
    ඞཁ͕͋Δ

    View full-size slide

  73. ৽TSDBΞʔΩςΫνϟͷ֓ཁ·ͱΊ
    • ৴པੑ޲্ͱӡ༻ίετ࡟ݮͷͨΊϚωʔδ
    υαʔϏεΛར༻
    • ੑೳͱۚમίετͷ࠷దԽͷͨΊʹɺΞΫη
    εύλʔϯʹԠͯ͡ෳ਺ͷҟͳΔDBΛซ༻
    • ϝοηʔδΩϡʔΛڬΉ͜ͱͰγεςϜΛૄ
    ݁߹Խ͢Δ

    View full-size slide

  74. WebαʔϏεӡ༻
    ͷΤοηϯε

    View full-size slide

  75. ܭࢉػΞʔΩςΫνϟ/OSͷجຊͷԠ༻
    • ϝϞϦͱετϨʔδ
    • write-through cache
    • ௿ղ૾౓ςʔϒϧ͸ࣄલܭࢉ΍Ωϟογϡͷ
    ֓೦ʹ૬౰

    View full-size slide

  76. ҟͳΔෳ਺ͷDBͷ૊Έ߹Θͤ
    • WebΞϓϦέʔγϣϯͷੈքͰ͸Α͘࠾༻͞Ε
    Δख๏
    • ϝΠϯσʔλϕʔε͸RDBMSͱͯ͠MySQL΍
    PostgreSQLΛར༻
    • ੑೳ͕ٻΊΒΒΕΔͱ͜Ζ͸ɺKVSͱͯ͠Redis
    ΍memcachedΛར༻
    • ݕࡧΤϯδϯͱͯ͠ɺElasticsearch΍SolrΛར༻

    View full-size slide

  77. ٕज़બ୒
    • OSSͰ͋Ε͹ίϛϡχςΟϕʔε͔Ͳ͏͔
    • ੈͷதʹӡ༻ࣄྫ͕͋Δ͔
    • ୤ग़Մೳ͔Ͳ͏͔
    • ଞͷࣄۀʹطʹ࣮੷͕͋Δ͔΋͘͠͸ԣల։
    Ͱ͖Δ͔
    • ϏδωεΠϯύΫτΛߟྀ͢Δ

    View full-size slide

  78. http://developer.hatenastaff.com/entry/2015/12/25/140233

    View full-size slide

  79. http://blog.yuuk.io/entry/infra-for-newlang

    View full-size slide

  80. ϞμϯͳΞʔΩςΫνϟ
    • Serverless Architecture
    • Microservices
    • Immutable Infrastructure

    View full-size slide

  81. WebαʔϏεӡ༻ͷಛੑͱ՝୊
    • WebαʔϏε͸σʔλʹՁ஋͕͋Δ
    • σʔλΛ΋ͭ(stateful)αʔόͷӡ༻ʹ͸෼ࢄ
    γεςϜͷ؍఺Ͱ՝୊͕ଟ͍
    • ϚωʔδυαʔϏεΛ࢖͏͚ͩͰ͸ɺ݁ہӡ
    ༻͕Ϋϥ΢υͷԼଆʹҠߦ͢Δ͚ͩ
    • ੈͷத͔Βӡ༻͕ফ͑ΔΘ͚Ͱ͸ͳ͍

    View full-size slide

  82. WebαʔϏεӡ༻ͷະདྷ

    View full-size slide

  83. ਓ͕ؒཧղͰ͖ΔγεςϜ
    ਓ͕ؒཧղͰ͖ͳ͍γεςϜ

    View full-size slide

  84. ਓ͕ؒཧղͰ͖ΔγεςϜ
    • γεςϜΛཧղՄೳͳখ͞ͳ୯Ґʹ෼ׂ͢Δ
    • খ͞ͳίϯϙʔωϯτΛཁٻʹԠͯ͡૊Έ߹
    ΘͤΔ
    • ਓखʹΑΔӡ༻ίετΛ࠷খԽ͢Δ

    View full-size slide

  85. ਓ͕ؒཧղͰ͖ͳ͍γεςϜ
    • શࣗಈͳγεςϜ
    • ੜ໋׆ಈͷΑ͏ʹγεςϜ͕ࣗ཯తʹम෮Ͱ
    ͖Δ
    • ͨͩ͠ɺશମ૾͸ਓ͕ؒཧղͰ͖ͳ͍ͷͰɺ
    ͨͱ͑όά͕͋ͬͯ΋ຊ౰ʹಈ͖ଓ͚ΔͳΜ
    Β͔ͷอূ͕ඞཁ

    View full-size slide

  86. ·ͱΊ
    • αʔόϞχλϦϯά͸γεςϜӡ༻ͷॏཁͳج൫
    • αʔόϞχλϦϯάͷίϯϙʔωϯτͷ͏ͪɺಛʹ
    TSDBʹ՝୊͕͋Δ
    • TSDBͷཁٻϨϕϧ͕ߴ·͍ͬͯΔҰํͰɺܾఆ൛͕ͳ͍
    • ϚωʔδυαʔϏε͔ͭෳ਺ͷ൚༻ετϨʔδΛ૊Έ߹
    ΘͤͨTSDBΛ࣮૷͢Δख๏Λ঺հ
    • TSDBͷ։ൃΛྫʹWebαʔϏεͷӡ༻ࣄ৘Λؒ઀తʹ
    ঺հ

    View full-size slide

  87. http://hatenacorp.jp/recruit/fresh/operation-engineer

    View full-size slide

  88. ຊεϥΠυͷKeynoteςϯϓϨʔτͱͯ͠
    shoya140͞ΜͷZebra(http://shoya.io/blog/zebra/) Λ࢖Θ͍͖ͤͯͨͩ·ͨ͠
    αʔόϞχλϦϯά޲͚
    ࣌ܥྻσʔλϕʔεͷ୳ڀ
    גࣜձࣾ͸ͯͳ
    ΢ΣϒΦϖϨʔγϣϯΤϯδχΞ
    TSUBOUCHI, Yuuki
    @y_uuk1

    View full-size slide