Time series database world hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 1

Time series database (TSDB) ͱ͸ • ࣌ܥྻσʔλΛ֨ೲ͢Δ໨తʹಛԽͨ͠σʔλϕʔε • ࣌ܥྻσʔλ: ࣌ؒͰมԽ͢Δ࿈ଓతͳ஋ͷ͜ͱ • IoT ػث͕ൃల͖ͯͯ͠ྲྀߦͬͯΔײ͕͋Δ • ؅ཧαʔό୆਺૿ՃɾϝτϦΫεͷߴ෼ղೳԽͳͲͰ஫໨ • ΠϯϑϥΤϯδχΞతͳ؍఺Ͱ͢ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 2

࣌ܥྻσʔλϕʔεͷྺ࢙ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 3

Round Robin Database (RRD) • ੲ͔Β͋Δγϯϓϧͳ࣌ܥྻσʔλϕʔε • ࣌ܥྻσʔλϕʔεΛֶͿʹ͸ஸ౓Α͍ • γϯϓϧͳͷͰ෼͔Γ΍͍͢ • σʔλϑΥʔϚοτ͕ͱͯ΋γϯϓϧ • ݹ͍σʔλͷॲཧํ๏͸ؙΊࠐΈΛ͢Δɾ্ॻ͖͢Δ • rrdtool ίϚϯυ͸ਓؒʹ͸গ͠೉͍͠ؾ͕͢Δ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 4

round robin database file format 1 +-------------------------------+ | RRD Header | |-------------------------------| | DS Header (one per DS) | |-------------------------------| | RRA Header (one per RRA) | |===============================| < 1 kByte (normally) | RRA Data Area (first RRA) | ................................. The bulk of the space | RRA Data Area (last RRA) | +-------------------------------+ > 1 h$p:/ /oss.oe,ker.ch/rrdtool-trac/wiki/TuningRRD hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 5

rrdtool graph command example 2 $ rrdtool graph ./graph.png \ --start="1360892438-1w" \ --end="1360892438+1w" \ --imgformat=PNG \ --title="Load Average (1min)" \ --height=200 \ --width=500 \ --lower-limit=0 \ DEF:load=test.rrd:load_1min:MAX \ VDEF:a=load,LSLSLOPE \ VDEF:b=load,100,PERCENTNAN \ CDEF:predict=load,POP,a,COUNT,*,b,+ \ SHIFT:predict:604800 \ AREA:load#CC0000:"Load Average (1min)" \ LINE1:predict#00CC00:"Predict" \ LINE2:2#CCCCCC:"Limit" \ 2 h$ps:/ /heartbeats.jp/hbblog/2013/02/predict-rrdtool.html hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 6

Graphite(whisper) • RRD ͷ՝୊Λղܾ͢ΔͨΊʹॻ͔Εͨ 3 • σʔλߏ଄ͳͲ͸ RRD ʹׂͱࣅ͍ͯΔ • carbon ΍ graphite-web ͳͲͷσʔϞϯͱڠௐͯ͠ಈ͘ • ωοτϫʔΫܦ༝Ͱͷσʔλॻ͖ࠐΈɾಡΈग़͕͠ग़དྷΔ • 2016 ೥ݱࡏͰओྲྀʹ͍ۙͰ͋Ζ͏ TSDB • Grafana ౳ͷՄࢹԽπʔϧ΋ॆ࣮͍ͯ͠Δ 3 h$p:/ /graphite.wikidot.com/whisper hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 7

Whisper file format 4 4 h$p:/ /www.aosabook.org/en/graphite.html hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 8

TSDB ʹඞཁʹͳ͖ͬͯͨ͜ͱ • ॻ͖ࠐΈཁٻΛߴ଎ʹࡹ͚Δ͜ͱ • ॻ͖ࠐΈࣗମ͕ߴ଎Ͱ͋Δඞཁ͸͋·Γͳ͍ • ߴ͍σʔλѹॖ཰ • σʔλྔ͕ RDBMS ౳ʹൺ΂ͯେྔʹͳΔͷͰѹॖ͕ඞཁ • ಛఆͷੈ୅ผʹύʔδͳͲ΋ߦ͏ඞཁ͕͋Δ • ࣌ܥྻσʔλΛ(ߴ଎ʹ)౷ܭॲཧग़དྷΔ͜ͱ • ౷ܭॲཧΛߦ͏ݴޠ͸෼͔Γ΍͍͢ͱͳ͓ྑ͍ • ࣌ܥྻσʔλϕʔεࣗମͷ৑௕ੑͷ֬อ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 9

ۙ୅ͷ࣌ܥྻσʔλϕʔεͨͪ 5 5 h$ps:/ /docs.google.com/spreadsheets/d/1sMQe9oOKhMhIVw9WmuCEWdPtAoccJ4a-IuZv4fXDHxM hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 10

ͱʹ͔ͨ͘͘͞Μ͋Δ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 11

Ͳ͏΍ͬͯબͿ͔ • ౰વࣗ෼ͷ༻్ʹԠͯ͡બͿ • ॻ͖ࠐΈ଎౓ʁ • ಡΈࠐΈ଎౓ʁ • σʔλѹॖ཰ʁ • ౷ܭΫΤϦͷॻ͖΍͢͞ʁ • ӡ༻͠΍͢͞ʁ • શ෦ཉ͍͠ͷ͸෼͔Δ͕ඞཁʹԠͯ͡औࣺબ୒͢Δ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 12

ۙ୅ͷ࣌ܥྻσʔλϕʔεͨͪ • ιϑτ΢ΣΞ޻ֶతͳ஌ݟ͔Βઃܭ͠௚͍ͯ͠Δࣄ͕ଟ͍ • σʔλѹॖʹͪΌΜͱࠩ෼ූ߸ԽΛ࢖͏ͱ͔ • λΠϜελϯϓͷΔ ͕ಉ͡ʹͳΓ΍͍͢͜ͱʹண໨ • ϝτϦΫε͸ࣅͨΑ͏ͳ஋ΛਪҠ͠΍͍͢͜ͱʹண໨ • ॻ͖ࠐΈཁٻΛߴ଎ʹॲཧ͢ΔͨΊʹ SEDA Λ࠾༻͢Δͱ͔ • ৑௕Խػߏʹ Paxos, Ra/ Λ࢖͏ͱ͔ • σʔλߏ଄ʹಠࣗͷ Memory Map ΍ LSM-based ͳߏ଄΍ CoW tree Λ࢖͏ͱ͔ • ಺෦ߏ଄Λਖ਼͘͠஌͓ͬͯ͘͜ͱͰͲ͏͍͏ಛੑ͕͋Δ͔෼͔Δ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 13

Time series databases • ࠷ۙ࿦จͱ͔υΩϡϝϯτΛಡΜͩ 3 ͭͷ TSDB Λ঺հ • Gorilla • h/p:/ /www.vldb.org/pvldb/vol8/p1816-teller.pdf • InfluxDB • h/ps:/ /www.influxdata.com/ • BtrDB • h/p:/ /btrdb.io/ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 14

Gorilla 6 • Facebook ͷ։ൃ͍ͯ͠Δ TSDB • ΠϯϑϥͷϝτϦΫεऔಘͷͨΊʹ࢖͍ͬͯΔΒ͍͠ • 2013 ೥ʹ HBase-based ͳ TSDB Λஔ͖׵͑ΔͨΊʹ࡞ΒΕͨ • Timestamp, Value(float) ͦΕͧΕʹѹॖΞϧΰϦζϜΛ͍࣋ͬͯΔ • ϝϞϦ্ͷσʔλߏ଄(TSmap) ΋ಠࣗʹ͍࣋ͬͯΔ • ৑௕Խػߏ(Paxos ϕʔεͷ΋ͷ౳) ͕උΘ͍ͬͯΔ • CorrelaEon(PPMCC)/CharEng/AggregaEons ͷػೳ΋͋ΔͬΆ͍ 6 h$p:/ /www.vldb.org/pvldb/vol8/p1816-teller.pdf hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 15

Gorilla requiremets • ઃܭ࣌ͷཁٻ࢓༷ (refs: 2.2 Gorilla requirements) • Ωʔͷ਺͸ 20 ԯ • ຖ෼ 7 ԯͷσʔλϙΠϯτ(= 9mestamp,value) ͕௥Ճ͞ΕΔ • σʔλ͸ 26 ࣌ؒอ࣋ • ϐʔΫ࣌ʹ 40000/sec ΫΤϦ • ࣌ܥྻσʔλͷཻ౓͸ 15 ඵ·Ͱαϙʔτ • : hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 16

hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 17

InfluxDB • InfluxData ͕ࣾ࡞͍ͬͯΔ Golang ੡ͷ TSDB • 0.9.5 ͔ΒಠࣗͷετϨʔδΤϯδϯΛ։ൃ͍ͯ͠Δ 7 • Time Structured Merge Tree • LSM-tree ϕʔε͕ͩ࣌ܥྻσʔλ༻ʹΧελϚΠζ 8 • σʔλܕ͸ࣗ༝Ͱ Bool, Integer, String ౳σʔλܕ͝ͱʹѹॖΞϧΰϦζϜ͕͋Δ • Float, Integer ౳ͷѹॖख๏͸ Gorilla ͱಉ͡ • String ʹ͸ Snappy ѹॖΛ࢖༻͍ͯ͠Δ • Query language ͸ SQL-like ͳ InfluxQLɻͦΕͳΓʹૣ͍͠ѻ͍΍͍͢ɻ 8 h$ps:/ /speakerdeck.com/pauldix/influxdbs-new-storage-engine-the-=me-structured-merge-tree 7 h$ps:/ /docs.influxdata.com/influxdb/v1.0/concepts/storage_engine hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 18

5 h$ps:/ /docs.google.com/spreadsheets/d/1sMQe9oOKhMhIVw9WmuCEWdPtAoccJ4a-IuZv4fXDHxM hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 19

BtrDB 9 • Berkeley Tree DataBase (FAST '16 Ͱൃද͞Ε͍ͯͨ 10) • Golang Ͱॻ͔Ε͍ͯΔ • σʔλߏ଄͸ CoW tree Λ࢖͍ͬͯΔ • Max/Min/Avg Λอ࣋ͨ͠Γੈ୅Λอ࣋ͨ͠Γ͍ͯ͠Δ • SEDA architecture Λ࠾༻͍ͯ͠Δ໛༷ • Request stage ͱ Read/Write stage ͕͋Δ • Block storage ʹ Ceph αϙʔτ͕͋Δ 10 h%ps:/ /www.usenix.org/system/files/conference/fast16/fast16-papers-andersen.pdf 9 h$p:/ /btrdb.io/ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 20

10 h%ps:/ /www.usenix.org/system/files/conference/fast16/fast16-papers-andersen.pdf hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 21

10 h%ps:/ /www.usenix.org/system/files/conference/fast16/fast16-papers-andersen.pdf hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 22

·ͱΊͱײ૝ • ੲ͔Β͋Δ TSDB ʙ ࠷ۙͷ TSDB Λ؆୯ʹ঺հ • TSDB ʹٻΊΒΕ͍ͯΔࣄͱ࠷ۙͷ TSDB ͕Ͳ͏ରԠ͍ͯ͠Δ͔঺հ • ಺෦ߏ଄ͷυΩϡϝϯτɾ࿦จΛಡΉͱڍಈ͕ਪ࡯Ͱ͖ͯ໘ന͍ • ྫ͑͹ BtrDB ͸౷ܭॲཧ଎౓͕݁ߏૣͦ͏ɺͳͲ • ٯʹσʔλѹॖ཰͸ͦΜͳʹٻΊͯແͦ͞͏ • ಺෦ߏ଄͕෼͔ΔͱٻΊΔ΋ͷʹԠͯ͡ྑ͍ TSDB Λબ΂ΔΑ͏ʹͳΓͦ͏ • ಡΜͰͯษڧʹͳΔ͠໘ന͍ͷͰΈΜͳಡΈ·͠ΐ͏ hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 23

ࢀߟจݙ • h#p:/ /oss.oe+ker.ch/rrdtool-trac/wiki/TuningRRD • h#ps:/ /heartbeats.jp/hbblog/2013/02/predict-rrdtool.html • h#p:/ /graphite.wikidot.com/whisper • h#p:/ /blog.yuuk.io/entry/high-performance-graphite • h#p:/ /www.aosabook.org/en/graphite.html • h#ps:/ /docs.google.com/spreadsheets/d/1sMQe9oOKhMhIVw9WmuCEWdPtAoccJ4a-IuZv4fXDHxM • h#p:/ /www.vldb.org/pvldb/vol8/p1816-teller.pdf • h#ps:/ /docs.influxdata.com/influxdb/v1.0/concepts/storage_engine • h#ps:/ /speakerdeck.com/pauldix/influxdbs-new-storage-engine-the-+me-structured-merge-tree • h#p:/ /btrdb.io/ • h#ps:/ /www.usenix.org/system/files/conference/fast16/fast16-papers-andersen.pdf hbstyle 2016/10/13 - Yoshikawa Ryota ( @rrreeeyyy ) 24