Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Dgraph - A high performance graph database writ...
Search
Yuichi Saito
April 15, 2018
Technology
7
7.3k
Dgraph - A high performance graph database written in pure Go
Go Conference 2018 Spring
Yuichi Saito
April 15, 2018
Tweet
Share
More Decks by Yuichi Saito
See All by Yuichi Saito
失敗から学ぶ - ポストモーテム / Postmotem culture at Wantedly
munisystem
3
37k
Distributed Tracing with OpenCensus at Wantedly, Inc.
munisystem
3
5.1k
OpenCensus による APM の実現と、未来 / Implementing APM with OpenCensus
munisystem
8
6.9k
Effective Health Checking
munisystem
2
740
Learn more about Admission Webhooks
munisystem
1
1.7k
An introduction to monitoring Go Application with OpenCensus
munisystem
1
1.2k
Other Decks in Technology
See All in Technology
AI自体のOps 〜LLMアプリの運用、AWSサービスとOSSの使い分け〜
minorun365
PRO
9
770
IoTシステム開発の複雑さを低減するための統合的アーキテクチャ
kentaro
1
120
Amazon Q Developerの無料利用枠を使い倒してHello worldを表示させよう!
nrinetcom
PRO
2
120
LINE NEWSにおけるバックエンド開発
lycorptech_jp
PRO
0
330
実は強い 非ViTな画像認識モデル
tattaka
3
1.4k
Two Blades, One Journey: Engineering While Managing
ohbarye
4
2.3k
Snowflake ML モデルを dbt データパイプラインに組み込む
estie
0
110
Exadata Database Service on Cloud@Customer セキュリティ、ネットワーク、および管理について
oracle4engineer
PRO
2
1.5k
Introduction to OpenSearch Project - Search Engineering Tech Talk 2025 Winter
tkykenmt
2
150
どちらかだけじゃもったいないかも? ECSとEKSを適材適所で併用するメリット、運用課題とそれらの対応について
tk3fftk
2
240
遷移の高速化 ヤフートップの試行錯誤
narirou
6
1.9k
DeepSeekとは?何がいいの? - Databricksと学ぶDeepSeek! 〜これからのLLMに備えよ!〜
taka_aki
1
160
Featured
See All Featured
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
227
22k
Being A Developer After 40
akosma
89
590k
GitHub's CSS Performance
jonrohan
1030
460k
Writing Fast Ruby
sferik
628
61k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
366
25k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
160
15k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.5k
It's Worth the Effort
3n
184
28k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
13
1k
Scaling GitHub
holman
459
140k
StorybookのUI Testing Handbookを読んだ
zakiyama
28
5.5k
Bash Introduction
62gerente
611
210k
Transcript
©2018 Wantedly, Inc. Dgraph - A high performance graph database
written in pure Go Go Conference 2018 Spring - @munisystem
©2018 Wantedly, Inc. Yuichi Saito @munisystem • Wantedly, Inc. •
Infrastructure Engineer • Microservices • Productivity • My Favorite Golang Products • Istio / Istio • wercker / stern • ory / dockertest ࣗݾհ
©2018 Wantedly, Inc. άϥϑͱ άϥϑͰදݱͰ͖Δͷ ࢁͷ8FC"QQMJDBUJPO͕άϥϑΛแ͍ͯ͠Δ w ίʔυϕʔεͱґଘύοέʔδ w υΩϡϝϯτͱΧςΰϦλά
w ͱจཤྺ w ໊ަ
©2018 Wantedly, Inc. ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ طଘͷάϥϑσʔλϕʔεͷ %HSBQIͷհ
·ͱΊ ΞδΣϯμ
©2018 Wantedly, Inc. άϥϑΛ3%#.4Ͱѻ͏ͷ͍͠ w 3%#.4άϥϑΛϦϨʔγϣϯͱͯ͠දݱ͢Δ w άϥϑΛḷΔʹ+0*/Λ͏ඞཁ͕͋Δ w +0*/ίετ͕͔͔Δ
w ਓɺɺ໊ͳͲ༷ʑͳσʔλؒͷάϥϑΛදݱ͢ΔʹɺҎԼͷͲͪΒ͔͔͠ͳ͍ ਓਓɺਓͳͲͦΕͧΕʹରͯ͠ϦϨʔγϣϯΛఆٛ ϙϦϞʔϑΟοΫͳϦϨʔγϣϯΛఆٛ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔
©2018 Wantedly, Inc. άϥϑσʔλϕʔεݸʑͷͭͳ͕ΓϑΥʔΧεͨ͠σʔλετΞ w શͯͷσʔλ/PEFͱ&EHFͰදݱ͞ΕΔ w /PEF w σʔλͷΤϯςΟςΟΛࣔ͢
w FHϢʔβɺɺ໊ w &EHF w σʔλؒͷؔͱํΛࣔ͢ w FH༑ͩͪɺߪೖɺ໊ަ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. 8BOUFEMZ1FPQMF͕Ϣʔβʹఏڙ͍ͯ͠ΔՁ w ʮ໊ΛऔΓɺਓͱͭͳ͕Δʯ w ໊ͱਓͷ͍͋ͩͷάϥϑΛετΞ͍ͨ͠ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔
©2018 Wantedly, Inc. طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. ॻ͖ࠐΈͷੑೳ͕͔ͬͨ w 3FBEຆͲͷάϥϑσʔλϕʔεͰߴ͍ੑೳΛग़͍ͯ͠Δ w ໊ΛࡱΔɺͱ͍͏ߦҝ8BOUFEMZ1FPQMFͷίΞͳମݧ w ຖຖඵେྔʹૹΒΕͯ͘Δ໊ΛετΞͰ͖Δ͚ͩͷ8SJUFͷύϑΥʔϚϯε͕ඞཁ
w ͜ΕΛຬͨ͢άϥϑσʔλϕʔεଘࡏ͠ͳ͔ͬͨ طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. εέʔϧ͠ͳ͍ߏ • αʔϏεͷʹεέʔϧͰԠ͡ΒΕΔඞཁ͕͋Δ w /FPKσʔλϕʔεଆͰͷσʔλͷγϟʔσΟϯάʹରԠ͍ͯ͠ͳ͍ w 0SJFOU%#$MBTTʢςʔϒϧʣຖʹγϟʔσΟϯάΛ͢Δ
w ͋Δςʔϒϧ͚ͩॻ͖ࠐΈ͕ܹ͍͠ɺΈ͍ͨͳέʔεʹରԠͰ͖ͳ͍ طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. %HSBQIͷհ
©2018 Wantedly, Inc. (Pͷࢄάϥϑσʔλϕʔε w (SBQI2-ϥΠΫͳΫΤϦγϯλοΫε w H31$ͱ)551ͷαϙʔτ w ϋΠύϑΥʔϚϯε
%HSBQIͱ
©2018 Wantedly, Inc. ॻ͖ࠐΈͷύϑΥʔϚϯε͕༏Ε͍ͯΔ w ެࣜͷϕϯνϚʔΫ w ಡΈॻ͖ڞʹ/FPKͷഒ w 1FPQMFͷ̎ޙΛఆͨ͠σʔλྔ
εϧʔϓοτͰϕϯνϚʔΫΛ࣮ࢪ w εέʔϧͳ͠Ͱ͑ΒΕͨ %HSBQIͱ https://blog.dgraph.io/post/benchmark-
©2018 Wantedly, Inc. %HSBQIͷσʔλϞσϧ w %HSBQIͰάϥϑϞσϧͱͯ͠3%'5SJQMFΛ࠾༻ w 8FCϦιʔεʹϝλใΛ༩͢ΔͨΊͷΈ w ϦιʔεͷؔΛ4VCKFDU
1SFEJDBUF 0CKFDUͰදݱ w 4VCKFDUࢧͱͳΔ/PEF w 1SFEJDBUFؔΛද͢&EHF w 0CKFDUऴͱͳΔ/PEF %HSBQIͱ <a> <is> <person> . <b> <is> <person> . <a> <named> <Alice> . <b> <named> <Bob> . <a> <friend> <b> . RDF ٖࣅίʔυʹ͓͚ΔϑϨϯυϦϨʔγϣϯ
©2018 Wantedly, Inc. 3%'5SJQMF Dgraphͱ <a> <is> <person> . <b>
<is> <person> . <a> <named> <Alice> . <b> <named> <Bob> . <a> <friend> <b> . RDF ٖࣅίʔυʹ͓͚ΔϑϨϯυϦϨʔγϣϯ C Bob b a Alice friend named ==
©2018 Wantedly, Inc. 3%'5SJQMF w QFSTPO͕ͳ͍ w 0CKFDU4VCKFDU&OUJUZͱ"UUSJCVUF͕͋Δ w
&OUJUZ6TFSͳͲͷ֓೦Λද͢ w "UUSJCVUFΛද͢ w ຊ࣭తʹϞσϦϯάʹ͕ؔͳ͍ w OBNFEQSFEJDBUF͕͋ΔͳΒͦΕQFSTPO w ·ΔͰ%VDL5ZQJOH Dgraphͱ C Bob b a Alice friend named
©2018 Wantedly, Inc. %HSBQIͷσʔλߏ
©2018 Wantedly, Inc. άϥϑ̎ͭͷσʔλߏʹΑͬͯදݱ͞Ε͍ͯΔ w 1PTUJOH w 1PTUJOH-JTU %HSBQIͷσʔλߏ
©2018 Wantedly, Inc. 1PTUJOH w %HSBQIͷσʔλͷ࠷খ୯Ґ w 1SFEJDBUFͱͦΕʹඥͮ͘ 4VCKFDUͱ0CKFDUΛแ͢Δ w
WBMVF@UZQF͕6*% w 4VCKFDU&OUJUZͰ͋Δ w ͦΕҎ֎"UUSJCVUF w WBMVFʹ࣮σʔλ͕ೖΔ %HSBQIͷσʔλߏ message Posting { fixed64 uid = 1; bytes value = 2; enum ValType { DEFAULT = 0; BINARY = 1; INT = 2; FLOAT = 3; BOOL = 4; DATETIME = 5; GEO = 6; UID = 7; PASSWORD = 8; STRING = 9; } ValType val_type = 3; ... ... enum PostingType { REF=0; VALUE=1; VALUE_LANG=2; } PostingType posting_type = 4; bytes lang_tag = 5; string label = 6; repeated api.Facet facets = 9; uint32 op = 12; uint64 start_ts = 13; uint64 commit_ts = 14; } Protobuf ʹΑΔ Posting ͷఆٛ
©2018 Wantedly, Inc. 1PTUJOH-JTU w 1SFEJDBUFͱ4VCKFDUͷϖΞʹඥͮ͘1PTUJOHΛͻͱ·ͱΊʹͨ͠ͷ w ͻͱ·ͱΊʹ͢Δ͜ͱͰɺ͋ΔϢʔβͷ༑ୡҰཡ ΛҰճͷ୳ࡧͰऔಘͰ͖Δ w
,74ͷLFZʹ1SFEJDBUFͱ4VCKFDUؚ͕·ΕΔ w ͦͷҝ1PTUJOHʹ͕ࣗԿऀͰ͋Δ͔Λ දݱ͢ΔGJFME͕ͳ͍ %HSBQIͷσʔλߏ message PostingList { repeated Posting postings = 1; bytes checksum = 2; uint64 commit = 3; bytes uids = 4; } Protobuf ʹΑΔ Posting ͷఆٛ (Predicate,Subject) -> PostingList KVS ͷετΞํ๏
©2018 Wantedly, Inc. #BEHFS w %HSBQIͷ։ൃνʔϜ͕࡞ͨ͠(PͰॻ͔Εͨ,74 w ϋΠύϑΥʔϚϯε w 3BOEPN3FBE'BDFCPPLͷ,74Ͱ͋Δ3PDLT%#ͷഒ
w 44%ͷ࠷దԽ w "$*%5SBOTBDUJPOͷαϙʔτ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. Կނ#BEHFSΛ࡞ͬͨͷ͔ w ͱͱ3PDLT%#Λ͍ͬͯͨ w ͔͠͠$HPΛ͔ͭ͏σϝϦοτ͕େ͖͗ͨ͢ w (PͷΤίγεςϜʹΕͳ͍
w QQSPGSBDFEFUFDUPS w QUISFBE@DSFBUFͰόάΛҾ͖ى͜͠ɺύϑΥʔϚϯε͕Լ͢Δ w (Pͷࠒ w ($͕ޮ͔ͳ͍ͷͰ$GSFF ΛΕΔͱϝϞϦϦʔΫΛҾ͖ى͜͢ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. Կނ#BEHFSΛ࡞ͬͨͷ͔ w (PͰॻ͔Ε͍ͯΔ#PMU%#Ͳ͏ͩͬͨͷ͔ w #PMU%#3FBE8SJUFڞʹάϩʔόϧۭؒʹ͋ΔTZOD38.VUFYͷϩοΫΛͱΔ w ฒྻॲཧ͕ग़དྷͳ͍
w 1PTUJOH-JTUͷมߋʹେ͖ͳӨڹΛ༩͑ͨ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. #BEHFSͷΈ
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF w ίϯηϓτͰ͖Δ͚ͩγʔΫΛগͳ͘͢Δ w ॻ͖ࠐΈͷϑϩʔ ॻ͖ࠐΈ͕͋ͬͨ߹8"-ͱϝϞϦʢ.FN5BCMFʣʹॻ͖ग़͢
ॻ͖ࠐΉͱ͖ʹ.FN5BCMFʹσʔλ͕ೖΒͳ͍߹ϑΝΠϧʹϑϥογϡ γʔΫͷྔΛݮΒͨ͢ΊʹɺϑΝΠϧ͕ଟ͘ͳ͖ͬͯͨΒϑΝΠϧΛϚʔδͤ͞Δ w $PNQBDUJPO Storage architecture of Badger
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Carol 30 Level 0 (Mem Table)
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Carol 30 muni 30 Level 0 (Mem Table)
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Set: (Alice, 20) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Set: (Alice, 20) ... will remove
(Alice, 10) when merge Storage architecture of Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 mike 50 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 20 mike 50 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20 Bob 20 Level 2
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF w Կ͕خ͍͠ʁ w 8SJUF͕ΔͷϚʔδͷλΠϛϯά͚ͩ w 8"-ॻ͍͍ͯΔ͚Ͳɺ࣮σʔλʹൺΕίετ
w ಡΈࠐΉͱ͖Ϩϕϧ͔Βॱʹߦ͏ͷͰɺஞ࣍σʔλΛߋ৽͢Δඞཁ͕ͳ͍ w ϑΝΠϧͷ͚ͩγʔΫ͢ΕΑ͍ w ϑΝΠϧʹͲͷ,FZؚ͕·Ε͍ͯΔͷ͔֬ೝ͢ΔͨΊͷ#MPPN'JMUFS͕͋Δ w #MPPN'JMUFS͕GBMTFΛฦͨ͠߹ͦͷϑΝΠϧʹ,FZඞͣଘࡏ͠ͳ͍ Storage architecture of Badger
©2018 Wantedly, Inc. "$*%5SBOTBDUJPO w ࣮.VMUJ7FSTJPO$PODVSSFODZ$POUSPM w 3FBEͨ͠ͱ͖ɺߦʹUJNFTUBNQΛ༩͢Δ w ผͷτϥϯβΫγϣϯͰಉ͡ߦʹະίϛοτͷมߋ͕ͬͨ
w มߋϝϞϦʹରͯ͠ߦͳΘΕɺଞͷτϥϯβΫγϣϯ͔Βݟ͑ͳ͍ w ίϛοτͯ͠ߦͷ࣮σʔλͷUJNFTUBNQΛߋ৽ w τϥϯβΫγϣϯͰҰճ3FBEͨ͠ߦΛ͏Ұࢀর w طʹUJNFTUBNQ͕༩͞Ε͍ͯΔͷͰલͷΛฦ͢ ACID Transaction of Badger
©2018 Wantedly, Inc. %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. %HSBQIΛߏ͢Δίϯϙʔωϯτ w [FSP w Ϋϥελͷཧϊʔυ w (SPVQʢޙड़ʣͷཧ
w όοΫΞοϓγϟοτμϯͱ໋͍ͬͨྩΛ(SPVQʹൖͤ͞Δ w TFSWFS w σʔλετΞɺΫΤϦͷ࣮ߦϊʔυ w ̍ͭͷ(SPVQʹඞͣଐ͢Δ %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. (SPVQ w Ҏ্ͷTFSWFSʹΑͬͯߏ͞ΕΔ w ಉ͡(SPVQʹଐ͢Δશͯͷϊʔυಉ͡σʔλΛ࣋ͭ w 3BGUͷ2VPSVN
w γϟʔσΟϯάͷ୯Ґ w (SPVQʹׂΓͯΒΕΔσʔλ1SFEJDBUFຖʹܾ·Δ w ͭ·ΓGSJFOEʹؔ͢Δσʔλશͯಉ͡(SPVQͰཧ͞ΕΔ w Ͳͷ1SFEJDBUFΛड͚͔࣋ͭ[FSPʹΑܾͬͯఆ͞ΕΔ %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. w ͭͳ͕ΓΛอଘ͢ΔσʔλετΞͱͯ͠ άϥϑσʔλϕʔεΛ࠾༻ w %HSBQIطଘͷάϥϑσʔλϕʔε͕ ๊͍͑ͯͨΛղܾ͍ͯ͠Δ w
ॻ͖ࠐΈͷύϑΥʔϚϯε w ਫฏεέʔϦϯά ·ͱΊ