Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Dgraph - A high performance graph database writ...
Search
Yuichi Saito
April 15, 2018
Technology
7
7.4k
Dgraph - A high performance graph database written in pure Go
Go Conference 2018 Spring
Yuichi Saito
April 15, 2018
Tweet
Share
More Decks by Yuichi Saito
See All by Yuichi Saito
失敗から学ぶ - ポストモーテム / Postmotem culture at Wantedly
munisystem
3
38k
Distributed Tracing with OpenCensus at Wantedly, Inc.
munisystem
3
5.2k
OpenCensus による APM の実現と、未来 / Implementing APM with OpenCensus
munisystem
8
7k
Effective Health Checking
munisystem
2
770
Learn more about Admission Webhooks
munisystem
1
1.7k
An introduction to monitoring Go Application with OpenCensus
munisystem
1
1.3k
Other Decks in Technology
See All in Technology
評価の納得感を2段階高める「構造化フィードバック」
aloerina
1
260
kubellが挑むBPaaSにおける、人とAIエージェントによるサービス開発の最前線と技術展望
kubell_hr
1
340
AWS アーキテクチャ作図入門/aws-architecture-diagram-101
ma2shita
24
8.8k
Clineを含めたAIエージェントを 大規模組織に導入し、投資対効果を考える / Introducing AI agents into your organization
i35_267
4
800
Cloud Native Scalability for Internal Developer Platforms
hhiroshell
2
480
Amazon Q Developer for GitHubとAmplify Hosting でサクッとデジタル名刺を作ってみた
kmiya84377
0
3.5k
Perk アプリの技術選定とリリースから1年弱経ってのふりかえり
stomk
0
120
活きてなかったデータを活かしてみた話 / Shirokane Kougyou vol 19
sansan_randd
1
380
TODAY 看世界(?) 是我們在看扣啦!
line_developers_tw
PRO
0
280
原則から考える保守しやすいComposable関数設計
moriatsushi
3
480
CSS、JSをHTMLテンプレートにまとめるフロントエンド戦略
d120145
0
140
Workflows から Agents へ ~ 生成 AI アプリの成長過程とアプローチ~
belongadmin
3
170
Featured
See All Featured
Product Roadmaps are Hard
iamctodd
PRO
53
11k
Embracing the Ebb and Flow
colly
86
4.7k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
43
2.4k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
Visualization
eitanlees
146
16k
Gamification - CAS2011
davidbonilla
81
5.3k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.8k
Statistics for Hackers
jakevdp
799
220k
Building an army of robots
kneath
306
45k
Documentation Writing (for coders)
carmenintech
71
4.9k
Transcript
©2018 Wantedly, Inc. Dgraph - A high performance graph database
written in pure Go Go Conference 2018 Spring - @munisystem
©2018 Wantedly, Inc. Yuichi Saito @munisystem • Wantedly, Inc. •
Infrastructure Engineer • Microservices • Productivity • My Favorite Golang Products • Istio / Istio • wercker / stern • ory / dockertest ࣗݾհ
©2018 Wantedly, Inc. άϥϑͱ άϥϑͰදݱͰ͖Δͷ ࢁͷ8FC"QQMJDBUJPO͕άϥϑΛแ͍ͯ͠Δ w ίʔυϕʔεͱґଘύοέʔδ w υΩϡϝϯτͱΧςΰϦλά
w ͱจཤྺ w ໊ަ
©2018 Wantedly, Inc. ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ طଘͷάϥϑσʔλϕʔεͷ %HSBQIͷհ
·ͱΊ ΞδΣϯμ
©2018 Wantedly, Inc. άϥϑΛ3%#.4Ͱѻ͏ͷ͍͠ w 3%#.4άϥϑΛϦϨʔγϣϯͱͯ͠දݱ͢Δ w άϥϑΛḷΔʹ+0*/Λ͏ඞཁ͕͋Δ w +0*/ίετ͕͔͔Δ
w ਓɺɺ໊ͳͲ༷ʑͳσʔλؒͷάϥϑΛදݱ͢ΔʹɺҎԼͷͲͪΒ͔͔͠ͳ͍ ਓਓɺਓͳͲͦΕͧΕʹରͯ͠ϦϨʔγϣϯΛఆٛ ϙϦϞʔϑΟοΫͳϦϨʔγϣϯΛఆٛ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔
©2018 Wantedly, Inc. άϥϑσʔλϕʔεݸʑͷͭͳ͕ΓϑΥʔΧεͨ͠σʔλετΞ w શͯͷσʔλ/PEFͱ&EHFͰදݱ͞ΕΔ w /PEF w σʔλͷΤϯςΟςΟΛࣔ͢
w FHϢʔβɺɺ໊ w &EHF w σʔλؒͷؔͱํΛࣔ͢ w FH༑ͩͪɺߪೖɺ໊ަ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. Index Free Adjacency w /PEF࣍ͷ/PEFΞΫηε͢ΔͨΊͷΞυϨεΛ࣋ͭ w ୳ࡧࣗͷ࣋ͭ&EHFͷ͚ͩͰΑ͍
w -JOLFE-JTUͳͲͰ࣮͞ΕΔ w 0 Ͱ࣍ͷ/PEFΛࢀরͰ͖Δ w ྫ"͔Β#Λࢀর͢Δ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔ A B C
©2018 Wantedly, Inc. 8BOUFEMZ1FPQMF͕Ϣʔβʹఏڙ͍ͯ͠ΔՁ w ʮ໊ΛऔΓɺਓͱͭͳ͕Δʯ w ໊ͱਓͷ͍͋ͩͷάϥϑΛετΞ͍ͨ͠ ԿނάϥϑσʔλϕʔεΛ͏ͷ͔
©2018 Wantedly, Inc. طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. ॻ͖ࠐΈͷੑೳ͕͔ͬͨ w 3FBEຆͲͷάϥϑσʔλϕʔεͰߴ͍ੑೳΛग़͍ͯ͠Δ w ໊ΛࡱΔɺͱ͍͏ߦҝ8BOUFEMZ1FPQMFͷίΞͳମݧ w ຖຖඵେྔʹૹΒΕͯ͘Δ໊ΛετΞͰ͖Δ͚ͩͷ8SJUFͷύϑΥʔϚϯε͕ඞཁ
w ͜ΕΛຬͨ͢άϥϑσʔλϕʔεଘࡏ͠ͳ͔ͬͨ طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. εέʔϧ͠ͳ͍ߏ • αʔϏεͷʹεέʔϧͰԠ͡ΒΕΔඞཁ͕͋Δ w /FPKσʔλϕʔεଆͰͷσʔλͷγϟʔσΟϯάʹରԠ͍ͯ͠ͳ͍ w 0SJFOU%#$MBTTʢςʔϒϧʣຖʹγϟʔσΟϯάΛ͢Δ
w ͋Δςʔϒϧ͚ͩॻ͖ࠐΈ͕ܹ͍͠ɺΈ͍ͨͳέʔεʹରԠͰ͖ͳ͍ طଘͷάϥϑσʔλϕʔεͷ
©2018 Wantedly, Inc. %HSBQIͷհ
©2018 Wantedly, Inc. (Pͷࢄάϥϑσʔλϕʔε w (SBQI2-ϥΠΫͳΫΤϦγϯλοΫε w H31$ͱ)551ͷαϙʔτ w ϋΠύϑΥʔϚϯε
%HSBQIͱ
©2018 Wantedly, Inc. ॻ͖ࠐΈͷύϑΥʔϚϯε͕༏Ε͍ͯΔ w ެࣜͷϕϯνϚʔΫ w ಡΈॻ͖ڞʹ/FPKͷഒ w 1FPQMFͷ̎ޙΛఆͨ͠σʔλྔ
εϧʔϓοτͰϕϯνϚʔΫΛ࣮ࢪ w εέʔϧͳ͠Ͱ͑ΒΕͨ %HSBQIͱ https://blog.dgraph.io/post/benchmark-
©2018 Wantedly, Inc. %HSBQIͷσʔλϞσϧ w %HSBQIͰάϥϑϞσϧͱͯ͠3%'5SJQMFΛ࠾༻ w 8FCϦιʔεʹϝλใΛ༩͢ΔͨΊͷΈ w ϦιʔεͷؔΛ4VCKFDU
1SFEJDBUF 0CKFDUͰදݱ w 4VCKFDUࢧͱͳΔ/PEF w 1SFEJDBUFؔΛද͢&EHF w 0CKFDUऴͱͳΔ/PEF %HSBQIͱ <a> <is> <person> . <b> <is> <person> . <a> <named> <Alice> . <b> <named> <Bob> . <a> <friend> <b> . RDF ٖࣅίʔυʹ͓͚ΔϑϨϯυϦϨʔγϣϯ
©2018 Wantedly, Inc. 3%'5SJQMF Dgraphͱ <a> <is> <person> . <b>
<is> <person> . <a> <named> <Alice> . <b> <named> <Bob> . <a> <friend> <b> . RDF ٖࣅίʔυʹ͓͚ΔϑϨϯυϦϨʔγϣϯ C Bob b a Alice friend named ==
©2018 Wantedly, Inc. 3%'5SJQMF w QFSTPO͕ͳ͍ w 0CKFDU4VCKFDU&OUJUZͱ"UUSJCVUF͕͋Δ w
&OUJUZ6TFSͳͲͷ֓೦Λද͢ w "UUSJCVUFΛද͢ w ຊ࣭తʹϞσϦϯάʹ͕ؔͳ͍ w OBNFEQSFEJDBUF͕͋ΔͳΒͦΕQFSTPO w ·ΔͰ%VDL5ZQJOH Dgraphͱ C Bob b a Alice friend named
©2018 Wantedly, Inc. %HSBQIͷσʔλߏ
©2018 Wantedly, Inc. άϥϑ̎ͭͷσʔλߏʹΑͬͯදݱ͞Ε͍ͯΔ w 1PTUJOH w 1PTUJOH-JTU %HSBQIͷσʔλߏ
©2018 Wantedly, Inc. 1PTUJOH w %HSBQIͷσʔλͷ࠷খ୯Ґ w 1SFEJDBUFͱͦΕʹඥͮ͘ 4VCKFDUͱ0CKFDUΛแ͢Δ w
WBMVF@UZQF͕6*% w 4VCKFDU&OUJUZͰ͋Δ w ͦΕҎ֎"UUSJCVUF w WBMVFʹ࣮σʔλ͕ೖΔ %HSBQIͷσʔλߏ message Posting { fixed64 uid = 1; bytes value = 2; enum ValType { DEFAULT = 0; BINARY = 1; INT = 2; FLOAT = 3; BOOL = 4; DATETIME = 5; GEO = 6; UID = 7; PASSWORD = 8; STRING = 9; } ValType val_type = 3; ... ... enum PostingType { REF=0; VALUE=1; VALUE_LANG=2; } PostingType posting_type = 4; bytes lang_tag = 5; string label = 6; repeated api.Facet facets = 9; uint32 op = 12; uint64 start_ts = 13; uint64 commit_ts = 14; } Protobuf ʹΑΔ Posting ͷఆٛ
©2018 Wantedly, Inc. 1PTUJOH-JTU w 1SFEJDBUFͱ4VCKFDUͷϖΞʹඥͮ͘1PTUJOHΛͻͱ·ͱΊʹͨ͠ͷ w ͻͱ·ͱΊʹ͢Δ͜ͱͰɺ͋ΔϢʔβͷ༑ୡҰཡ ΛҰճͷ୳ࡧͰऔಘͰ͖Δ w
,74ͷLFZʹ1SFEJDBUFͱ4VCKFDUؚ͕·ΕΔ w ͦͷҝ1PTUJOHʹ͕ࣗԿऀͰ͋Δ͔Λ දݱ͢ΔGJFME͕ͳ͍ %HSBQIͷσʔλߏ message PostingList { repeated Posting postings = 1; bytes checksum = 2; uint64 commit = 3; bytes uids = 4; } Protobuf ʹΑΔ Posting ͷఆٛ (Predicate,Subject) -> PostingList KVS ͷετΞํ๏
©2018 Wantedly, Inc. #BEHFS w %HSBQIͷ։ൃνʔϜ͕࡞ͨ͠(PͰॻ͔Εͨ,74 w ϋΠύϑΥʔϚϯε w 3BOEPN3FBE'BDFCPPLͷ,74Ͱ͋Δ3PDLT%#ͷഒ
w 44%ͷ࠷దԽ w "$*%5SBOTBDUJPOͷαϙʔτ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. Կނ#BEHFSΛ࡞ͬͨͷ͔ w ͱͱ3PDLT%#Λ͍ͬͯͨ w ͔͠͠$HPΛ͔ͭ͏σϝϦοτ͕େ͖͗ͨ͢ w (PͷΤίγεςϜʹΕͳ͍
w QQSPGSBDFEFUFDUPS w QUISFBE@DSFBUFͰόάΛҾ͖ى͜͠ɺύϑΥʔϚϯε͕Լ͢Δ w (Pͷࠒ w ($͕ޮ͔ͳ͍ͷͰ$GSFF ΛΕΔͱϝϞϦϦʔΫΛҾ͖ى͜͢ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. Կނ#BEHFSΛ࡞ͬͨͷ͔ w (PͰॻ͔Ε͍ͯΔ#PMU%#Ͳ͏ͩͬͨͷ͔ w #PMU%#3FBE8SJUFڞʹάϩʔόϧۭؒʹ͋ΔTZOD38.VUFYͷϩοΫΛͱΔ w ฒྻॲཧ͕ग़དྷͳ͍
w 1PTUJOH-JTUͷมߋʹେ͖ͳӨڹΛ༩͑ͨ DdgraphͷσʔλετϨʔδ
©2018 Wantedly, Inc. #BEHFSͷΈ
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF w ίϯηϓτͰ͖Δ͚ͩγʔΫΛগͳ͘͢Δ w ॻ͖ࠐΈͷϑϩʔ ॻ͖ࠐΈ͕͋ͬͨ߹8"-ͱϝϞϦʢ.FN5BCMFʣʹॻ͖ग़͢
ॻ͖ࠐΉͱ͖ʹ.FN5BCMFʹσʔλ͕ೖΒͳ͍߹ϑΝΠϧʹϑϥογϡ γʔΫͷྔΛݮΒͨ͢ΊʹɺϑΝΠϧ͕ଟ͘ͳ͖ͬͯͨΒϑΝΠϧΛϚʔδͤ͞Δ w $PNQBDUJPO Storage architecture of Badger
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Carol 30 Level 0 (Mem Table)
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Carol 30 muni 30 Level 0 (Mem Table)
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (muni, 30) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Set: (Alice, 20) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Set: (Alice, 20) ... will remove
(Alice, 10) when merge Storage architecture of Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 10 Bob 20 Level 1 L W Alice 20 mike 50 Level 0 (Mem Table) L W Carol 10 muni 20
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF Add: (mike, 50) Storage architecture of
Badger L W Alice 20 mike 50 Level 1 L W Level 0 (Mem Table) L W Carol 10 muni 20 Bob 20 Level 2
©2018 Wantedly, Inc. -PH4USVDUVSFE.FSHF5SFF w Կ͕خ͍͠ʁ w 8SJUF͕ΔͷϚʔδͷλΠϛϯά͚ͩ w 8"-ॻ͍͍ͯΔ͚Ͳɺ࣮σʔλʹൺΕίετ
w ಡΈࠐΉͱ͖Ϩϕϧ͔Βॱʹߦ͏ͷͰɺஞ࣍σʔλΛߋ৽͢Δඞཁ͕ͳ͍ w ϑΝΠϧͷ͚ͩγʔΫ͢ΕΑ͍ w ϑΝΠϧʹͲͷ,FZؚ͕·Ε͍ͯΔͷ͔֬ೝ͢ΔͨΊͷ#MPPN'JMUFS͕͋Δ w #MPPN'JMUFS͕GBMTFΛฦͨ͠߹ͦͷϑΝΠϧʹ,FZඞͣଘࡏ͠ͳ͍ Storage architecture of Badger
©2018 Wantedly, Inc. "$*%5SBOTBDUJPO w ࣮.VMUJ7FSTJPO$PODVSSFODZ$POUSPM w 3FBEͨ͠ͱ͖ɺߦʹUJNFTUBNQΛ༩͢Δ w ผͷτϥϯβΫγϣϯͰಉ͡ߦʹະίϛοτͷมߋ͕ͬͨ
w มߋϝϞϦʹରͯ͠ߦͳΘΕɺଞͷτϥϯβΫγϣϯ͔Βݟ͑ͳ͍ w ίϛοτͯ͠ߦͷ࣮σʔλͷUJNFTUBNQΛߋ৽ w τϥϯβΫγϣϯͰҰճ3FBEͨ͠ߦΛ͏Ұࢀর w طʹUJNFTUBNQ͕༩͞Ε͍ͯΔͷͰલͷΛฦ͢ ACID Transaction of Badger
©2018 Wantedly, Inc. %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. %HSBQIΛߏ͢Δίϯϙʔωϯτ w [FSP w Ϋϥελͷཧϊʔυ w (SPVQʢޙड़ʣͷཧ
w όοΫΞοϓγϟοτμϯͱ໋͍ͬͨྩΛ(SPVQʹൖͤ͞Δ w TFSWFS w σʔλετΞɺΫΤϦͷ࣮ߦϊʔυ w ̍ͭͷ(SPVQʹඞͣଐ͢Δ %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. (SPVQ w Ҏ্ͷTFSWFSʹΑͬͯߏ͞ΕΔ w ಉ͡(SPVQʹଐ͢Δશͯͷϊʔυಉ͡σʔλΛ࣋ͭ w 3BGUͷ2VPSVN
w γϟʔσΟϯάͷ୯Ґ w (SPVQʹׂΓͯΒΕΔσʔλ1SFEJDBUFຖʹܾ·Δ w ͭ·ΓGSJFOEʹؔ͢Δσʔλશͯಉ͡(SPVQͰཧ͞ΕΔ w Ͳͷ1SFEJDBUFΛड͚͔࣋ͭ[FSPʹΑܾͬͯఆ͞ΕΔ %HSBQIͷεέʔϦϯά
©2018 Wantedly, Inc. w ͭͳ͕ΓΛอଘ͢ΔσʔλετΞͱͯ͠ άϥϑσʔλϕʔεΛ࠾༻ w %HSBQIطଘͷάϥϑσʔλϕʔε͕ ๊͍͑ͯͨΛղܾ͍ͯ͠Δ w
ॻ͖ࠐΈͷύϑΥʔϚϯε w ਫฏεέʔϦϯά ·ͱΊ