Dgraph: Graph database for production environment

Talk given at Gophercon India about Dgraph: https://dgraph.io.

Transcript

1. Dgraph: Graph database for production environment Manish R Jain, Dgraph Labs Gophercon India
2. Agenda 1. Introduction 2. Benchmarks 3. Design 4. Opinions on Go 5. Demo 6. Q & A
3. 1. Introduction: Hi, Diggy! He's badass, and doesn't give a sh**!
4. What are graph databases Optimized for key-value lookups AND edge traversals ( joins ).
5. When should you use a graph database? Relational database with more than 5 foreign ids, better handled by graph databases. Ideal for newsfeeds, recommendation engines, context based search, pattern and fraud detection, and more.
6. What is Dgraph? Dgraph is an open source graph database built for web-scale production environments written entirely in Go. Sharded and Distributed (Distributed Joins, Filters and Sorts) Horizontally scalable. Automatic Replication Consistency (CP in CAP) via Raft Highly Available by design Fault tolerant
7. Why build it? Any company doing anything smart is using graphs. No native scalable solution.
8. 2. Benchmarks
9. Neo4j data loading Dgraph is 100x faster.
10. Neo4j query Dgraph is 3x-6x faster on a read-write workload. At least as fast on read-only. Link to full benchmark code (https://github.com/dgraph-io/benchmarks/tree/master/data/neo4j)
11. Cayley (with Bolt on Macbook) Cayley is a graph layer written in Go. Loading 21M RDFs Dgraph is 9.7x faster Queries (Gremlin) Dgraph is 36.6x faster Queries (MQL) Draph is 5x faster Link to full benchmark code (https://github.com/ankurayadav/graphdb-benchmarks)
12. 3. Design
13. Concepts
14. Concepts
15. Mutations
16. Atomic Consistency (aka Linearizability) Reads after a successful write are guaranteed to return that (or future) write irrespective of which replica is queried.
17. Life of a Query Aka how we solve the problem of distributed joins, distributed lters and distributed sorts e ciently.
18. Life of a Branch
19. Life of a Branch
20. Life of a Branch
21. Life of a Branch
22. Life of a Query
23. Life of a Query { # Find movies by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
24. Life of a Query: Expand Out { # Find movies by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
25. Life of a Query: Apply Filters { # Find movies by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
26. Life of a Query: Sort and Paginate { # Find movies by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
27. Life of a Query: Process Children { # Find movies by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
28. Life of a Query: ToJSON { "steven": [ { "director.film": [ { "initial_release_date": "2008-05-18", "name.en": "Indiana Jones and the Kingdom of the Crystal Skull" }, { "initial_release_date": "1989-05-24", "name.en": "Indiana Jones and the Last Crusade" }, { "initial_release_date": "1984-05-23", "name.en": "Indiana Jones and the Temple of Doom" }, { "initial_release_date": "1981-06-12", "name.en": "Indiana Jones and the Raiders of the Lost Ark" } ] } ] }
29. 4. Opinions on Go
30. Things we like about Go Simplicity of code Concurrency: Goroutines and Channels Pro ling via pprof Benchmarks RPC tracing via contexts Advanced memory management using sync.Pool or Slice tricks. Gofmt Compiler
31. Things pushing us away from Go Having to use Cgo, due to lack of good library support. CJK tokenizer, equivalent of ICU Well designed key-value store
32. Removing Cgo from Dgraph entirely T raffic is driving me nuts. Am going to build a tunnel boring machine and just start digging... 12:05 AM - 18 Dec 2016 13,580 42,245 Elon Musk @elonmusk Follow Fed up with Cgo! Planning to write an LSM based KV store from scratch in #golang based on WiscKey paper and RocksDB. Got any advice? Ping me 11:39 AM - 26 Jan 2017 1 Manish Rai Jain @manishrjain Follow
33. 5. Demo Try it out: curl https://get.dgraph.io -sSf | bash Get Started with Dgraph (https://wiki.dgraph.io/Get_Started)
34. 6. Q & A Got questions? Find me and Dgraph team on breaks, or Knock at our doors. We're staying at Hyatt.
35. Thank you Manish R Jain, Dgraph Labs Gophercon India manish@dgraph.io (mailto:manish@dgraph.io) https://dgraph.io (https://dgraph.io)