Dgraph: Graph database for production environment

Dgraph: Graph database for production environment

Talk given at Gophercon India about Dgraph: https://dgraph.io.

63954a3369ccc7025d19b4ab3ea87433?s=128

Manish R Jain

February 24, 2017
Tweet

Transcript

  1. Dgraph: Graph database for production environment Manish R Jain, Dgraph

    Labs Gophercon India
  2. Agenda 1. Introduction 2. Benchmarks 3. Design 4. Opinions on

    Go 5. Demo 6. Q & A
  3. 1. Introduction: Hi, Diggy! He's badass, and doesn't give a

    sh**!
  4. What are graph databases Optimized for key-value lookups AND edge

    traversals ( joins ).
  5. When should you use a graph database? Relational database with

    more than 5 foreign ids, better handled by graph databases. Ideal for newsfeeds, recommendation engines, context based search, pattern and fraud detection, and more.
  6. What is Dgraph? Dgraph is an open source graph database

    built for web-scale production environments written entirely in Go. Sharded and Distributed (Distributed Joins, Filters and Sorts) Horizontally scalable. Automatic Replication Consistency (CP in CAP) via Raft Highly Available by design Fault tolerant
  7. Why build it? Any company doing anything smart is using

    graphs. No native scalable solution.
  8. 2. Benchmarks

  9. Neo4j data loading Dgraph is 100x faster.

  10. Neo4j query Dgraph is 3x-6x faster on a read-write workload.

    At least as fast on read-only. Link to full benchmark code (https://github.com/dgraph-io/benchmarks/tree/master/data/neo4j)
  11. Cayley (with Bolt on Macbook) Cayley is a graph layer

    written in Go. Loading 21M RDFs Dgraph is 9.7x faster Queries (Gremlin) Dgraph is 36.6x faster Queries (MQL) Draph is 5x faster Link to full benchmark code (https://github.com/ankurayadav/graphdb-benchmarks)
  12. 3. Design

  13. Concepts

  14. Concepts

  15. Mutations

  16. Atomic Consistency (aka Linearizability) Reads after a successful write are

    guaranteed to return that (or future) write irrespective of which replica is queried.
  17. Life of a Query Aka how we solve the problem

    of distributed joins, distributed lters and distributed sorts e ciently.
  18. Life of a Branch

  19. Life of a Branch

  20. Life of a Branch

  21. Life of a Branch

  22. Life of a Query

  23. Life of a Query { # Find movies by Steven

    Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
  24. Life of a Query: Expand Out { # Find movies

    by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
  25. Life of a Query: Apply Filters { # Find movies

    by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
  26. Life of a Query: Sort and Paginate { # Find

    movies by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
  27. Life of a Query: Process Children { # Find movies

    by Steven Spielberg named 'Indiana', sorted in desc order. steven(id: m.06pj8) { director.film @filter(anyof(name.en, "indiana")) (orderdesc: initial_release_date) { name.en initial_release_date } } }
  28. Life of a Query: ToJSON { "steven": [ { "director.film":

    [ { "initial_release_date": "2008-05-18", "name.en": "Indiana Jones and the Kingdom of the Crystal Skull" }, { "initial_release_date": "1989-05-24", "name.en": "Indiana Jones and the Last Crusade" }, { "initial_release_date": "1984-05-23", "name.en": "Indiana Jones and the Temple of Doom" }, { "initial_release_date": "1981-06-12", "name.en": "Indiana Jones and the Raiders of the Lost Ark" } ] } ] }
  29. 4. Opinions on Go

  30. Things we like about Go Simplicity of code Concurrency: Goroutines

    and Channels Pro ling via pprof Benchmarks RPC tracing via contexts Advanced memory management using sync.Pool or Slice tricks. Gofmt Compiler
  31. Things pushing us away from Go Having to use Cgo,

    due to lack of good library support. CJK tokenizer, equivalent of ICU Well designed key-value store
  32. Removing Cgo from Dgraph entirely T raffic is driving me

    nuts. Am going to build a tunnel boring machine and just start digging... 12:05 AM - 18 Dec 2016 13,580 42,245 Elon Musk @elonmusk Follow Fed up with Cgo! Planning to write an LSM based KV store from scratch in #golang based on WiscKey paper and RocksDB. Got any advice? Ping me 11:39 AM - 26 Jan 2017 1 Manish Rai Jain @manishrjain Follow
  33. 5. Demo Try it out: curl https://get.dgraph.io -sSf | bash

    Get Started with Dgraph (https://wiki.dgraph.io/Get_Started)
  34. 6. Q & A Got questions? Find me and Dgraph

    team on breaks, or Knock at our doors. We're staying at Hyatt.
  35. Thank you Manish R Jain, Dgraph Labs Gophercon India manish@dgraph.io

    (mailto:manish@dgraph.io) https://dgraph.io (https://dgraph.io)