I Know Kung Fu

"Reality is a graph, embrace it!" Sure graph databases are really cool and have a timestamp dated next week, but do you know when you should actually use one? Sometimes living on the bleeding edge pays off and in this talk, I'll show you how you can simplify your application and model your data more naturally with a graph database. They are suited towards a specific problem that is actually quite common in today's applications and they may even apply to that thing you'll be working on during this talk!

I will use Neo4j to demonstrate how to model graph data and integrate it with Rails. We'll start with the basics of graph databases, go over working in a polyglot environment, and get started on our first Rails app using Neo4j. The tools and practices learned here will not only dazzle your boss but make your life easier when you introduce a graph database to your application back home.

Transcript

1. I Know Kung Fu! PRESENTED BY ROGELIO J. SAMOUR HASHROCKET EXPERTLY CRAFTED WEB 1 So how can we simply and power up our app with graph databases? First let's see the current database landscape…
2. databases key/value store columnar document relational 2 key/value: riak, memcached/KC/redis => huge catalog, session data columnar: hbase, cassandra => search engine, SETI document: mongo, couch => insurance, health records relational: mysql, postgres => bank graphs: flockDB, neo4j => social network, genealogy Graph dbs are as general purpose as relational databases
3. graph database 3 graphs: flockDB, neo4j => social network, genealogy Graph dbs are as general purpose as relational databases
4. linked lists trees graphs 4
5. property graph properties nodes relationships 5 Nodes and Relationships are BOTH first class citizens. They both can contain key/value properties.
6. 6 Relationships are directed and typed.
7. Real-world solutions 7
8. recommendations geospatial social computing business intelligence biotechnology your brain access control linguistics catalogs genealogy routing compensation market vectors 8
9. property graph whiteboard-friendly Neo4j REST API 9 - a property graph with nodes, relationships, and properties on both - perfect for complex, highly connected data - scalable: 32 Billion Nodes, 32 Billion Relationships, 64 Billion Properties - hosted, you wanna use this because there's not a lot of hosting options!
10. 1000 person social graph Average of 50 friends/ person Query: does path exist between person A and B? 10 Similar to 6 degrees of Kevin Bacon, given any actor in Hollywood, they can be traced back to Kevin Bacon within 6 steps. Recursive SQL - parent_id - innefficient Closure Tree - implement a manner of storing hierarchical data in a RDBMS - extra code and/or overhead, O(N squared) of storage needed Graphs do this naturally and for you
11. </* BEGIN recursive user id gathering */ WITH RECURSIVE search_graph(id, path) AS ( SELECT users.id, ARRAY[id] FROM users INNER JOIN friendships ON friendships.friend_id = users.id WHERE users.id = 1 UNION ALL SELECT users.id, path || users.id FROM search_graph INNER JOIN friendships ON friendships.user_id = search_graph.id INNER JOIN users u ON friendships.friend_id = u.id WHERE NOT u.id = ANY(path) ) SELECT * FROM search_graph WHERE id != 1 AND length(path) < 4 /* Results | id | path | | 2 | {1, 2} | | 3 | {1, 3} | | 4 | {1, 3, 4} | | 5 | {1, 3, 5} | */ /* END recursive user id gathering */ 11 Similar to 6 degrees of Kevin Bacon, given any actor in Hollywood, they can be traced back to Kevin Bacon within 6 steps. Recursive SQL - parent_id - innefficient Closure Tree - implement a manner of storing hierarchical data in a RDBMS - extra code and/or overhead, O(N squared) of storage needed Graphs do this naturally and for you
12. Relational database took on average: 2000 ms Neo4j took 2 ms 12
13. interacting 13
14. START n=node:users(name=‘Morpheus’) traverse the graph START n=node:users(name=‘Morpheus’) MATCH (n)--()--(foaf) RETURN foaf 14 Graph databases focus on the relationships between data, rather than the commonalities among values.
15. cypher pattern matching SQL-like mutating 15 - query language - declarative grammar with clauses (like SQL) - create, update, delete - aggregation, ordering, limits
16. {location: 'Nebuchadnezzar', disclosure: 'public'} {location: 'Club Hel'} {location: 'Windows Server', disclosure: 'secret'} {location: 'The Matrix', disclosure: 'public'} {type: 'Human'} {type: 'Human'} {type: 'Program'} {type: 'Program'} {type: 'Program'} 16
17. # Create a node in Cypher CREATE neo={name: 'Neo', type: 'Human'} RETURN neo; # Result Node[1] {name: 'Neo', type: 'Human'} 17
18. # Create a relationship in Cypher START neo=node:humans(name='Neo'), morpheus=node:humans(name='Morpheus') CREATE (neo)-[r:knows_location_of { location: 'Nebuchadnezzar', disclosure: 'public'} ]-(morpheus) RETURN r; 18
19. # Result output : knows_location_of[1] { location: 'Nebuchadnezzar', disclosure: 'public'} 19
20. # Neo's immediate network START neo=node:humans(name='Neo') MATCH neo-[:knows_location_of]-network RETURN network; # Result Node[2]{name:"Morpheus", type:"Human"} 20
21. {location: 'Nebuchadnezzar', disclosure: 'public'} {location: 'Club Hel'} {location: 'Windows Server', disclosure: 'secret'} {location: 'The Matrix', disclosure: 'public'} {type: 'Human'} {type: 'Human'} {type: 'Program'} {type: 'Program'} {type: 'Program'} 21
22. # Path to the Keymaker START neo=node:humans(name='Neo') MATCH neo-[r:knows_location_of*1..4]-potentials RETURN potentials.name; 22
23. # Path to the Keymaker: Results +------------------+ | potentials.name | +------------------+ | "Morpheus" | | "Oracle" | | "Merovingian" ; | | "Keymaker" ; | +------------------+ 23
25. implementing 25
26. JRuby and Neo4j.rb 26 If you’re app is already in JRuby then, this is your best bet, otherwise, the REST API and a Ruby wrapper
27. REST API 27 Awesome. Available as a Heroku add-on. Handful of Ruby Gems.
28. Ruby gems that wrap the Neo4J REST API: Neography Architect4r Keymaker 28 KEYMAKER story: neography, needed more, not much else out there at time I needed it.
29. Keymaker multi-layer Ruby wrapper for neo4j 29 The layers in Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
30. Keymaker layer 1 interacts with the Neo4j REST API raw requests 30 The layers in Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
31. Keymaker layer 2 binds the raw HTTP requests into Ruby Objects 31 The layers in Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
32. Keymaker layer 3 implements ActiveModel and treats Nodes and Relationships as Ruby Objects 32 The layers in Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
33. # Add this line to your application's Gemfile: gem 'keymaker' # And then execute: $ bundle install Keymaker 33
34. # Install neo4j for development rake neo4j:install rake neo4j:start # for testing rake neo4j:install RAILS_ENV=test rake neo4j:start RAILS_ENV=test 34
35. # Create a config/neo4j.yml development: server: localhost port: 7474 test: server: localhost port: 7475 35
36. # Create a config/initializers/keymaker.rb if Rails.env.development? || Rails.env.test? database_config = YAML::load_file('config/neo4j.yml') Keymaker.configure do |c| c.server = database_config["#{Rails.env}"]['server'] c.port = database_config["#{Rails.env}"]['port'] end else # Heroku neo4j add-on Keymaker.configure do |c| c.server = ENV['NEO4J_HOST'] c.port = ENV['NEO4J_PORT'].to_i c.username = ENV['NEO4J_LOGIN'] c.password = ENV['NEO4J_PASSWORD'] end end 36
37. class Program include Keymaker::Node property :name index :programs, on: :name def knows_location_of(friend, properties={}) cypher_query = <<-CYPHER START me=node:progams(name={name}), friend=node({friend_node_id}) CREATE (me)-[r:knows_location_of #{properties}]- (friend) RETURN r; CYPHER execute_cypher(cypher_query, {name: name, friend_node_id: friend.node_id}) end end 37
38. class Program include Keymaker::Node property :name index :programs, on: :name def knows_location_of(friend, properties={}) neo_service. create_relationship(:knows_location_of, node_id, friend.node_id, properties) end end 38
39. class Human include Keymaker::Node property :name index :humans, on: :name def knows_location_of(friend, properties={}) neo_service. create_relationship(:knows_location_of, node_id, friend.node_id, properties) end end 39
40. class Human include Keymaker::Node # [...] snipped code def first_degree_network cypher_query = <<-CYPHER START me=node:humans(name={name}) MATCH me-[:knows_location_of]-network RETURN network; CYPHER execute_cypher(cypher_query, {name: name}) end end 40
41. class Human include Keymaker::Node # [...] snipped code def fourth_degree_network cypher_query = <<-CYPHER START me=node:humans(name={name}) MATCH me-[:knows_location_of]-first- [:knows_location_of]-second- [:knows_location_of]-third- [:knows_location_of]-fourth RETURN fourth; CYPHER execute_cypher(cypher_query, {name: name}) end end 41
42. # Create Nodes neo = Human.create({name: 'Neo', type: 'Human'}) # Find neo = Human.find(name: 'Neo') morpheus = Human.find(name: 'Morpheus') oracle = Program.find(name: 'Oracle') merovingian = Program.find(name: 'Merovingian') keymaker = Program.find(name: 'Keymaker') 42
43. # Create Relationships neo.knows_location_of(morpheus, {location: "Nebuchadnezzar", disclosure: "public"}}) morpheus.knows_location_of(oracle, {location: "The Matrix", disclosure: "public"}}) oracle.knows_location_of(merovingian, {location: "Club Hel"}) merovingian.knows_location_of(keymaker, {location: "Windows Server", disclosure: "secret"}) 43
44. # Querying neo.first_degree_network # => [#<Program @name="Morpheus", @node_id=2>] neo.fourth_degree_network # => [#<Program @name="Keymaker", @node_id=5>] 44
45. questions? 45
46. Resources http://docs.neo4j.org http://github.com/therubymug/keymaker 46