Upgrade to Pro — share decks privately, control downloads, hide ads and more …

I Know Kung Fu

I Know Kung Fu

"Reality is a graph, embrace it!" Sure graph databases are really cool and have a timestamp dated next week, but do you know when you should actually use one? Sometimes living on the bleeding edge pays off and in this talk, I'll show you how you can simplify your application and model your data more naturally with a graph database. They are suited towards a specific problem that is actually quite common in today's applications and they may even apply to that thing you'll be working on during this talk!

I will use Neo4j to demonstrate how to model graph data and integrate it with Rails. We'll start with the basics of graph databases, go over working in a polyglot environment, and get started on our first Rails app using Neo4j. The tools and practices learned here will not only dazzle your boss but make your life easier when you introduce a graph database to your application back home.

therubymug

June 26, 2012
Tweet

Other Decks in Programming

Transcript

  1. I Know Kung Fu! PRESENTED BY ROGELIO J. SAMOUR HASHROCKET

    EXPERTLY CRAFTED WEB 1 So how can we simply and power up our app with graph databases? First let's see the current database landscape…
  2. databases key/value store columnar document relational 2 key/value: riak, memcached/KC/redis

    => huge catalog, session data columnar: hbase, cassandra => search engine, SETI document: mongo, couch => insurance, health records relational: mysql, postgres => bank graphs: flockDB, neo4j => social network, genealogy Graph dbs are as general purpose as relational databases
  3. graph database 3 graphs: flockDB, neo4j => social network, genealogy

    Graph dbs are as general purpose as relational databases
  4. property graph properties nodes relationships 5 Nodes and Relationships are

    BOTH first class citizens. They both can contain key/value properties.
  5. recommendations geospatial social computing business intelligence biotechnology your brain access

    control linguistics catalogs genealogy routing compensation market vectors 8
  6. property graph whiteboard-friendly Neo4j REST API 9 - a property

    graph with nodes, relationships, and properties on both - perfect for complex, highly connected data - scalable: 32 Billion Nodes, 32 Billion Relationships, 64 Billion Properties - hosted, you wanna use this because there's not a lot of hosting options!
  7. 1000 person social graph Average of 50 friends/ person Query:

    does path exist between person A and B? 10 Similar to 6 degrees of Kevin Bacon, given any actor in Hollywood, they can be traced back to Kevin Bacon within 6 steps. Recursive SQL - parent_id - innefficient Closure Tree - implement a manner of storing hierarchical data in a RDBMS - extra code and/or overhead, O(N squared) of storage needed Graphs do this naturally and for you
  8. </* BEGIN recursive user id gathering */ WITH RECURSIVE search_graph(id,

    path) AS ( SELECT users.id, ARRAY[id] FROM users INNER JOIN friendships ON friendships.friend_id = users.id WHERE users.id = 1 UNION ALL SELECT users.id, path || users.id FROM search_graph INNER JOIN friendships ON friendships.user_id = search_graph.id INNER JOIN users u ON friendships.friend_id = u.id WHERE NOT u.id = ANY(path) ) SELECT * FROM search_graph WHERE id != 1 AND length(path) < 4 /* Results | id | path | | 2 | {1, 2} | | 3 | {1, 3} | | 4 | {1, 3, 4} | | 5 | {1, 3, 5} | */ /* END recursive user id gathering */ 11 Similar to 6 degrees of Kevin Bacon, given any actor in Hollywood, they can be traced back to Kevin Bacon within 6 steps. Recursive SQL - parent_id - innefficient Closure Tree - implement a manner of storing hierarchical data in a RDBMS - extra code and/or overhead, O(N squared) of storage needed Graphs do this naturally and for you
  9. START n=node:users(name=‘Morpheus’) traverse the graph START n=node:users(name=‘Morpheus’) MATCH (n)--()--(foaf) RETURN

    foaf 14 Graph databases focus on the relationships between data, rather than the commonalities among values.
  10. cypher pattern matching SQL-like mutating 15 - query language -

    declarative grammar with clauses (like SQL) - create, update, delete - aggregation, ordering, limits
  11. {location: 'Nebuchadnezzar', disclosure: 'public'} {location: 'Club Hel'} {location: 'Windows Server',

    disclosure: 'secret'} {location: 'The Matrix', disclosure: 'public'} {type: 'Human'} {type: 'Human'} {type: 'Program'} {type: 'Program'} {type: 'Program'} 16
  12. # Create a node in Cypher CREATE neo={name: 'Neo', type:

    'Human'} RETURN neo; # Result Node[1] {name: 'Neo', type: 'Human'} 17
  13. # Create a relationship in Cypher START neo=node:humans(name='Neo'), morpheus=node:humans(name='Morpheus') CREATE

    (neo)-[r:knows_location_of { location: 'Nebuchadnezzar', disclosure: 'public'} ]-(morpheus) RETURN r; 18
  14. {location: 'Nebuchadnezzar', disclosure: 'public'} {location: 'Club Hel'} {location: 'Windows Server',

    disclosure: 'secret'} {location: 'The Matrix', disclosure: 'public'} {type: 'Human'} {type: 'Human'} {type: 'Program'} {type: 'Program'} {type: 'Program'} 21
  15. # Path to the Keymaker: Results +------------------+ | potentials.name |

    +------------------+ | "Morpheus" | | "Oracle" | | "Merovingian" ; | | "Keymaker" ; | +------------------+ 23
  16. 24

  17. JRuby and Neo4j.rb 26 If you’re app is already in

    JRuby then, this is your best bet, otherwise, the REST API and a Ruby wrapper
  18. Ruby gems that wrap the Neo4J REST API: Neography Architect4r

    Keymaker 28 KEYMAKER story: neography, needed more, not much else out there at time I needed it.
  19. Keymaker multi-layer Ruby wrapper for neo4j 29 The layers in

    Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
  20. Keymaker layer 1 interacts with the Neo4j REST API raw

    requests 30 The layers in Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
  21. Keymaker layer 2 binds the raw HTTP requests into Ruby

    Objects 31 The layers in Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
  22. Keymaker layer 3 implements ActiveModel and treats Nodes and Relationships

    as Ruby Objects 32 The layers in Keymaker try to emulate the | You can always write a Java plugin for layers provided by the Neo4j.rb gem | neo4j and access it via the REST API (with the difference being that with | if needed. JRuby you can tap into neo4j's Java API directly). |
  23. # Add this line to your application's Gemfile: gem 'keymaker'

    # And then execute: $ bundle install Keymaker 33
  24. # Install neo4j for development rake neo4j:install rake neo4j:start #

    for testing rake neo4j:install RAILS_ENV=test rake neo4j:start RAILS_ENV=test 34
  25. # Create a config/initializers/keymaker.rb if Rails.env.development? || Rails.env.test? database_config =

    YAML::load_file('config/neo4j.yml') Keymaker.configure do |c| c.server = database_config["#{Rails.env}"]['server'] c.port = database_config["#{Rails.env}"]['port'] end else # Heroku neo4j add-on Keymaker.configure do |c| c.server = ENV['NEO4J_HOST'] c.port = ENV['NEO4J_PORT'].to_i c.username = ENV['NEO4J_LOGIN'] c.password = ENV['NEO4J_PASSWORD'] end end 36
  26. class Program include Keymaker::Node property :name index :programs, on: :name

    def knows_location_of(friend, properties={}) cypher_query = <<-CYPHER START me=node:progams(name={name}), friend=node({friend_node_id}) CREATE (me)-[r:knows_location_of #{properties}]- (friend) RETURN r; CYPHER execute_cypher(cypher_query, {name: name, friend_node_id: friend.node_id}) end end 37
  27. class Program include Keymaker::Node property :name index :programs, on: :name

    def knows_location_of(friend, properties={}) neo_service. create_relationship(:knows_location_of, node_id, friend.node_id, properties) end end 38
  28. class Human include Keymaker::Node property :name index :humans, on: :name

    def knows_location_of(friend, properties={}) neo_service. create_relationship(:knows_location_of, node_id, friend.node_id, properties) end end 39
  29. class Human include Keymaker::Node # [...] snipped code def first_degree_network

    cypher_query = <<-CYPHER START me=node:humans(name={name}) MATCH me-[:knows_location_of]-network RETURN network; CYPHER execute_cypher(cypher_query, {name: name}) end end 40
  30. class Human include Keymaker::Node # [...] snipped code def fourth_degree_network

    cypher_query = <<-CYPHER START me=node:humans(name={name}) MATCH me-[:knows_location_of]-first- [:knows_location_of]-second- [:knows_location_of]-third- [:knows_location_of]-fourth RETURN fourth; CYPHER execute_cypher(cypher_query, {name: name}) end end 41
  31. # Create Nodes neo = Human.create({name: 'Neo', type: 'Human'}) #

    Find neo = Human.find(name: 'Neo') morpheus = Human.find(name: 'Morpheus') oracle = Program.find(name: 'Oracle') merovingian = Program.find(name: 'Merovingian') keymaker = Program.find(name: 'Keymaker') 42
  32. # Create Relationships neo.knows_location_of(morpheus, {location: "Nebuchadnezzar", disclosure: "public"}}) morpheus.knows_location_of(oracle, {location:

    "The Matrix", disclosure: "public"}}) oracle.knows_location_of(merovingian, {location: "Club Hel"}) merovingian.knows_location_of(keymaker, {location: "Windows Server", disclosure: "secret"}) 43
  33. 47