Let Me Graph That For You: An Introduction to Neo4j - Ian Robinson

Let Me Graph That For You: An Introduction to Neo4j - Ian Robinson

A846fc46522b396026adcb62e162b7dc?s=128

Joy of Coding

March 07, 2014
Tweet

Transcript

  1. An  Introduc+on  to  Neo4j   Twi3er:  @ianSrobinson   #neo4j  

     
  2. Neo4j  Graph  Database  

  3. Neo4j  is  a  Graph  Database  

  4. complexity = f(size, variable structure, connectedness)  

  5. Four  Building  Blocks   •  Nodes   – En++es   • 

    Rela+onships   – Connect  en++es  and  structure  domain   •  Proper+es   – En+ty  a3ributes,  rela+onship  quali+es,  and   metadata   •  Labels   – Group  nodes  by  role  
  6. Cypher  Query  Language  

  7. Nodes  and  Rela+onships   ()-->()

  8. Labels  and  Rela+onship  Types   (:Person)-[:FRIEND]->(:Person)

  9. Proper+es   (:Person{name:'Peter'})-[:FRIEND]->(:Person{name:'Lucy'})

  10. Iden+fiers   (p1:Person{name:'Peter'})-[r:FRIEND]->(p2:Person{name:'Lucy'})

  11. Cypher   MATCH graph_pattern RETURN results http://docs.neo4j.org/chunked/milestone/query-match.html http://docs.neo4j.org/chunked/milestone/query-return.html

  12. Example  Query   MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

  13. Find  This  Pa3ern   MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

  14. Lookup  Using  Iden+fier  +  Label     MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends)

    RETURN friends Search  nodes  labeled   ‘Person’,  matching  on   ‘name’  property  
  15. Return  Nodes   MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

  16. Exercise  1   Querying  Data  

  17. Install  Neo4j   Windows   Run  the  installer  and  use

     the  Neo4j  Desktop  to   start  and  stop  your  server   Linux/Mac   Unzip  the  distribu+on,  then  start  the  server   from  a  terminal:   bin/neo4j start    
  18. Clean  The  Database  

  19. Create  Some  Data   •  Clear  the  command  bar  

    •  Drag  setup.txt  into  the  command  bar   •  Click  Execute  
  20. View  Some  Data   Try  this  query:   MATCH (n)

    RETURN n Then  this:   MATCH (person:Person)-[r:HAS_SKILL]-(skill) RETURN person.username, skill.name, r.level
  21. Run  Another  Query   •  Execute  query-­‐1.txt   – Find  colleagues

     with  similar  skills  to  Ian  
  22. query-­‐1.txt   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username,

    count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC
  23. MATCH   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username,

    count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC
  24. WHERE   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username,

    count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC Search  nodes  labeled   ‘Person’,  matching  on   ‘username’  property  
  25. Column  Alias   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS

    username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-return.html#return-column-alias
  26. Count   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username,

    count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-aggregation.html#_count
  27. Collect   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username,

    count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query- aggregation.html#aggregation-collect
  28. ORDER  BY   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS

    username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-order.html
  29. Your  Turn   Exercise  1-­‐1   Write  a  query  that

     finds  ALL  the  people  who   have  similar  skills  to  Ian   – Return  nodes   – No  need  to  use  count,  collect  or  ORDER BY
  30. WHERE   •  Supply  filtering  criteria   – Predicates  that  apply

     to  any  part  of  the  pa3ern  in   the  MATCH  clause   •  Can  use  AND  and  OR  operators,  and  NOT   func+on   –  WHERE p.age > 30 AND NOT (s.name = 'Java') •  Filter  on  pa3erns   –  WHERE (colleague)-[:WORKS_FOR]->(company)
  31. Your  Turn   Exercise  1-­‐2   Find  all  the  people

     who  are  level  2  or  above  in   Neo4j   – Return  username  and  skill  level – Remember:  HAS_SKILL  has  a  level    property  
  32. Your  Turn   Exercise  1-­‐3   Find  all  the  people

     who  have  similar  skills  to  Ian,   but  who  DO  NOT  work  for  the  same  company   – Use  query-­‐1.txt  as  a  star+ng  point   – You’ll  want  to  exclude  (filter)  a  path  
  33. Exercise  2   Crea+ng  Data  

  34. Create  Some  data   •  Clean  the  database   • 

    Execute  create-­‐1.txt   •  View  the  results     –  MATCH (n) RETURN n  
  35. create-­‐1.txt   CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c), (p)-[:HAS_SKILL{level:1}]->(s1),

    (p)-[:HAS_SKILL{level:3}]->(s2)
  36. Create  Nodes   CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c),

    (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)
  37. Connect  Nodes   CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c),

    (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)
  38. Create  Some  More  Data   •  Create  more  people  

    – Same  skills  (Neo4j  and  REST)   – Same  company  (Acme)   •  View  the  results   –  MATCH (n) RETURN n  
  39. Crea+ng  Unique  Nodes  and  Rela+onships   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'})

    MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
  40. Create  Unique  Nodes   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'})

    MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
  41. Create  Unique  Rela+onships   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'})

    MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
  42. Set  Rela+onship  Proper+es   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'})

    MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
  43. MERGE   MERGE  ensures  that  a  pa3ern  exists  in  the

     graph.   Either  the  pa3ern  already  exists,  or  it  needs  to   be  created.   http://docs.neo4j.org/chunked/milestone/query-merge.html
  44. Your  Turn   •  Clean  the  database   •  Execute

     create-­‐2.txt,  create-­‐3.txt  and   create-­‐4.txt   – Acer  each  opera+on,  view  the  results   •  What  happens  if  you  add  or  remove   proper+es  when  specifying  unique  nodes  and   rela+onships?  
  45. Exercise  3   Upda+ng  Data  

  46. SET   Set  a  Property   MATCH (n:Person{username:'arden'}) SET n.age

    = 33 Set  a  Label   MATCH (n:Person{username:'arden'}) SET n:Administrator http://docs.neo4j.org/chunked/milestone/query-set.html
  47. REMOVE   Remove  a  Property   MATCH (n:Person{username:'arden'}) REMOVE n.passwordHint

    Remove  a  Label   MATCH (n:Person{username:'arden'}) REMOVE n:Administrator http://docs.neo4j.org/chunked/milestone/query-remove.html
  48. Your  Turn   Exercise  3-­‐1   Using  the  datacenter  dataset,

     amend  machine   node  proper+es,  from  nam  to  name: 1.  Clean  the  database   2.  Execute  setup.txt   3.  Write  a  query  that  replaces  the  nam  property  on   all  machine  nodes  with  a  name  property  
  49. Named  and  Variable-­‐Length  Paths   REPORTS_TO  RelaDonships  Up  To  Depth

     5   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() RETURN p All  Paths  TerminaDng  in  Person  Node   MATCH p = (n:Person{employeeId:754}) <-[*]-(:Person) RETURN p http://docs.neo4j.org/chunked/milestone/query-match.html#match-named-path http://docs.neo4j.org/chunked/milestone/query-match.html#match-variable- length-relationships
  50. Collec+on  Func+ons   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() RETURN nodes(p),

    relationships(p) http://docs.neo4j.org/chunked/milestone/query-functions- collection.html#functions-nodes
  51. FOREACH   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() FOREACH (person in

    nodes(p) | CREATE UNIQUE (n)-[:EMPLOYS]->(person)) http://docs.neo4j.org/chunked/milestone/query-foreach.html
  52. FOREACH   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() FOREACH (person in

    nodes(p) | CREATE UNIQUE (n)-[:EMPLOYS]->(person)) http://docs.neo4j.org/chunked/milestone/query-foreach.html Collec+on   Element   Ac+on  
  53. Your  Turn   Exercise  3-­‐2   a.  Find  all  dependency

     chains  (paths)  that   extend  from  applica+on  App  1   b.  Add  a  status='fault'  property  to  all   nodes  on  these  paths
  54. Exercise  4   Common  Graph  Structures  

  55. Intermediate  Nodes   •  Connect  more  than  2  nodes  in

     a  single  context   – Hyperedges  (n-­‐ary  rela+onships)   •  Relate  something  to  a  rela+onship  
  56. Rich  Context,  Mul+ple  Dimensions  

  57. Dimensions  Shared  Between  Contexts  

  58. Mul+ple  Par+es  

  59. Considera+ons   •  An  intermediate  node  provides  flexibility   – It

     allows  more  than  two  nodes  to  be  connected  in   a  single  context     •  But  it  can  be  overkill,  and  will  have  an  impact   on  performance  
  60. Linked  Lists   •  En++es  are  linked  in  a  sequence

      •  You  need  to  traverse  the  sequence   •  You  may  need  to  iden+fy  the  beginning  or  end   (first/last,  earliest/latest,  etc.)   •  Examples   – Event  stream   – Episodes  of  a  TV  series   – Job  history  
  61. Linked  List  

  62. Doubly  Linked  List  

  63. Interleaved  Linked  Lists  

  64. Pointers  to  Head  and  Tail  

  65. Your  Turn   Exercise  4-­‐1   •  Clean  the  database

      •  Execute  setup.txt   –  Creates  Season  12  node   •  Execute  add-­‐node.txt   –  Creates  story  node •  View  the  results   –  MATCH (n) RETURN n   •  Con+nue  adding  stories,  viewing  the  results   acer  each  addi+on  
  66. WITH   •  Chain  queries  together   – Pipes  result  from

     one  query  into  next     MATCH (person:Person{username:'ian'}) WITH person MATCH (person)-[:WORKS_FOR]->(company) RETURN company
  67. Variable  Length  Paths   MATCH (p:Person{username:'ian'}) -[:FRIEND*1..5]-(friend) WHERE p.username =

    'ian' RETURN friend
  68. Named  Paths   MATCH p = (person:Person{username:'ian'}) -[:FRIEND]-(friend) RETURN p

  69. Your  Turn   Exercise  4-­‐2   Write  a  query  that

     returns  all  the  stories  in   Season  12,  in  order   –  Hint:  use  chained  queries   –  Query  1:  find  FIRST  and  LAST  nodes   –  Query  2:  named  variable-­‐length  path,  from  first   to  last  node  
  70. Exercise  5   Labels,  Indexes  and  Constraints  

  71. What’s  Happening  Here?   MATCH (p:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

  72. Brute-­‐Force  Scan   MATCH (p:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends for each

    node labeled Person if node.name = 'Peter' yield node
  73. Add  An  Index   CREATE INDEX ON :Person(name) http://docs.neo4j.org/chunked/milestone/query-schema-index.html Property

      Label  
  74. Indexes   •  Eventually  available   •  Populated  in  the

     background   •  Using  the  Java  API,  you  can  wait  for  an  index   to  be  populated   http://docs.neo4j.org/chunked/milestone/tutorials-java-embedded-new-index.html
  75. Add  Uniqueness  Constraint   CREATE CONSTRAINT ON (person:Person) ASSERT person.name

    IS UNIQUE http://docs.neo4j.org/chunked/milestone/query-constraints.html Property   Label   Iden+fier   Iden+fier  
  76. Your  Turn   Exercise  5-­‐1   •  Clean  the  database

      •  Execute  setup.txt   •  Execute  create-­‐constraint-­‐1.txt   –  Uniqueness  constraint  on  :Person(username) •  Execute  create-­‐1.txt   –  CREATE  new  user  with  username  ‘ian’   •  Execute  create-­‐2.txt   –  MERGE  new  user  with  username  ‘ian’  
  77. Your  Turn   Exercise  5-­‐2   •  Execute  drop-­‐constraint-­‐1.txt  

    –  Drop  uniqueness  constraint   •  Execute  create-­‐1.txt   –  CREATE  new  user  with  username  ‘ian’   •  Execute  constraint-­‐1.txt   –  Uniqueness  constraint  on  :Person(username)  
  78. Exercise  6   Unmanaged  Extensions  

  79. •  Execute  complex  logic  on  server   – “stored  procedures”  

    •  Use  Java  APIs  or  Cypher   •  Control  HTTP  request/response  format   Unmanaged  Extensions   REST  API   Extensions  
  80. JAX-­‐RS  Annotated  Resources   @Path("/similar-skills") public class ColleagueFinderExtension { private

    static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } }
  81. Map  HTTP  Request  to  Object  +  Method   @Path("/similar-skills") public

    class ColleagueFinderExtension { private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } } GET   /similar-­‐skills   /Ian  
  82. CypherExecutor  Injected  by  Server   @Path("/similar-skills") public class ColleagueFinderExtension {

    private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } } Ensures   Execu+onEngine   reused  across   resource  instances  
  83. Your  Turn   Exercise  6-­‐1   Implement  a  ColleagueFinderExtension – 

    Use  MAPPER.writeValueAsString()  to  create   a  JSON  string   –  Use  Response.ok()  to  create  a  response
  84. Conclusion  

  85. Resources   •  Online  documenta+on   – h3p://docs.neo4j.org/   •  Neo4j

     website   – h3p://www.neo4j.org/learn   •  Stackoverflow   – h3p://stackoverflow.com/ques+ons/tagged/neo4j   •  Google  group   – h3ps://groups.google.com/forum/#!forum/neo4j  
  86. Events   •  Meetups   – h3p://www.neo4j.org/par+cipate/events/meetups   •  GraphConnect  

    – h3p://www.graphconnect.com/   – London,  18-­‐19  November  
  87. h3p://graphdatabases.com   Ian Robinson, Jim Webber & Emil Eifrem Graph

    Databases h Com plim ents of Neo Technology Thank  you   @ianSrobinson   ian@neotechnology.com   github.com/iansrobinson