Let Me Graph That For You: An Introduction to Neo4j - Ian Robinson

Transcript

1. An  Introduc+on  to  Neo4j   Twi3er:  @ianSrobinson   #neo4j    
2. Neo4j  Graph  Database  
3. Neo4j  is  a  Graph  Database  
4. complexity = f(size, variable structure, connectedness)  
5. Four  Building  Blocks   •  Nodes   – En++es   •  Rela+onships   – Connect  en++es  and  structure  domain   •  Proper+es   – En+ty  a3ributes,  rela+onship  quali+es,  and   metadata   •  Labels   – Group  nodes  by  role  
6. Cypher  Query  Language  
7. Nodes  and  Rela+onships   ()-->()
8. Labels  and  Rela+onship  Types   (:Person)-[:FRIEND]->(:Person)
9. Proper+es   (:Person{name:'Peter'})-[:FRIEND]->(:Person{name:'Lucy'})
10. Iden+fiers   (p1:Person{name:'Peter'})-[r:FRIEND]->(p2:Person{name:'Lucy'})
11. Cypher   MATCH graph_pattern RETURN results http://docs.neo4j.org/chunked/milestone/query-match.html http://docs.neo4j.org/chunked/milestone/query-return.html
12. Example  Query   MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends
13. Find  This  Pa3ern   MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends
14. Lookup  Using  Iden+fier  +  Label     MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends Search  nodes  labeled   ‘Person’,  matching  on   ‘name’  property  
15. Return  Nodes   MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends
16. Exercise  1   Querying  Data  
17. Install  Neo4j   Windows   Run  the  installer  and  use  the  Neo4j  Desktop  to   start  and  stop  your  server   Linux/Mac   Unzip  the  distribu+on,  then  start  the  server   from  a  terminal:   bin/neo4j start    
18. Clean  The  Database  
19. Create  Some  Data   •  Clear  the  command  bar   •  Drag  setup.txt  into  the  command  bar   •  Click  Execute  
20. View  Some  Data   Try  this  query:   MATCH (n) RETURN n Then  this:   MATCH (person:Person)-[r:HAS_SKILL]-(skill) RETURN person.username, skill.name, r.level
21. Run  Another  Query   •  Execute  query-­‐1.txt   – Find  colleagues  with  similar  skills  to  Ian  
22. query-­‐1.txt   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC
23. MATCH   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC
24. WHERE   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC Search  nodes  labeled   ‘Person’,  matching  on   ‘username’  property  
25. Column  Alias   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-return.html#return-column-alias
26. Count   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-aggregation.html#_count
27. Collect   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query- aggregation.html#aggregation-collect
28. ORDER  BY   MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-order.html
29. Your  Turn   Exercise  1-­‐1   Write  a  query  that  finds  ALL  the  people  who   have  similar  skills  to  Ian   – Return  nodes   – No  need  to  use  count,  collect  or  ORDER BY
30. WHERE   •  Supply  filtering  criteria   – Predicates  that  apply  to  any  part  of  the  pa3ern  in   the  MATCH  clause   •  Can  use  AND  and  OR  operators,  and  NOT   func+on   –  WHERE p.age > 30 AND NOT (s.name = 'Java') •  Filter  on  pa3erns   –  WHERE (colleague)-[:WORKS_FOR]->(company)
31. Your  Turn   Exercise  1-­‐2   Find  all  the  people  who  are  level  2  or  above  in   Neo4j   – Return  username  and  skill  level – Remember:  HAS_SKILL  has  a  level    property  
32. Your  Turn   Exercise  1-­‐3   Find  all  the  people  who  have  similar  skills  to  Ian,   but  who  DO  NOT  work  for  the  same  company   – Use  query-­‐1.txt  as  a  star+ng  point   – You’ll  want  to  exclude  (filter)  a  path  
33. Exercise  2   Crea+ng  Data  
34. Create  Some  data   •  Clean  the  database   •  Execute  create-­‐1.txt   •  View  the  results     –  MATCH (n) RETURN n  
35. create-­‐1.txt   CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c), (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)
36. Create  Nodes   CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c), (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)
37. Connect  Nodes   CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c), (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)
38. Create  Some  More  Data   •  Create  more  people   – Same  skills  (Neo4j  and  REST)   – Same  company  (Acme)   •  View  the  results   –  MATCH (n) RETURN n  
39. Crea+ng  Unique  Nodes  and  Rela+onships   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
40. Create  Unique  Nodes   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
41. Create  Unique  Rela+onships   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
42. Set  Rela+onship  Proper+es   MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3
43. MERGE   MERGE  ensures  that  a  pa3ern  exists  in  the  graph.   Either  the  pa3ern  already  exists,  or  it  needs  to   be  created.   http://docs.neo4j.org/chunked/milestone/query-merge.html
44. Your  Turn   •  Clean  the  database   •  Execute  create-­‐2.txt,  create-­‐3.txt  and   create-­‐4.txt   – Acer  each  opera+on,  view  the  results   •  What  happens  if  you  add  or  remove   proper+es  when  specifying  unique  nodes  and   rela+onships?  
45. Exercise  3   Upda+ng  Data  
46. SET   Set  a  Property   MATCH (n:Person{username:'arden'}) SET n.age = 33 Set  a  Label   MATCH (n:Person{username:'arden'}) SET n:Administrator http://docs.neo4j.org/chunked/milestone/query-set.html
47. REMOVE   Remove  a  Property   MATCH (n:Person{username:'arden'}) REMOVE n.passwordHint Remove  a  Label   MATCH (n:Person{username:'arden'}) REMOVE n:Administrator http://docs.neo4j.org/chunked/milestone/query-remove.html
48. Your  Turn   Exercise  3-­‐1   Using  the  datacenter  dataset,  amend  machine   node  proper+es,  from  nam  to  name: 1.  Clean  the  database   2.  Execute  setup.txt   3.  Write  a  query  that  replaces  the  nam  property  on   all  machine  nodes  with  a  name  property  
49. Named  and  Variable-­‐Length  Paths   REPORTS_TO  RelaDonships  Up  To  Depth  5   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() RETURN p All  Paths  TerminaDng  in  Person  Node   MATCH p = (n:Person{employeeId:754}) <-[*]-(:Person) RETURN p http://docs.neo4j.org/chunked/milestone/query-match.html#match-named-path http://docs.neo4j.org/chunked/milestone/query-match.html#match-variable- length-relationships
50. Collec+on  Func+ons   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() RETURN nodes(p), relationships(p) http://docs.neo4j.org/chunked/milestone/query-functions- collection.html#functions-nodes
51. FOREACH   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() FOREACH (person in nodes(p) | CREATE UNIQUE (n)-[:EMPLOYS]->(person)) http://docs.neo4j.org/chunked/milestone/query-foreach.html
52. FOREACH   MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() FOREACH (person in nodes(p) | CREATE UNIQUE (n)-[:EMPLOYS]->(person)) http://docs.neo4j.org/chunked/milestone/query-foreach.html Collec+on   Element   Ac+on  
53. Your  Turn   Exercise  3-­‐2   a.  Find  all  dependency  chains  (paths)  that   extend  from  applica+on  App  1   b.  Add  a  status='fault'  property  to  all   nodes  on  these  paths
54. Exercise  4   Common  Graph  Structures  
55. Intermediate  Nodes   •  Connect  more  than  2  nodes  in  a  single  context   – Hyperedges  (n-­‐ary  rela+onships)   •  Relate  something  to  a  rela+onship  
56. Rich  Context,  Mul+ple  Dimensions  
57. Dimensions  Shared  Between  Contexts  
58. Mul+ple  Par+es  
59. Considera+ons   •  An  intermediate  node  provides  flexibility   – It  allows  more  than  two  nodes  to  be  connected  in   a  single  context     •  But  it  can  be  overkill,  and  will  have  an  impact   on  performance  
60. Linked  Lists   •  En++es  are  linked  in  a  sequence   •  You  need  to  traverse  the  sequence   •  You  may  need  to  iden+fy  the  beginning  or  end   (first/last,  earliest/latest,  etc.)   •  Examples   – Event  stream   – Episodes  of  a  TV  series   – Job  history  
61. Linked  List  
62. Doubly  Linked  List  
63. Interleaved  Linked  Lists  
64. Pointers  to  Head  and  Tail  
65. Your  Turn   Exercise  4-­‐1   •  Clean  the  database   •  Execute  setup.txt   –  Creates  Season  12  node   •  Execute  add-­‐node.txt   –  Creates  story  node •  View  the  results   –  MATCH (n) RETURN n   •  Con+nue  adding  stories,  viewing  the  results   acer  each  addi+on  
66. WITH   •  Chain  queries  together   – Pipes  result  from  one  query  into  next     MATCH (person:Person{username:'ian'}) WITH person MATCH (person)-[:WORKS_FOR]->(company) RETURN company
67. Variable  Length  Paths   MATCH (p:Person{username:'ian'}) -[:FRIEND*1..5]-(friend) WHERE p.username = 'ian' RETURN friend
68. Named  Paths   MATCH p = (person:Person{username:'ian'}) -[:FRIEND]-(friend) RETURN p
69. Your  Turn   Exercise  4-­‐2   Write  a  query  that  returns  all  the  stories  in   Season  12,  in  order   –  Hint:  use  chained  queries   –  Query  1:  find  FIRST  and  LAST  nodes   –  Query  2:  named  variable-­‐length  path,  from  first   to  last  node  
70. Exercise  5   Labels,  Indexes  and  Constraints  
71. What’s  Happening  Here?   MATCH (p:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends
72. Brute-­‐Force  Scan   MATCH (p:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends for each node labeled Person if node.name = 'Peter' yield node
73. Add  An  Index   CREATE INDEX ON :Person(name) http://docs.neo4j.org/chunked/milestone/query-schema-index.html Property   Label  
74. Indexes   •  Eventually  available   •  Populated  in  the  background   •  Using  the  Java  API,  you  can  wait  for  an  index   to  be  populated   http://docs.neo4j.org/chunked/milestone/tutorials-java-embedded-new-index.html
75. Add  Uniqueness  Constraint   CREATE CONSTRAINT ON (person:Person) ASSERT person.name IS UNIQUE http://docs.neo4j.org/chunked/milestone/query-constraints.html Property   Label   Iden+fier   Iden+fier  
76. Your  Turn   Exercise  5-­‐1   •  Clean  the  database   •  Execute  setup.txt   •  Execute  create-­‐constraint-­‐1.txt   –  Uniqueness  constraint  on  :Person(username) •  Execute  create-­‐1.txt   –  CREATE  new  user  with  username  ‘ian’   •  Execute  create-­‐2.txt   –  MERGE  new  user  with  username  ‘ian’  
77. Your  Turn   Exercise  5-­‐2   •  Execute  drop-­‐constraint-­‐1.txt   –  Drop  uniqueness  constraint   •  Execute  create-­‐1.txt   –  CREATE  new  user  with  username  ‘ian’   •  Execute  constraint-­‐1.txt   –  Uniqueness  constraint  on  :Person(username)  
78. Exercise  6   Unmanaged  Extensions  
79. •  Execute  complex  logic  on  server   – “stored  procedures”   •  Use  Java  APIs  or  Cypher   •  Control  HTTP  request/response  format   Unmanaged  Extensions   REST  API   Extensions  
80. JAX-­‐RS  Annotated  Resources   @Path("/similar-skills") public class ColleagueFinderExtension { private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } }
81. Map  HTTP  Request  to  Object  +  Method   @Path("/similar-skills") public class ColleagueFinderExtension { private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } } GET   /similar-­‐skills   /Ian  
82. CypherExecutor  Injected  by  Server   @Path("/similar-skills") public class ColleagueFinderExtension { private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } } Ensures   Execu+onEngine   reused  across   resource  instances  
83. Your  Turn   Exercise  6-­‐1   Implement  a  ColleagueFinderExtension –  Use  MAPPER.writeValueAsString()  to  create   a  JSON  string   –  Use  Response.ok()  to  create  a  response
84. Conclusion  
85. Resources   •  Online  documenta+on   – h3p://docs.neo4j.org/   •  Neo4j  website   – h3p://www.neo4j.org/learn   •  Stackoverflow   – h3p://stackoverflow.com/ques+ons/tagged/neo4j   •  Google  group   – h3ps://groups.google.com/forum/#!forum/neo4j  
86. Events   •  Meetups   – h3p://www.neo4j.org/par+cipate/events/meetups   •  GraphConnect   – h3p://www.graphconnect.com/   – London,  18-­‐19  November  
87. h3p://graphdatabases.com   Ian Robinson, Jim Webber & Emil Eifrem Graph Databases h Com plim ents of Neo Technology Thank  you   @ianSrobinson   ian@neotechnology.com   github.com/iansrobinson