Let Me Graph That For You: An Introduction to Neo4j - Ian Robinson

Slide 1

Slide 1 text

An Introduc+on to Neo4j Twi3er: @ianSrobinson #neo4j

Slide 2

Slide 2 text

Neo4j Graph Database

Slide 3

Slide 3 text

Neo4j is a Graph Database

Slide 4

Slide 4 text

complexity = f(size, variable structure, connectedness)

Slide 5

Slide 5 text

Four Building Blocks •  Nodes – En++es •  Rela+onships – Connect en++es and structure domain •  Proper+es – En+ty a3ributes, rela+onship quali+es, and metadata •  Labels – Group nodes by role

Slide 6

Slide 6 text

Cypher Query Language

Slide 7

Slide 7 text

Nodes and Rela+onships ()-->()

Slide 8

Slide 8 text

Labels and Rela+onship Types (:Person)-[:FRIEND]->(:Person)

Slide 9

Slide 9 text

Proper+es (:Person{name:'Peter'})-[:FRIEND]->(:Person{name:'Lucy'})

Slide 10

Slide 10 text

Iden+ﬁers (p1:Person{name:'Peter'})-[r:FRIEND]->(p2:Person{name:'Lucy'})

Slide 11

Slide 11 text

Cypher MATCH graph_pattern RETURN results http://docs.neo4j.org/chunked/milestone/query-match.html http://docs.neo4j.org/chunked/milestone/query-return.html

Slide 12

Slide 12 text

Example Query MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

Slide 13

Slide 13 text

Find This Pa3ern MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

Slide 14

Slide 14 text

Lookup Using Iden+ﬁer + Label MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends Search nodes labeled ‘Person’, matching on ‘name’ property

Slide 15

Slide 15 text

Return Nodes MATCH (:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

Slide 16

Slide 16 text

Exercise 1 Querying Data

Slide 17

Slide 17 text

Install Neo4j Windows Run the installer and use the Neo4j Desktop to start and stop your server Linux/Mac Unzip the distribu+on, then start the server from a terminal: bin/neo4j start

Slide 18

Slide 18 text

Clean The Database

Slide 19

Slide 19 text

Create Some Data •  Clear the command bar •  Drag setup.txt into the command bar •  Click Execute

Slide 20

Slide 20 text

View Some Data Try this query: MATCH (n) RETURN n Then this: MATCH (person:Person)-[r:HAS_SKILL]-(skill) RETURN person.username, skill.name, r.level

Slide 21

Slide 21 text

Run Another Query •  Execute query-‐1.txt – Find colleagues with similar skills to Ian

Slide 22

Slide 22 text

query-‐1.txt MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC

Slide 23

Slide 23 text

MATCH MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC

Slide 24

Slide 24 text

WHERE MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC Search nodes labeled ‘Person’, matching on ‘username’ property

Slide 25

Slide 25 text

Column Alias MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-return.html#return-column-alias

Slide 26

Slide 26 text

Count MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-aggregation.html#_count

Slide 27

Slide 27 text

Collect MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query- aggregation.html#aggregation-collect

Slide 28

Slide 28 text

ORDER BY MATCH (company)<-[:WORKS_FOR]-(me:Person{username:'ian'}) -[:HAS_SKILL]->(skill), (company)<-[:WORKS_FOR]-(colleague)-[:HAS_SKILL]->(skill) RETURN colleague.username AS username, count(skill) AS score, collect(skill.name) AS skills ORDER BY score DESC http://docs.neo4j.org/chunked/milestone/query-order.html

Slide 29

Slide 29 text

Your Turn Exercise 1-‐1 Write a query that ﬁnds ALL the people who have similar skills to Ian – Return nodes – No need to use count, collect or ORDER BY

Slide 30

Slide 30 text

WHERE •  Supply ﬁltering criteria – Predicates that apply to any part of the pa3ern in the MATCH clause •  Can use AND and OR operators, and NOT func+on –  WHERE p.age > 30 AND NOT (s.name = 'Java') •  Filter on pa3erns –  WHERE (colleague)-[:WORKS_FOR]->(company)

Slide 31

Slide 31 text

Your Turn Exercise 1-‐2 Find all the people who are level 2 or above in Neo4j – Return username and skill level – Remember: HAS_SKILL has a level property

Slide 32

Slide 32 text

Your Turn Exercise 1-‐3 Find all the people who have similar skills to Ian, but who DO NOT work for the same company – Use query-‐1.txt as a star+ng point – You’ll want to exclude (ﬁlter) a path

Slide 33

Slide 33 text

Exercise 2 Crea+ng Data

Slide 34

Slide 34 text

Create Some data •  Clean the database •  Execute create-‐1.txt •  View the results –  MATCH (n) RETURN n

Slide 35

Slide 35 text

create-‐1.txt CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c), (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)

Slide 36

Slide 36 text

Create Nodes CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c), (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)

Slide 37

Slide 37 text

Connect Nodes CREATE (p:Person{username:'ben'}), (c:Company{name:'Acme, Inc'}), (s1:Skill{name:'Neo4j'}), (s2:Skill{name:'REST'}), (p)-[:WORKS_FOR]->(c), (p)-[:HAS_SKILL{level:1}]->(s1), (p)-[:HAS_SKILL{level:3}]->(s2)

Slide 38

Slide 38 text

Create Some More Data •  Create more people – Same skills (Neo4j and REST) – Same company (Acme) •  View the results –  MATCH (n) RETURN n

Slide 39

Slide 39 text

Crea+ng Unique Nodes and Rela+onships MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3

Slide 40

Slide 40 text

Create Unique Nodes MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3

Slide 41

Slide 41 text

Create Unique Rela+onships MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3

Slide 42

Slide 42 text

Set Rela+onship Proper+es MERGE (c:Company{name:'Acme'}) MERGE (p:Person{username:'ian'}) MERGE (s1:Skill{name:'Java'}) MERGE (s2:Skill{name:'C#'}) MERGE (s3:Skill{name:'Neo4j'}) MERGE (c)<-[:WORKS_FOR]-(p) MERGE (p)-[r1:HAS_SKILL]->(s1) MERGE (p)-[r2:HAS_SKILL]->(s2) MERGE (p)-[r3:HAS_SKILL]->(s3) SET r1.level = 2 SET r2.level = 2 SET r3.level = 3 RETURN c, p, s1, s2, s3

Slide 43

Slide 43 text

MERGE MERGE ensures that a pa3ern exists in the graph. Either the pa3ern already exists, or it needs to be created. http://docs.neo4j.org/chunked/milestone/query-merge.html

Slide 44

Slide 44 text

Your Turn •  Clean the database •  Execute create-‐2.txt, create-‐3.txt and create-‐4.txt – Acer each opera+on, view the results •  What happens if you add or remove proper+es when specifying unique nodes and rela+onships?

Slide 45

Slide 45 text

Exercise 3 Upda+ng Data

Slide 46

Slide 46 text

SET Set a Property MATCH (n:Person{username:'arden'}) SET n.age = 33 Set a Label MATCH (n:Person{username:'arden'}) SET n:Administrator http://docs.neo4j.org/chunked/milestone/query-set.html

Slide 47

Slide 47 text

REMOVE Remove a Property MATCH (n:Person{username:'arden'}) REMOVE n.passwordHint Remove a Label MATCH (n:Person{username:'arden'}) REMOVE n:Administrator http://docs.neo4j.org/chunked/milestone/query-remove.html

Slide 48

Slide 48 text

Your Turn Exercise 3-‐1 Using the datacenter dataset, amend machine node proper+es, from nam to name: 1.  Clean the database 2.  Execute setup.txt 3.  Write a query that replaces the nam property on all machine nodes with a name property

Slide 49

Slide 49 text

Named and Variable-‐Length Paths REPORTS_TO RelaDonships Up To Depth 5 MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() RETURN p All Paths TerminaDng in Person Node MATCH p = (n:Person{employeeId:754}) <-[*]-(:Person) RETURN p http://docs.neo4j.org/chunked/milestone/query-match.html#match-named-path http://docs.neo4j.org/chunked/milestone/query-match.html#match-variable- length-relationships

Slide 50

Slide 50 text

Collec+on Func+ons MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() RETURN nodes(p), relationships(p) http://docs.neo4j.org/chunked/milestone/query-functions- collection.html#functions-nodes

Slide 51

Slide 51 text

FOREACH MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() FOREACH (person in nodes(p) | CREATE UNIQUE (n)-[:EMPLOYS]->(person)) http://docs.neo4j.org/chunked/milestone/query-foreach.html

Slide 52

Slide 52 text

FOREACH MATCH p = (n:Person{employeeId:754}) <-[:REPORTS_TO*1..5]-() FOREACH (person in nodes(p) | CREATE UNIQUE (n)-[:EMPLOYS]->(person)) http://docs.neo4j.org/chunked/milestone/query-foreach.html Collec+on Element Ac+on

Slide 53

Slide 53 text

Your Turn Exercise 3-‐2 a.  Find all dependency chains (paths) that extend from applica+on App 1 b.  Add a status='fault' property to all nodes on these paths

Slide 54

Slide 54 text

Exercise 4 Common Graph Structures

Slide 55

Slide 55 text

Intermediate Nodes •  Connect more than 2 nodes in a single context – Hyperedges (n-‐ary rela+onships) •  Relate something to a rela+onship

Slide 56

Slide 56 text

Rich Context, Mul+ple Dimensions

Slide 57

Slide 57 text

Dimensions Shared Between Contexts

Slide 58

Slide 58 text

Mul+ple Par+es

Slide 59

Slide 59 text

Considera+ons •  An intermediate node provides ﬂexibility – It allows more than two nodes to be connected in a single context •  But it can be overkill, and will have an impact on performance

Slide 60

Slide 60 text

Linked Lists •  En++es are linked in a sequence •  You need to traverse the sequence •  You may need to iden+fy the beginning or end (ﬁrst/last, earliest/latest, etc.) •  Examples – Event stream – Episodes of a TV series – Job history

Slide 61

Slide 61 text

Linked List

Slide 62

Slide 62 text

Doubly Linked List

Slide 63

Slide 63 text

Interleaved Linked Lists

Slide 64

Slide 64 text

Pointers to Head and Tail

Slide 65

Slide 65 text

Your Turn Exercise 4-‐1 •  Clean the database •  Execute setup.txt –  Creates Season 12 node •  Execute add-‐node.txt –  Creates story node •  View the results –  MATCH (n) RETURN n •  Con+nue adding stories, viewing the results acer each addi+on

Slide 66

Slide 66 text

WITH •  Chain queries together – Pipes result from one query into next MATCH (person:Person{username:'ian'}) WITH person MATCH (person)-[:WORKS_FOR]->(company) RETURN company

Slide 67

Slide 67 text

Variable Length Paths MATCH (p:Person{username:'ian'}) -[:FRIEND*1..5]-(friend) WHERE p.username = 'ian' RETURN friend

Slide 68

Slide 68 text

Named Paths MATCH p = (person:Person{username:'ian'}) -[:FRIEND]-(friend) RETURN p

Slide 69

Slide 69 text

Your Turn Exercise 4-‐2 Write a query that returns all the stories in Season 12, in order –  Hint: use chained queries –  Query 1: ﬁnd FIRST and LAST nodes –  Query 2: named variable-‐length path, from ﬁrst to last node

Slide 70

Slide 70 text

Exercise 5 Labels, Indexes and Constraints

Slide 71

Slide 71 text

What’s Happening Here? MATCH (p:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends

Slide 72

Slide 72 text

Brute-‐Force Scan MATCH (p:Person{name:'Peter'}) -[:FRIEND]->(friends) RETURN friends for each node labeled Person if node.name = 'Peter' yield node

Slide 73

Slide 73 text

Add An Index CREATE INDEX ON :Person(name) http://docs.neo4j.org/chunked/milestone/query-schema-index.html Property Label

Slide 74

Slide 74 text

Indexes •  Eventually available •  Populated in the background •  Using the Java API, you can wait for an index to be populated http://docs.neo4j.org/chunked/milestone/tutorials-java-embedded-new-index.html

Slide 75

Slide 75 text

Add Uniqueness Constraint CREATE CONSTRAINT ON (person:Person) ASSERT person.name IS UNIQUE http://docs.neo4j.org/chunked/milestone/query-constraints.html Property Label Iden+ﬁer Iden+ﬁer

Slide 76

Slide 76 text

Your Turn Exercise 5-‐1 •  Clean the database •  Execute setup.txt •  Execute create-‐constraint-‐1.txt –  Uniqueness constraint on :Person(username) •  Execute create-‐1.txt –  CREATE new user with username ‘ian’ •  Execute create-‐2.txt –  MERGE new user with username ‘ian’

Slide 77

Slide 77 text

Your Turn Exercise 5-‐2 •  Execute drop-‐constraint-‐1.txt –  Drop uniqueness constraint •  Execute create-‐1.txt –  CREATE new user with username ‘ian’ •  Execute constraint-‐1.txt –  Uniqueness constraint on :Person(username)

Slide 78

Slide 78 text

Exercise 6 Unmanaged Extensions

Slide 79

Slide 79 text

•  Execute complex logic on server – “stored procedures” •  Use Java APIs or Cypher •  Control HTTP request/response format Unmanaged Extensions REST API Extensions

Slide 80

Slide 80 text

JAX-‐RS Annotated Resources @Path("/similar-skills") public class ColleagueFinderExtension { private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } }

Slide 81

Slide 81 text

Map HTTP Request to Object + Method @Path("/similar-skills") public class ColleagueFinderExtension { private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } } GET /similar-‐skills /Ian

Slide 82

Slide 82 text

CypherExecutor Injected by Server @Path("/similar-skills") public class ColleagueFinderExtension { private static final ObjectMapper MAPPER = new ObjectMapper(); private ColleagueFinder colleagueFinder; public ColleagueFinderExtension( @Context CypherExecutor cypherExecutor ) { // Initialize colleague finder } @GET @Produces(MediaType.APPLICATION_JSON) @Path("/{name}") public Response getColleagues( @PathParam("name") String name ) throws IOException { // Find colleagues // Return response } } Ensures Execu+onEngine reused across resource instances

Slide 83

Slide 83 text

Your Turn Exercise 6-‐1 Implement a ColleagueFinderExtension –  Use MAPPER.writeValueAsString() to create a JSON string –  Use Response.ok() to create a response

Slide 84

Slide 84 text

Conclusion

Slide 85

Slide 85 text

Resources •  Online documenta+on – h3p://docs.neo4j.org/ •  Neo4j website – h3p://www.neo4j.org/learn •  Stackoverﬂow – h3p://stackoverﬂow.com/ques+ons/tagged/neo4j •  Google group – h3ps://groups.google.com/forum/#!forum/neo4j

Slide 86

Slide 86 text

Events •  Meetups – h3p://www.neo4j.org/par+cipate/events/meetups •  GraphConnect – h3p://www.graphconnect.com/ – London, 18-‐19 November

Slide 87

Slide 87 text

h3p://graphdatabases.com Ian Robinson, Jim Webber & Emil Eifrem Graph Databases h Com plim ents of Neo Technology Thank you @ianSrobinson ian@neotechnology.com github.com/iansrobinson