Graph Databases: The internal works of Neo4j

Slide 1

Slide 1 text

@michellesanver Neo4J: Graph Database NoSQL Day 2014

Slide 2

Slide 2 text

About: Michelle Sanver @michellesanver

Slide 3

Slide 3 text

About: Michelle Sanver @michellesanver President (Michelle)-[:LOVES]->(Neo4j)

Slide 4

Slide 4 text

@michellesanver Code Addict (Michelle)-[:LOVES]->(Neo4j) About: Michelle Sanver

Slide 5

Slide 5 text

@michellesanver Zürich, Switzerland (Michelle)-[:LOVES]->(Neo4j) About: Michelle Sanver

Slide 6

Slide 6 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Michelle is_from is_from lives_in lived_in About: Michelle Sanver

Slide 7

Slide 7 text

Setting expectations. @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 8

Slide 8 text

Graphs \o/ @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 9

Slide 9 text

Graphs \o/ @michellesanver (Michelle)-[:LOVES]->(Neo4j) Graph theory has been studied since Leonard Euler’s Bridges 1736

Slide 10

Slide 10 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Graphs \o/

Slide 11

Slide 11 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Graphs \o/

Slide 12

Slide 12 text

Graphs \o/ @michellesanver Graphs are really just connected data… They are everywhere (Michelle)-[:LOVES]->(Neo4j)

Slide 13

Slide 13 text

Graphs \o/ @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 14

Slide 14 text

@michellesanver It’s modern. (Michelle)-[:LOVES]->(Neo4j) Graphs \o/

Slide 15

Slide 15 text

@michellesanver Facebook open graph (Michelle)-[:LOVES]->(Neo4j) Graphs \o/

Slide 16

Slide 16 text

@michellesanver A graph is an easy way to visualise connected data. Michelle Graphs likes (Michelle)-[:LOVES]->(Neo4j) Graphs \o/

Slide 17

Slide 17 text

Graphs vs. Relational databases @michellesanver Relational question: Average age of everyone in this list? Graph question: Who knows me from third parties? (Michelle)-[:LOVES]->(Neo4j)

Slide 18

Slide 18 text

@michellesanver Relational databases have tables Recipes (Michelle)-[:LOVES]->(Neo4j) Graphs vs. Relational databases

Slide 19

Slide 19 text

@michellesanver Tables have relationships Recipes Groups (Michelle)-[:LOVES]->(Neo4j) Graphs vs. Relational databases

Slide 20

Slide 20 text

@michellesanver Which causes a join table… Recipes Groups RecipeToGroup (Michelle)-[:LOVES]->(Neo4j) Graphs vs. Relational databases

Slide 21

Slide 21 text

@michellesanver You query via the table Recipes Groups RecipeToGroup (Michelle)-[:LOVES]->(Neo4j) Graphs vs. Relational databases

Slide 22

Slide 22 text

@michellesanver Imagine having *actual* relations Pasta group Spaghetti Italian group (Michelle)-[:LOVES]->(Neo4j) Graphs vs. Relational databases

Slide 23

Slide 23 text

@michellesanver Recipes Groups RecipeToGroup Pasta group Spaghetti Italian group (Michelle)-[:LOVES]->(Neo4j) Graphs vs. Relational databases

Slide 24

Slide 24 text

@michellesanver A big query. The EAV model. (Michelle)-[:LOVES]->(Neo4j) Graphs vs. Relational databases

Slide 25

Slide 25 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) SET @entityid = '3'; SELECT ea.attribute_id, ea.attribute_code, eav.value AS 'value', 'varchar' AS 'type' FROM catalog_category_entity e JOIN catalog_category_entity_varchar eav ON e.entity_id = eav.entity_id JOIN eav_attribute ea ON eav.attribute_id = ea.attribute_id WHERE e.entity_id = @entityid UNION SELECT ea.attribute_id, ea.attribute_code, eav.value AS 'value', 'int' AS 'type' FROM catalog_category_entity e JOIN catalog_category_entity_int eav ON e.entity_id = eav.entity_id JOIN eav_attribute ea ON eav.attribute_id = ea.attribute_id WHERE e.entity_id = @entityid UNION Graphs vs. Relational databases

Slide 26

Slide 26 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) SELECT ea.attribute_id, ea.attribute_code, eav.value AS 'value', 'decimal' AS 'type' FROM catalog_category_entity e JOIN catalog_category_entity_d ecimal eav ON e.entity_id = eav.entity_id JOIN eav_attribute ea ON eav.attribute_id = ea.attribute_id WHERE e.entity_id = @entityid UNION SELECT ea.attribute_id, ea.attribute_code, eav.value AS 'value', 'datetime' AS 'type' FROM catalog_category_entity e JOIN catalog_category_entity_datetime eav ON e.entity_id = eav.entity_id JOIN eav_attribute ea ON eav.attribute_id = ea.attribute_id WHERE e.entity_id = @entityid UNION SELECT ea.attribute_id, ea.attribute_code, eav.value AS 'value', 'text' AS 'type' FROM catalog_category_entity e JOIN catalog_category_entity_text eav ON e.entity_id = eav.entity_id JOIN eav_attribute ea ON eav.attribute_id = ea.attribute_id WHERE e.entity_id = @entityid Graphs vs. Relational databases

Slide 27

Slide 27 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Having a ﬂexible schema database costs a lot. Graphs vs. Relational databases

Slide 28

Slide 28 text

Neo4j @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 29

Slide 29 text

neo4j.org @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 30

Slide 30 text

neotechnology.com @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 31

Slide 31 text

Java Based @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 32

Slide 32 text

Open Source! github.com/neo4j @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 33

Slide 33 text

Meetups everywhere neo4j.meetup.com @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 34

Slide 34 text

Graphs and Neo4j @michellesanver Graphs have… (Michelle)-[:LOVES]->(Neo4j)

Slide 35

Slide 35 text

@michellesanver ( Nodes ) (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 36

Slide 36 text

@michellesanver (Node) { Properties } As many as you want Type: Person Nick: geekie Type: Person Nick: WyriHaximus (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 37

Slide 37 text

@michellesanver (Node) [ Relationships ] As many as you want Type: Person Nick: geekie Type: Person Nick: WyriHaximus (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 38

Slide 38 text

@michellesanver [Relationship { properties } ] As many as you want Nick: geekie Nick: WyriHaximus knows (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 39

Slide 39 text

@michellesanver Labels Type: Person Nick: geekie (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 40

Slide 40 text

@michellesanver Indexes for easy lookup Type: Person Nick: geekie (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 41

Slide 41 text

@michellesanver Common named graphs (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 42

Slide 42 text

@michellesanver Common named graphs Local McLaughlin graph (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 43

Slide 43 text

@michellesanver Diamond Butterﬂy Bull Franklin Tutte (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j Common named graphs

Slide 44

Slide 44 text

@michellesanver You can make art out of your DB. (Don’t) (Michelle)-[:LOVES]->(Neo4j) Graphs and Neo4j

Slide 45

Slide 45 text

OmNomHub https://github.com/Omnomhub @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 46

Slide 46 text

OmNomHub @michellesanver Like GitHub but for recipes! (Michelle)-[:LOVES]->(Neo4j)

Slide 47

Slide 47 text

OmNomHub @michellesanver Fork a recipe (Michelle)-[:LOVES]->(Neo4j)

Slide 48

Slide 48 text

@michellesanver See all forks (Michelle)-[:LOVES]->(Neo4j) OmNomHub

Slide 49

Slide 49 text

@michellesanver Join groups (Michelle)-[:LOVES]->(Neo4j) OmNomHub

Slide 50

Slide 50 text

@michellesanver Search similar recipes (Michelle)-[:LOVES]->(Neo4j) OmNomHub

Slide 51

Slide 51 text

@michellesanver Have fun! ;-) (Michelle)-[:LOVES]->(Neo4j) OmNomHub

Slide 52

Slide 52 text

Neo4j architecture @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 53

Slide 53 text

Neo4j architecture @michellesanver (Michelle)-[:LOVES]->(Neo4j) Disks File System Cache Record Files Object Cache Cypher Core API Traversal API Transaction Management Transaction Log

Slide 54

Slide 54 text

Hang in there (Michelle)-[:LOVES]->(Kittens) @michellesanver

Slide 55

Slide 55 text

Neo4j architecture: Traversals @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 56

Slide 56 text

@michellesanver Pathexpanders: Deﬁne what to traverse (Michelle)-[:LOVES]->(Neo4j) Traversals

Slide 57

Slide 57 text

@michellesanver Order: Depth-ﬁrst or breadth-ﬁrst. (Michelle)-[:LOVES]->(Neo4j) Traversals

Slide 58

Slide 58 text

@michellesanver Uniqueness : Visit once. (Michelle)-[:LOVES]->(Neo4j) Traversals

Slide 59

Slide 59 text

@michellesanver Evaluator : Should I stay or should I go? (Michelle)-[:LOVES]->(Neo4j) Traversals

Slide 60

Slide 60 text

@michellesanver Starting nodes: And so it begins. (Michelle)-[:LOVES]->(Neo4j) Traversals

Slide 61

Slide 61 text

Neo4j architecture: Cypher and Browser @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 62

Slide 62 text

Cypher @michellesanver Cypher: SQL for graph databases. (Michelle)-[:LOVES]->(Neo4j)

Slide 63

Slide 63 text

@michellesanver ) (Michelle)-[:LOVES]->(Neo4j) Parentheses means nodes (Or hugs) ( Cypher

Slide 64

Slide 64 text

@michellesanver } (Michelle)-[:LOVES]->(Neo4j) Curly braces means properties { Cypher

Slide 65

Slide 65 text

@michellesanver ] (Michelle)-[:LOVES]->(Neo4j) Square brackets means relationships [ Cypher

Slide 66

Slide 66 text

@michellesanver Makes it easy to visualise and query the data. (Michelle)-[:LOVES]->(Neo4j) Browser

Slide 67

Slide 67 text

@michellesanver Let’s learn cypher in the browser! (Michelle)-[:LOVES]->(Neo4j) Browser

Slide 68

Slide 68 text

DEMO: Cypher and Browser @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 69

Slide 69 text

Neo4j architecture Neostore File Storage @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 70

Slide 70 text

Neostore @michellesanver Several diﬀerent store ﬁles (Michelle)-[:LOVES]->(Neo4j)

Slide 71

Slide 71 text

@michellesanver Each store has speciﬁc data (Michelle)-[:LOVES]->(Neo4j) Neostore

Slide 72

Slide 72 text

@michellesanver Nodes neostore.nodestore.db 9 bytes (Michelle)-[:LOVES]->(Neo4j) Neostore

Slide 73

Slide 73 text

@michellesanver Nodes in-use key: 1 byte (Michelle)-[:LOVES]->(Neo4j) Neostore

Slide 74

Slide 74 text

Neostore Fixed size == FAST @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 75

Slide 75 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Relationships neostore.relationshipstore.db 33 bytes Neostore

Slide 76

Slide 76 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Properties neostore.propertystore.db neostore.propertystore.db.index neostore.propertystore.db.strings neostore.propertystore.db.arrays Neostore

Slide 77

Slide 77 text

Hardware matters! @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 78

Slide 78 text

Cache @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 79

Slide 79 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Two-tiered Cache Cache

Slide 80

Slide 80 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Filesystem cache Cache

Slide 81

Slide 81 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Filesystem cache Can be ﬁne-tuned Cache

Slide 82

Slide 82 text

@michellesanver (Michelle)-[:LOVES]->(Neo4j) Object Cache Properties and references to their relationships Cache

Slide 83

Slide 83 text

OmNomHub: Beyond Pizza @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 84

Slide 84 text

OmNomHub: Beyond Pizza @michellesanver Connecting users “You both like” (Michelle)-[:LOVES]->(Neo4j)

Slide 85

Slide 85 text

@michellesanver Connecting recipes “These have similar ingredients and user base” (Michelle)-[:LOVES]->(Neo4j) OmNomHub: Beyond Pizza

Slide 86

Slide 86 text

@michellesanver Being smart “You might like” (Michelle)-[:LOVES]->(Neo4j) OmNomHub: Beyond Pizza

Slide 87

Slide 87 text

@michellesanver Being smart Smart recipe collections! (Michelle)-[:LOVES]->(Neo4j) OmNomHub: Beyond Pizza

Slide 88

Slide 88 text

@michellesanver Being creepy “Don’t like meat huh?” (Michelle)-[:LOVES]->(Neo4j) OmNomHub: Beyond Pizza

Slide 89

Slide 89 text

Wrapup @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 90

Slide 90 text

Wrapup @michellesanver Graphs are everywhere (Michelle)-[:LOVES]->(Neo4j)

Slide 91

Slide 91 text

@michellesanver They make it easy to connect data (Michelle)-[:LOVES]->(Neo4j) Wrapup

Slide 92

Slide 92 text

@michellesanver Easier to be creepy <3 (Michelle)-[:LOVES]->(Neo4j) Wrapup

Slide 93

Slide 93 text

@michellesanver OmNomHub will be EPIC :D (Michelle)-[:LOVES]->(Neo4j) Wrapup

Slide 94

Slide 94 text

@michellesanver Open Source! https://github.com/Omnomhub (Michelle)-[:LOVES]->(Neo4j) Wrapup

Slide 95

Slide 95 text

Resources @michellesanver docs.neo4j.org Graph Databases - Ian Robinson,   neo technology gives you the e-book version for free. neo4j.org/learn (Michelle)-[:LOVES]->(Neo4j)

Slide 96

Slide 96 text

Pleaaaaase…. @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 97

Slide 97 text

Please @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 98

Slide 98 text

Please @michellesanver (Michelle)-[:LOVES]->(Neo4j)

Slide 99

Slide 99 text

Give me feedback https://joind.in/12745 @michellesanver (Michelle)-[:LOVES]->(Neo4j)