Tweet
Tweet

Slide 1

Slide 1 text

IT'S 2017, AND I STILL WANT TO SELL YOU A GRAPH DATABASE

Slide 2

Slide 2 text

A CONFESSION

Slide 3

Slide 3 text

I DON'T REALLY WANT TO SELL YOU A GRAPH DATABASE

Slide 4

Slide 4 text

I WANT TO SELL YOU THE IDEA OF A GRAPH DATABASE

Slide 5

Slide 5 text

WHY?

Slide 6

Slide 6 text

A LANGUAGE THAT DOESN'T AFFECT THE WAY YOU THINK ABOUT PROGRAMMING IS NOT WORTH KNOWING. Alan Perlis, emphasis mine.

Slide 7

Slide 7 text

A DATABASE THAT DOESN'T AFFECT THE WAY YOU THINK ABOUT DATA MODELLING IS NOT WORTH KNOWING. Alan Perlis, paraphrased

Slide 8

Slide 8 text

I WANT YOU TO MAKE AN INFORMED DECISION

Slide 9

Slide 9 text

A QUICK STORY BUT BEFORE THAT,

Slide 10

Slide 10 text

WE SPENT ONE YEAR BUILDING A PRODUCT USING NEO4J

Slide 11

Slide 11 text

WE SPENT ONE YEAR BUILDING A PRODUCT USERS PAY FOR

Slide 12

Slide 12 text

WE SPENT ONE YEAR BUILDING A PRODUCT USERS LOVE

Slide 13

Slide 13 text

AND NOW WE'RE REWRITING IT IN POSTGRES

Slide 14

Slide 14 text

WE CONCLUDED THAT BOTH DECISIONS WERE CORRECT

Slide 15

Slide 15 text

USING NEO4J THEN, WAS CORRECT

Slide 16

Slide 16 text

USING POSTGRES NOW, IS CORRECT

Slide 17

Slide 17 text

THE FUN IS IN THE DETAILS NATURALLY,

Slide 18

Slide 18 text

1. SEMANTICS 2. TOOLING 3. OPS

Slide 19

Slide 19 text

MODELLING AND QUERYING DATA AS A GRAPH SEMANTICS

Slide 20

Slide 20 text

DOES IT MAKE YOUR BUSINESS LOGIC ANY SIMPLER OR BETTER?

Slide 21

Slide 21 text

TURNS OUT, IT DOES

Slide 22

Slide 22 text

EXAMPLE: SYMMETRICAL RELATIONSHIPS GRAPH SEMANTICS

Slide 23

Slide 23 text

REPRESENTING MARRIAGES

Slide 24

Slide 24 text

SAY, A PERSON IS MARRIED TO ANOTHER PERSON

Slide 25

Slide 25 text

BY DEFINITION, THAT PERSON IS MARRIED TO THIS PERSON

Slide 26

Slide 26 text

THE TRADITIONAL WAY TO SOLVE THIS WOULD BE A JOIN TABLE

Slide 27

Slide 27 text

GRAPH SEMANTICS SYMMETRICAL RELATIONSHIPS: MARRIAGES class Person < ActiveRecord::Base
 has_many :marriages
 has_many :spouses, through: :marriages, source: :spouse
 end
 class Marriage < ActiveRecord::Base
 belongs_to :person
 belongs_to :spouse, foreign_key: :spouse_id, class_name: "Person"
 end

Slide 28

Slide 28 text

GRAPH SEMANTICS SYMMETRICAL RELATIONSHIPS: MARRIAGES SELECT * FROM marriages m
 WHERE m.person_id = 42 OR m.spouse_id = 42

Slide 29

Slide 29 text

GRAPH SEMANTICS SYMMETRICAL RELATIONSHIPS: MARRIAGES if m.person_id == person.id
 Person.find(m.person_id)
 else
 Person.find(m.spouse_id)
 end

Slide 30

Slide 30 text

AWKWARD!

Slide 31

Slide 31 text

BUT HEY, THIS IS POSTGRES. WE CAN DO BETTER!

Slide 32

Slide 32 text

GRAPH SEMANTICS SYMMETRICAL RELATIONSHIPS: MARRIAGES class Marriage < ActiveRecord::Base
 belongs_to :person
 belongs_to :spouse, foreign_key: :spouse_id, class_name: "Person"
 
 after_create do
 inverse.first_or_create
 end
 
 after_destroy do
 inverse.first.try(:destroy)
 end
 
 def inverse
 Marriage.where(person: spouse, spouse: person)
 end
 end

Slide 33

Slide 33 text

THIS IS GOOD, AND IT WORKS

Slide 34

Slide 34 text

BUT…

Slide 35

Slide 35 text

EXTRANEOUS COMPLEXITY IN MAINTAINING EVEN ROWS

Slide 36

Slide 36 text

EXTRANEOUS COMPLEXITY IN ADDING FEATURES

Slide 37

Slide 37 text

EXTRANEOUS COMPLEXITY LIKE MULTIPLE SPOUSES

Slide 38

Slide 38 text

EXTRANEOUS COMPLEXITY LIKE PAST SPOUSES

Slide 39

Slide 39 text

COMPARE THAT TO NEO4J:

Slide 40

Slide 40 text

GRAPH SEMANTICS SYMMETRICAL RELATIONSHIPS: MARRIAGES class Person
 include Neo4j::ActiveNode
 
 # (Person)-[:MARRIES]->(Person)
 has_many :both, :spouses,
 type: "MARRIES", model_class: "Person"
 
 # (Person)-[:IS_MARRIED_TO]->(Person)
 has_one :both, :current_spouse,
 type: "IS_MARRIED_TO", model_class: "Person"
 end

Slide 41

Slide 41 text

SIMPLE, DECLARATIVE, DATABASE RELIANT

Slide 42

Slide 42 text

GRAPH SEMANTICS SYMMETRICAL RELATIONSHIPS: MARRIAGES class Person
 include Neo4j::ActiveNode
 
 # (Person)-[:MARRIES]->(Person)
 has_many :both, :spouses,
 type: "MARRIES", model_class: "Person"
 
 # (Person)-[:IS_MARRIED_TO]->(Person)
 has_one :both, :current_spouse,
 type: "IS_MARRIED_TO", model_class: "Person"
 end

Slide 43

Slide 43 text

AD-HOC POINTERS GRAPH SEMANTICS

Slide 44

Slide 44 text

GRAPH SEMANTICS AD-HOC POINTERS # (Author)-[:HAS_WRITTEN]->(Book)
 has_many :out, :books,
 type: "HAS_WRITTEN", model_class: "Book"
 
 
 # (Author)-[:IS_WRITING]->(Book)
 has_one :out, :current_book,
 type: "IS_WRITING", model_class: "Book"

Slide 45

Slide 45 text

GRAPH SEMANTICS AD-HOC POINTERS # (Student)-[:HAS_ATTEMPTED]->(Assignment)
 has_many :out, :assignments,
 type: "HAS_ATTEMPTED", model_class: "Assignment"
 
 
 # (Student)-[:HAS_BEST]->(Assignment)
 has_one :out, :best_submission,
 type: "HAS_BEST", model_class: "Assignment"

Slide 46

Slide 46 text

GRAPH SEMANTICS AD-HOC POINTERS # (Dentist)-[:TREATS]->(Patient)
 has_many :out, :patients,
 type: "TREATS", model_class: "Person"
 
 
 # (Dentist)<-[:TEACHES_RUBY]-(Patient)
 has_one :in, :ruby_teacher,
 type: "TEACHES_RUBY", model_class: "Person"

Slide 47

Slide 47 text

GRAPH SEMANTICS AD-HOC POINTERS # (Person)-[:HAS]->(MoveScore) has_many :out, :move_scores, type: "HAS", model_class: "MoveScore"
 # (Person)-[:HAS_CURRENT]->(MoveScore) has_one :out, :current_move_score, type: "HAS_CURRENT", model_class: "MoveScore"
 # (Person)-[:HAS_PREFERRED]->(*) has_one :out, :preferred_move_score, type: "HAS_PREFERRED", model_class: false

Slide 48

Slide 48 text

WHY IS THIS USEFUL?

Slide 49

Slide 49 text

GRAPH SEMANTICS AD-HOC POINTERS MATCH
 (realtor:Person)<-[:CONTACT_OF]-(contact:Person)<- [:CONTACT_INFOS]-(email:EmailContactInfo)
 WHERE
 (contact.rapportive_enqueued IS NULL)
 AND NOT ((contact)<-[:SOCIAL_PROFILES]- (:SocialProfile {type: 'linked_in'}))
 RETURN
 DISTINCT(contact) as contact, realtor.user_id as user_id, collect(email) as emails

Slide 50

Slide 50 text

GRAPH SEMANTICS AD-HOC POINTERS MATCH
 (realtor:Person)<-[:CONTACT_OF]-(contact:Person)<- [:CONTACT_INFOS]-(email:EmailContactInfo)
 WHERE
 (contact.rapportive_enqueued IS NULL)
 AND NOT ((contact)<-[:SOCIAL_PROFILES]- (:SocialProfile {type: 'linked_in'}))
 RETURN
 DISTINCT(contact) as contact, realtor.user_id as user_id, collect(email) as emails

Slide 51

Slide 51 text

GRAPH SEMANTICS AD-HOC POINTERS MATCH
 (node)<-[edge]-(node)<-[edge]-(node)
 WHERE
 (conditional)
 AND NOT ((node)<-[]-(node))
 RETURN stuff

Slide 52

Slide 52 text

WRITE QUERIES ABOUT GRAPH, USING A GRAPH!

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

GRAPH SEMANTICS AD-HOC POINTERS # (Person)-[:HAS]->(MoveScore) has_many :out, :move_scores, type: "HAS", model_class: "MoveScore"
 # (Person)-[:HAS_CURRENT]->(MoveScore) has_one :out, :current_move_score, type: "HAS_CURRENT", model_class: "MoveScore"
 # (Person)-[:HAS_PREFERRED]->(*) has_one :out, :preferred_move_score, type: "HAS_PREFERRED", model_class: false

Slide 55

Slide 55 text

GRAPH SEMANTICS AD-HOC POINTERS # (Person)-[:HAS]->(MoveScore) has_many :out, :move_scores, type: "HAS", model_class: "MoveScore"
 # (Person)-[:HAS_CURRENT]->(MoveScore) has_one :out, :current_move_score, type: "HAS_CURRENT", model_class: "MoveScore"
 # (Person)-[:HAS_PREFERRED]->(*) has_one :out, :preferred_move_score, type: "HAS_PREFERRED", model_class: false

Slide 56

Slide 56 text

GRAPH SEMANTICS AD-HOC POINTERS # (Person)-[:HAS]->(MoveScore) has_many :out, :move_scores, type: "HAS", model_class: "MoveScore"
 # (Person)-[:HAS_CURRENT]->(MoveScore) has_one :out, :current_move_score, type: "HAS_CURRENT", model_class: "MoveScore"
 # (Person)-[:HAS_PREFERRED]->(*) has_one :out, :preferred_move_score, type: "HAS_PREFERRED", model_class: false

Slide 57

Slide 57 text

GRAPH SEMANTICS AD-HOC POINTERS, POLYMORPHIC # Post, Article, Status has_many :comments, as: :commentable # Comment belongs_to :commentable, polymorphic: true

Slide 58

Slide 58 text

GOOD LUCK, WRITING A JOIN QUERY AGAINST THAT!

Slide 59

Slide 59 text

LABELS: POLYMORPHISM, MIXINS, & COMPOSITION GRAPH SEMANTICS

Slide 60

Slide 60 text

EACH NODE IN A GRAPH IS IDENTIFIED BY A LABEL

Slide 61

Slide 61 text

EACH EDGE IN A GRAPH IS IDENTIFIED BY A LABEL

Slide 62

Slide 62 text

ANY NODE IN A GRAPH CAN ANY NUMBER OF LABELS

Slide 63

Slide 63 text

ANY EDGE IN A GRAPH CAN ANY NUMBER OF LABELS

Slide 64

Slide 64 text

LET'S EQUATE A LABEL TO A CLASS

Slide 65

Slide 65 text

EXAMPLE: EMAIL, PHONE, CONTACT INFO

Slide 66

Slide 66 text

EMAIL IS A CONTACT INFO

Slide 67

Slide 67 text

PHONE IS A CONTACT INFO

Slide 68

Slide 68 text

SO, SHARED BEHAVIOUR

Slide 69

Slide 69 text

BUT, DIFFERENT CONSTRAINTS

Slide 70

Slide 70 text

THIS IS CURRENTLY NOT POSSIBLE IN POSTGRES

Slide 71

Slide 71 text

WELL, NOT WITHOUT A LOT OF ADDITIONAL COMPLEXITY

Slide 72

Slide 72 text

EXAMPLE: SIX DEGREES OF KEVIN BACON GRAPH SEMANTICS

Slide 73

Slide 73 text

GOOGLE FOR YOUR FAVOURITE ACTOR'S BACON NUMBER

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

GRAPH SEMANTICS EXAMPLE: SIX DEGREES OF KEVIN BACON (Actor) -[:HAS_WORKED_WITH]->(Actor) -[:HAS_WORKED_WITH]->(Actor) -[:HAS_WORKED_WITH]->(Actor) -[:HAS_WORKED_WITH]->(Actor) -[:HAS_WORKED_WITH]->(Actor)

Slide 76

Slide 76 text

GRAPH SEMANTICS EXAMPLE: SIX DEGREES OF KEVIN BACON Answer to this question has been left to the reader as an exercise.

Slide 77

Slide 77 text

GRAPH SEMANTICS CYPHER THE QUERY LANGUAGE MATCH (you {name:"You"})
 MATCH (expert) -[:WORKED_WITH]-> (db:Database {name:"Neo4j"})
 MATCH path = shortestPath( (you)-[:FRIEND*..5]-(expert))
 RETURN db,expert,path

Slide 78

Slide 78 text

PLENTY OF MORE EXAMPLES, AND GREAT DOCS ON THE OFFICIAL SITE

Slide 79

Slide 79 text

LIBRARIES, ADAPTERS, DEV TOOLS, ADMIN TOOLS ETC. TOOLING

Slide 80

Slide 80 text

FIRST QUESTION THAT COMES TO THE MIND: ACTIVE RECORD?

Slide 81

Slide 81 text

YES AND NO. RUBY GEM "NEO4J" AVAILABLE

Slide 82

Slide 82 text

NO: NOT QUITE AS FULLY FEATURED AS ACTIVE RECORD

Slide 83

Slide 83 text

YES: OFFERS THAT WELL FAMILIAR API WE'VE SEEN EARLIER

Slide 84

Slide 84 text

TOOLING NEO4J.RB API # (Person)-[:HAS]->(MoveScore) has_many :out, :move_scores, type: "HAS", model_class: "MoveScore"
 # (Person)-[:HAS_CURRENT]->(MoveScore) has_one :out, :current_move_score, type: "HAS_CURRENT", model_class: "MoveScore"
 # (Person)-[:HAS_PREFERRED]->(*) has_one :out, :preferred_move_score, type: "HAS_PREFERRED", model_class: false

Slide 85

Slide 85 text

YES: OFFERS ALL THE TOOLS YOU ARE ACQUAINTED WITH

Slide 86

Slide 86 text

TOOLING NEO4J.RB API: ALL YOUR FAMILIAR TOOLS ▸ Properties ▸ Indexes / Constraints ▸ Callbacks ▸ Validation ▸ Associations

Slide 87

Slide 87 text

OF COURSE, THERE ARE CAVEATS

Slide 88

Slide 88 text

NEO4J DOESN'T MAINTAIN TYPES OF PROPERTIES

Slide 89

Slide 89 text

AND SO NEO4J.RB HAS TO

Slide 90

Slide 90 text

ADDITIONAL COGNITIVE OVERLOAD WHEN QUERYING

Slide 91

Slide 91 text

TOOLING NEO4J: NO TYPES # This is a sorted collection
 [2, 3, 5, 7, 11]
 
 # And so is this
 ["11", "2", "3", "5", "7"]

Slide 92

Slide 92 text

NEO4J DOESN'T ALLOW COMPOSITE INDEXES

Slide 93

Slide 93 text

NEO4J DID NOT SUPPORT A BINARY PROTOCOL

Slide 94

Slide 94 text

IT WAS ALL REST BASED

Slide 95

Slide 95 text

I SAID WAS, BECAUSE THIS IS BEING ADDRESSED NOW

Slide 96

Slide 96 text

A BINARY PROTOCOL IS OUT, BUT NOT PRODUCTION READY

Slide 97

Slide 97 text

YOU CAN ALWAYS USE THE JDBC CONNECTOR, FROM JAVA

Slide 98

Slide 98 text

IT DOESN'T SUFFER FROM ANY OF THESE ISSUES

Slide 99

Slide 99 text

WHAT ISSUES, YOU SAID?

Slide 100

Slide 100 text

TOOLING NEO4J: ALL REST, NO BINARY Neo4j::Session::CypherError
 # 1. Unable to load RELATIONSHIP with id 
 # 2. LockClient[80] can't wait on resource
 # 3. Expected a numeric value for empty iterator, but got null

Slide 101

Slide 101 text

TOOLING NEO4J: ALL REST, NO BINARY Faraday::ConnectionFailed
 # 1. too many connection resets
 # 2. connection refused to: 
 # 3. Failed to open TCP connection to
 # 4. execution expired

Slide 102

Slide 102 text

TOOLING NEO4J: ALL REST, NO BINARY Neo4j::ActiveRel::Persistence::RelCreateFaile dError
 # 1. Cannot create rel with unpersisted, invalid to_node

Slide 103

Slide 103 text

FAR TOO MANY TIMES TO COUNT

Slide 104

Slide 104 text

THESE ERRORS WERE OUR PRIMARY REASON FOR SWITCHING

Slide 105

Slide 105 text

THEY CRIPPLED OUR SCALE, AND LIMITED OUR CONCURRENCY

Slide 106

Slide 106 text

DEAL BREAKER!

Slide 107

Slide 107 text

THERE ARE PLENTY MORE THINGS I WANT TO TALK ABOUT

Slide 108

Slide 108 text

MONITORING, BACKUPS, DEPLOYMENTS, PERFORMANCE TUNING, ETC OPS

Slide 109

Slide 109 text

BUT I WILL STOP HERE.

Slide 110

Slide 110 text

CONCLUSION

Slide 111

Slide 111 text

POWERFUL QUERYING, EXCELLENT PERFORMANCE, BEAUTIFUL API, GREAT SEMANTICS

Slide 112

Slide 112 text

A NATURAL CHOICE FOR BUILDING APPLICATIONS WITH RELATIONSHIP FOCUSSED DATA

Slide 113

Slide 113 text

BUT…

Slide 114

Slide 114 text

THE RUBY ECOSYSTEM AND TOOLING LEAVES A LOT TO BE DESIRED

Slide 115

Slide 115 text

SPECIALLY WHEN YOU ARE DEALING WITH HIGH PERFORMANCE AND HIGH CONCURRENCY

Slide 116

Slide 116 text

THANK YOU! THIS IS IT.

Slide 117

Slide 117 text

SWANAND PAGNIS PRINCIPAL ENGINEER, FIRST.IO @_SWANAND ON TWITTER WHO AM I?

Slide 118

Slide 118 text

QUESTIONS? IF TIME PERMITS…