DeCYPHERing Graph Databases

A presentation by @stuherbert  for @GanbaroDigital DeCYPHERing Graph Databases With
PHP And Neo4J

Industry veteran: architect, engineer, leader, manager, mentor F/OSS contributor since
1994 Talking and writing about PHP since 2004 Chief Software Archaeologist @GanbaroDigital About Stuart

Follow me I do tweet a lot about non-tech stuff
though :) @stuherbert

@GanbaroDigital Why am I giving this talk?

@GanbaroDigital Story time with Stu!

@GanbaroDigital https://ﬂic.kr/p/nUiUbU Canary Wharf

@GanbaroDigital https://ﬂic.kr/p/Tjs2DM

@GanbaroDigital https://ﬂic.kr/p/ciLwbo

@GanbaroDigital

@GanbaroDigital After describing their business, they said something profound:

@GanbaroDigital ... “we don’t know how to model that in
a database.”

@GanbaroDigital

@GanbaroDigital https://ﬂic.kr/p/ee8yvi

@GanbaroDigital A crucial part of building back-ofﬁce systems is turning
the business model into a RDBMS model.

@GanbaroDigital ?? ?? What if we could use the same
model for the business and the database?

@GanbaroDigital Same Model • Same entities • ... with the
same names • ... and the same terms for connections

@GanbaroDigital I don’t mean use an ORM to pretend it’s
the same. I mean the database itself uses the same model that the business does.

@GanbaroDigital https://ﬂic.kr/p/5LsLri

@GanbaroDigital We can do that using a Graph database.

@GanbaroDigital “ Graph DBs are the BDD of databases.

@GanbaroDigital This isn’t your typical graph database talk.

@GanbaroDigital This isn’t a talk about how to use graph
algorithms.

@GanbaroDigital Other folks already have that covered.

@GanbaroDigital https://neo4j.com/graphgists/

@GanbaroDigital https://neo4j.com/graphgist/the-panamapapers-example-dataset-president-of-azerbaijan

@GanbaroDigital https://neo4j.com/webinars/

@GanbaroDigital https://www.youtube.com/watch?v=cvTFejfE1-k

@GanbaroDigital This is a talk about building (part of) a
business on a Graph DB.

@GanbaroDigital ... and how to do this using PHP.

@GanbaroDigital This is my experience. This is how I did
it, and why.

@GanbaroDigital Other approaches exist. I’m here to learn from you
too!

@GanbaroDigital Please ask questions as we go.

@GanbaroDigital In This Talk 1. 100 Years Of History 2.
Querying Graphs With CYPHER 3. BOLTing On PHP

@GanbaroDigital 100 Years Of History

@GanbaroDigital Let’s look at a real example. (Sorry if you
don’t like football!)

@GanbaroDigital https://chelseafan12.com

@GanbaroDigital https://chelseafan12.com/fanzone/historical-ﬁxtures/

@GanbaroDigital https://chelseafan12.com/fanzone/player-library/

@GanbaroDigital https://chelseafan12.com/fanzone/player-library/kerry-dixon/

@GanbaroDigital ChelseaFan12 Fanzone • Fixture list back to 1905 •
Key events from each match • Squad data to match • Rich data on each squad member • + data on Chelsea opponents

@GanbaroDigital Underlying Dataset • 420,000+ records • 3.2 million+ foreign
keys • average of nearly 8 foreign keys per record

@GanbaroDigital Person Spell Team Match Season Goal Media Biography Competition

@GanbaroDigital We’ll use this real dataset to learn how to
write queries.

@GanbaroDigital Querying Graphs With CYPHER

@GanbaroDigital CYPHER is Neo4J’s query language

@GanbaroDigital CYPHER is awesome*

@GanbaroDigital What does CYPHER look like?

@GanbaroDigital With CYPHER your queries look like a graph diagram.

@GanbaroDigital MATCH (a)-[r]->(b)

@GanbaroDigital How To Read CYPHER • Anything in brackets is
a record  (a node in graph terms) • Anything in square brackets is a foreign key  (a relationship in graph terms) • Relationships can have direction

@GanbaroDigital How do we ﬁnd records using CYPHER?

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label)

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label) { from

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label) { to

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label) { connected by

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label) { to { from { connected by

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label)

@GanbaroDigital CYPHER MATCH • ‘a’ and ‘b’ are named results
• if you want to use it later in your query,  give it a name • think of labels as record types / table names

@GanbaroDigital “ With CYPHER, you ﬁnd records by relationships and
ﬁlter them by content

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label)  WHERE a.ﬁeld = value

@GanbaroDigital Tell CYPHER what data to return from the query.

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label)  WHERE a.ﬁeld = value RETURN b

@GanbaroDigital MATCH can take multiple  relationships.

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label)<-[r2:label]-[c:label]  WHERE a.ﬁeld = value AND c.ﬁeld =
value RETURN b

@GanbaroDigital CYPHER supports aggregate queries :)

@GanbaroDigital MATCH (a:label)-[r:label]->(b:label)<-[r2:label]-(c:label)  WHERE a.ﬁeld = value AND c.ﬁeld =
value WITH b, c MATCH (c)-[r3:label]->(d:label)<-[r4:label]-(b) RETURN c, d

@GanbaroDigital https://neo4j.com/docs/developer-manual/current/cypher/

@GanbaroDigital http://markhneedham.com/blog/tag/cypher/

@GanbaroDigital Querying The Dataset

@GanbaroDigital 3 Actual Queries • A-Z list • Player proﬁle
cards • Matches played in a season

@GanbaroDigital Query #1 The A-Z List

@GanbaroDigital Which players do we show on the A-Z page?

@GanbaroDigital “ To design a basic CYPHER query, simply say
what you’re trying to ﬁnd.

@GanbaroDigital “Find players who had a spell at ‘Chelsea’”

@GanbaroDigital MATCH (p:person)

@GanbaroDigital MATCH (p:person)-[:had_spell]->(:spell)

@GanbaroDigital MATCH (p:person)-[:had_spell]->(:spell)-[:at]->(t:team)  WHERE t.name =~ “(?i)chelsea”

@GanbaroDigital MATCH (p:person)-[:had_spell]->(:spell)-[:at]->(t:team)  WHERE t.name =~ “(?i)chelsea”  RETURN DISTINCT(p)  ORDER
BY p.sortname ASC

@GanbaroDigital 3 Things About This Query • Text searching •
Duplicate rows • Pre-calculated sort order

BY p.sortname ASC

@GanbaroDigital String-matching is case-sensitive in Neo4J. I end up using
regexes for all my string matching.

BY p.sortname ASC

@GanbaroDigital Players may have played for the club multiple times.
Use DISTINCT() to avoid returning duplicate rows.

BY p.sortname ASC

@GanbaroDigital Our pre-calculated ‘sortname’ saves any duplication in different client
apps.

@GanbaroDigital “ CYPHER queries look like how we’d describe what
we’re trying to ﬁnd.

@GanbaroDigital The query that’s live today is a little different.

@GanbaroDigital Direct relationships often describe things clearer. Especially when it’s
complicated.

@GanbaroDigital Person Spell Match Season Goal Media Biography Competition Team

@GanbaroDigital MATCH (p:person)-[:played_for]->(t:team)  WHERE t.name =~ “(?i)chelsea”  RETURN p  ORDER
BY p.sortname ASC

@GanbaroDigital Just because you’re using a Graph DB, you don’t
have to traverse longer paths.

@GanbaroDigital “ Create new, shorter paths for anything you use
regularly.

@GanbaroDigital We can easily add different relationships between the same
two records.

@GanbaroDigital This is very handy when new requirements arrive!

@GanbaroDigital MATCH (t:team)-[:current_squad]->(p:person)  WHERE t.name =~ “(?i)chelsea”  RETURN p  ORDER
BY p.sortname ASC

@GanbaroDigital We can do all this in an RDBMS. It’s
just so much easier to do it in a Graph database.

@GanbaroDigital Graph Relationships • No schema changes / migrations •
No extra columns for the foreign keys • No NULLs for empty foreign keys • Just get on and do it :)

@GanbaroDigital Shipped and deployed an average of 3 updated schemas*
per month for 12 months. * equivalent to RDBMS

@GanbaroDigital Zero schema migrations required :-)

@GanbaroDigital Query #2 Proﬁle Cards

@GanbaroDigital How do we build each player’s proﬁle card for
the A-Z page?

@GanbaroDigital

@GanbaroDigital MATCH (t:team)-[:played_for]->(p:person)  WHERE t.name =~ “(?i)chelsea”  WITH DISTINCT(p), t
OPTIONAL MATCH (p)-[:proﬁle_pic]->(m:media)-[:team]->(t)  WHERE m.category = “thumbnail” RETURN p, COLLECT([m]) as proﬁle_pics  ORDER BY p.sortname ASC

@GanbaroDigital Use WITH to feed the result rows into a
second query.

@GanbaroDigital The second query can reduce the size of the
ﬁnal result set. It can also add new records to the ﬁnal result set.

@GanbaroDigital Use OPTIONAL MATCH when querying incomplete datasets.

@GanbaroDigital In this case, my client doesn’t have a proﬁle
picture for every player who has played for a given team.

@GanbaroDigital Note how the name of the relationship tells us
what kind of image the ‘media’ record is.

@GanbaroDigital “In real life, identity often depends on context. Graphs
make it easy for us to model that.

@GanbaroDigital For example, UK addresses can be many different things
all at once.

@GanbaroDigital The list of possible addresses would be the records.
The type of address - the context - would be the relationship name.

@GanbaroDigital CYPHER returns result rows. Use COLLECT to put multiple
records into a single result row.

@GanbaroDigital Query #3: Matches In A Season

@GanbaroDigital Which matches were played in a season?

@GanbaroDigital “Find every league match played by Chelsea in the
2016/2017 season”

@GanbaroDigital MATCH (m:match)-[:competition]->(c:competition)  WHERE c.name = “Premier League”

2016/2017 season”

@GanbaroDigital MATCH (t:team)<-[:home_team|away_team]-(m:match) -[:competition]->(c:competition)  WHERE c.name =~ “(?i)Premier League” AND
t.name =~ “(?i)Chelsea”

2016/2017 season”

@GanbaroDigital ?? ?? How do we ﬁnd the season?

@GanbaroDigital Football seasons start in one year and ﬁnish in
the next.

@GanbaroDigital “ Design data sets to make records discoverable.

@GanbaroDigital Data dimensions are a classic solution to making records
discoverable. And they are perfectly suited to graph databases :)

@GanbaroDigital https://www.kimballgroup.com

@GanbaroDigital Data dimensions are standardised data sets that you can
connect different record types to.

@GanbaroDigital For example, the list of UK addresses could be
a data dimension.

@GanbaroDigital Use relationships into the data dimension (e.g. “delivery address”,
“billing address”) to provide identity.

@GanbaroDigital You can use whatever datasets you want as data
dimensions. The point is to standardise them, and preload them in so that you can link to them.

@GanbaroDigital Different aspects of time are a classic set of
data dimensions.

@GanbaroDigital We link each ‘season’ to a ‘start year’ and
an ‘end year’ to make them easy to ﬁnd.

@GanbaroDigital Person Spell Team Match Season Goal Media Year Competition

@GanbaroDigital MATCH (s:season)-[:started_on_year]->(y:year)  WHERE y.year = 2016

2016/2017 season”

@GanbaroDigital The query could search for things in the same
order that we’d say it out loud.

@GanbaroDigital Sometimes, it’s better to change the query order to
ﬁnd the smaller result sets ﬁrst.

@GanbaroDigital MATCH (s:season)-[:started_on_year]->(y:year)  WHERE y.year = 2016  WITH s  MATCH
(t:team)<-[:home_team|away_team]-(m:match) -[:competition]->(c:competition)  WHERE c.name =~ “(?i)Premier League” AND t.name =~ “(?i)Chelsea”  AND (m)-[:season]->(s)

@GanbaroDigital In CYPHER, we can use relationships between records in
WHERE clauses too.

@GanbaroDigital We’ve Covered ... • Finding by relationships • Filtering
by content & relationships • Aggregate queries • Identity by context • Discoverability by data dimensions

@GanbaroDigital BOLTing On PHP

@GanbaroDigital How do we get at all this graph goodness
from the world’s best back-ofﬁce programming language?

@GanbaroDigital composer require graphaware/neo4j-php-client

@GanbaroDigital // connect to Neo4J using the BOLT protocol //
it’s a little faster than the HTTP API use GraphAware\Neo4j\Client\Client; use GraphAware\Neo4j\Client\ClientBuilder; $client = ClientBuilder::create() ->addConnection(‘default’, ‘bolt://neo4j:7687') ->build();

@GanbaroDigital // build the query using a HEREDOC // use
{ } as placeholders for query parameters  $query = <<<EOS MATCH (t:team)-[:current_squad]->(p:person)  WHERE t.name =~ {teamname}  RETURN p  ORDER BY p.sortname ASC EOS;

@GanbaroDigital // build the query parameters list $queryParams = [
‘teamname’ => ‘(?i)chelsea’, ];

@GanbaroDigital // run the query $result = $client->run($query, $queryParams); //
the records don’t come back as an assoc array :( $records = $result->getRecords();

@GanbaroDigital https://github.com/graphaware/neo4j-php-client

@GanbaroDigital Summing Up

@GanbaroDigital “ Graph DBs are the BDD of databases.

@GanbaroDigital “ CYPHER is easier to learn than SQL.

@GanbaroDigital “ The majority of queries are much easier to
write in CYPHER than in SQL.

@GanbaroDigital “ To design a basic CYPHER query, simply say
what you’re trying to ﬁnd.

@GanbaroDigital “ Create new, shorter paths for anything you use
regularly.

@GanbaroDigital “Graph DBs can take advantage of techniques developed for
RDBMS like data dimensions!

Thank You Any Questions? A presentation by @stuherbert  for @GanbaroDigital

DeCYPHERing Graph Databases

DeCYPHERing Graph Databases

More Decks by Stuart Herbert

Other Decks in Programming

Featured

Transcript