Traversing the Academic Graph

Graphs are a powerful abstraction in many highly connected domains. Social networks are an obvious example, but less obvious include biology and security. Scientific publishing is one of those domains. Citation and co-authorship in research form a huge graph. I'll share my experience using just such a graph to power our academic search engine, http://scholr.ly. I'll focus on how we use a graph database to solve a number of problems, including search, disambiguation, and recommendations.

A rough script of the presentation can be found at http://mattluongo.com/post/presentation-at-datadaytexas.

Transcript

1. Traversing the Academic Graph Matt Luongo
2. @mhluongo github.com/mhluongo Matt Luongo
4. Search for research.
5. Find relevant papers and authors.
6. Explore an author's work.
7. Motivation
8. Motivation
9. Search & Graphs
12. On the other hand...
14. The Web
15. Academic Papers
16. Academic Papers
17. “Deep belief networks”
18. Geoffrey Hinton, deep learning pioneer
19. Geoffrey Hinton, deep learning pioneer
20. Professor Hinton seems like a busy guy
21. Let's Get Technical
22. Why Neo4j?
26. Use Cases
27. A comment on property graphs
28. A comment on query examples • Cypher • Declarative • SQL-like • Easy, smooth pattern matching • Neo4j only • Gremlin • DSL atop JVM languages like Groovy • Lower-level, but more powerful • Cross-database
29. A comment on query examples These snippets are untested.
30. Similar Profiles
31. Similar Profiles
32. Similar Profiles
33. Similar Profiles START author=node(123) MATCH author-[:wrote]->(work)-[:cites]->(cited_work) \ <-[:cites]-(other_works)<-[:wrote]-(other_author) WITH author, other_author, COUNT(cited_work) AS work_in_common ORDER BY work_in_common RETURN other_author
34. Similar Profiles
35. Similar Profiles
36. Similar Profiles START author=node(123) MATCH author-[:wrote]->(work)<-[:wrote]-(coauthor) \ -[:wrote]->(other_work) \ <-[:wrote]-(second_coauthor) WITH author, second_coauthor, COUNT(coauthor) AS shared_coauthors ORDER BY shared_coauthors RETURN second_coauthor
37. Entity Resolution
38. Entity Resolution How do we reconcile different data sources? What happens when people share names? How do we know who's who?
39. Entity resolution is an active area of research.
41. E. Agichtein and L. Gravano. Snowball: Extracting relations from large plain-text collections. In Proceedings of the Fifth ACM International Conference on Digital Libraries, 2000.
42. E. Agichtein and L. Gravano. Snowball: Extracting relations from large plain-text collections. In Proceedings of the Fifth ACM International Conference on Digital Libraries, 2000.
43. E. Agichtein and L. Gravano. Snowball: Extracting relations from large plain-text collections. In Proceedings of the Fifth ACM International Conference on Digital Libraries, 2000.
44. E. Agichtein and L. Gravano. Snowball: Extracting relations from large plain-text collections. In Proceedings of the Fifth ACM International Conference on Digital Libraries, 2000.
45. Entity Resolution votes = [:] g.v(clusterIds).out(‘clusters’).map.each{ properties -> properties.each{ votes[it.key] = (votes[it.key] ?: [:]) votes[it.key][it.value] = \ (votes[it.key][it.value] ?: 0) + 1 } } newClusterProperties = votes.collectEntries{prop, valueVotes -> [prop, valueVotes.sort{-it.value}[0].key] }
46. Search
47. Search We'd like to show the expected publication results on the left.
48. Search We'd like to show the expected publication results on the left. On the right, we want to show influencers based on the publication results.
49. Users can search for a name + a topic.
50. Search authorCounts = [:] g.v(publicationIds).in(‘WROTE’).\ groupCount(authorCounts).iterate() return authorCounts.collect{author, count → [author, count * authorBoost]}.sort{-it[1]} }
51. Search authorCounts = [:] coauthorCounts = [:] g.v(publicationIds).in(‘WROTE’) \ .groupCount(authorCounts).out(‘WROTE’) \ .in(‘WROTE’).groupCount(coauthorCounts) \ .iterate() // IMAGINE - poor man’s “histogram” totalAuthorCounts = [:] return totalAuthorCounts.sort(-it.value}
52. Search citedAuthorCounts = [:] g.v(publicationIds).out(‘cite’).in(‘wrote’) \ .groupCount(citedAuthorCounts)
53. Still with me? http://scholr.ly
54. Get Involved neo4j.org meetup.com/graph-database-austin
55. April 15th – Austin Graph DB Meetup April 16th – Austin Neo4j Tutorial Upcoming Events More details at meetup.com/graph-database-austin
56. Questions?
57. Thanks!
58. Bibliography Nicholas Menghini and Alex Fuller from the Noun Project – thanks for the icons! TinkerPop – thanks for the graphic!