Gempact Ruby FTW

Transcript

1. • First held in 2007 • 1-4 Developers • 48 hours • Ruby based Rack framework http://railsrumble.com
2. • Heroku app provided for deployment • GitHub Repo provided for optional use • Free access to some paid Heroku add-ons http://railsrumble.com
3. • Judged by competitors • Must have submitted an app to judge • Favorite and rank up to ten apps • General public can also “favorite” an app • Judging period is 7 days following end of competition http://blog.railsrumble.com/rules/
4. • Ten Overall Winners • computed from competitors’ favorite lists • Best Solo Award • Public Favorite http://blog.railsrumble.com/rules/
5. Our Submission: GemPact
7. What is GemPact?
8. Can we measure a RubyGem’s relative impact in the Ruby world?
9. The“impact” of books and papers is measured by how often other authors cite them.
11. RubyGems “cite” other gems by including their code as a dependency.
13. And other gems depend on those gems And yet more gems depend on the gems that depend on gems, and still other gems depend on those gems…
14. Transitive Dependencies
15. A gem spec lists a gem’s dependencies. ! But we want the reverse of that.
16. We can put the gem index in a database.
17. But I’m not sure I could write the SQL to reverse all the dependencies.
18. SELECT gems.dependencies FROM gems WHERE gems.name = 'activesupport' AND INCLUDE TRANSITIVE DEPENDENCIES PLEASE; This is just not a question you can easily ask a RDBMS
19. *This is a speculative complaint, of course. We don’t know how long it would take because it sounded too painful to even write the query. We’re not saying RDBMSes are slow in general, but the part about transitive dependencies would definitely be a problem. How long would the query take*?
20. Graph Database to the rescue
21. But first we have to get the raw data. We can get a list of all the gems from rubygems.
22. RubyGems Index • http://rubygems.org/specs.4.8.gz • Gzipped and Marshalled ruby object • ["activerecord", #<Gem::Version "4.1.2">, "ruby"] https://blog.engineyard.com/2014/new-rubygems-index-format
23. Fetching the gem index
24. Next we need the Gem Specs for each gem. We have to get them one by one from the rubygems API. There are about 90k gems in rubygems.
25. https://rubygems.org/api/v1/gems/activerecord.json
26. How should we store it? It doesn’t make much sense as tables. That’s our data.
27. GRAPH DATABASE!!!
28. Our data is a bunch of interconnected nodes. Graph databases store data as a bunch of interconnected nodes.
29. No “impedance mismatch*” *I have no idea what impedance actually is The way our data actually looks: The way our data is stored in the database:
30. How does ruby interface with the graph database?
32. Works like activerecord • Ruby object represents a node in the database. • The object’s class represents the node’s “label” (e.g. User, RubyGem, Pickle) • Relationships between nodes (aka edges) work like AR relationships, but they also have a name and direction
33. RubyGem Model Class
34. Putting data into the graph database
35. Inserting Gems …… ……
36. Getting data out of the graph database The Cypher query language
37. The neo4j web console
38. We use Cypher in our application as well
39. How is Cypher different from SQL?
40. Pattern-matching “Using patterns, you describe the shape of the data you’re looking for.” // find first 5 dependents of the rails gem ! MATCH (rails:RubyGem { name: 'rails' }) MATCH rails<-[:depends_on]-(deps:RubyGem) RETURN rails, deps LIMIT 5;
41. Resources: • http://neo4j.com/developer/get-started/ • http://neo4j.com/books/ (FREE BOOKS!!!) • https://github.com/neo4jrb/neo4j • http://www.gempact.com
42. Questions / Demonstration