Easy Rewrites with Ruby and Science!

Ruby makes it easy to prototype a new data model or codepath in your application and get it into production quickly to test it out. At GitHub, we've built on top of this concept with our open source dat-science gem, which helps measure and validate two codepaths at runtime. This talk will cover how we used this gem and its companion analysis gem to undertake (and complete!) a large-scale rewrite of a critical piece of our Rails app -- the permissions model -- live, side-by-side, and in production.

Transcript

1. Easy* Rewrites with ruby and @jesseplusplus GitHub Science!
2. Jesse Toth @jesseplusplus backend ! # $ % things for &
3. Easy* Rewrites with ruby and Science!
4. ' * Rewrites are never easy
5. a rewrite ? ! ? !
6. ' “Create a more flexible system to grant and revoke access to repositories, forks, issues, pull requests, teams, and organizations”
7. history
8. First, there was collaboration
9. Then, there was organization
10. ! separate ways of granting permissions living side-by-side
11. ! ! mysql> describe permissions; +---------------+---------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +---------------+---------+------+-----+---------+-------+ | user_id | int(11) | YES | MUL | NULL | | | repository_id | int(11) | YES | MUL | NULL | | +---------------+---------+------+-----+---------+-------+ 2 rows in set (0.00 sec)
12. ! ! mysql> describe team_members; +-----------------+----------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-----------------+----------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | team_id | int(11) | YES | MUL | NULL | | | user_id | int(11) | YES | MUL | NULL | | | repository_id | int(11) | YES | MUL | NULL | | | created_at | datetime | YES | MUL | NULL | | | updated_at | datetime | YES | MUL | NULL | | +-----------------+----------+------+-----+---------+----------------+ 6 rows in set (0.01 sec)
13. lists of repositories, pull requests, teams, etc. # Find all repositories which this organization # controls the access to. ! def organization.controlled_repositories . . . end
14. lists of repositories, pull requests, teams, etc. # Find all pull requests to which this user # has access. Access to PRs follow the user’s # access rights to the PR’s repository. ! def user.accessible_pull_requests . . . end
15. ! separate ways of granting permissions living side-by-side " tons of bugs and edge cases around transitional states
17. ! separate ways of granting permissions living side-by-side " tons of bugs and edge cases around transitional states # performance degredation
18. SELECT r.id FROM r, (( SELECT r.id as r_ids, 2 as perms from r WHERE r.owner_id = 99999 AND (r.x = 0) ) UNION ALL ( SELECT r.id as r_ids, 1 as perms from r INNER JOIN p ON r.id = p.r_id WHERE p.u_id = 99999 AND (r.x = 0) ) UNION ALL ( SELECT r.id as r_ids, 2 as perms from r INNER JOIN t ON r.o_id = t.o_id INNER JOIN t_m ON t.id = t_m.t_id WHERE t.name = 'X' AND t_m.u_id = 99999 AND (r.x = 0) ) UNION ALL ( SELECT r.id as r_ids, GROUP_CONCAT(distinct t.p) as perms from r INNER JOIN t_m r_t ON r.id = r_t.r_id INNER JOIN t ON r_t.t_id = t.id INNER JOIN t_m u_t ON t.id = u_t.t_id WHERE u_t.u_id = 99999 AND t.name != 'X' AND t.p in (2, 1, 0) AND (r.x = 0) GROUP BY r.id ) UNION ALL ( SELECT r.id as r_ids, 0 as perms from r JOIN u ON r.plan_owner_id = u.id JOIN t ON t.o_id = u.id JOIN t_m ON t.id = t_m.t_id WHERE u.type = 'XX' AND t.name = 'X' AND t_m.u_id = 99999 AND (r.x = 0) AND r.parent_id IS NOT NULL )) AS unioned WHERE r.id = r_ids;
21. rewrite !
23. ' Goals • Simple, flexible interface to grant and revoke a general permission • Fast — sub-second permission lookups • Easy to integrate and operate — MySQL
24. $% spike
26. rewrite …and refactor
27. ' tests weren’t modeling production data
30. class Repository def pullable_by?(user) science "ability.repository.pullable-by" do |e| e.context :user => (user.id if user), :repo => id ! e.use { pullable_by_legacy?(user) } e.try { pullable_by_abs?(user) } end end ! def pullable_by_abs?(user) user.can? :read, self end ! def pullable_by_legacy?(user) # original code for pullable_by? # collaborators || team || special case end end
31. class Repository def pullable_by?(user) science "ability.repository.pullable-by" do |e| e.context :user => (user.id if user), :repo => id ! e.use { pullable_by_legacy?(user) } e.try { pullable_by_abs?(user) } end end ! def pullable_by_abs?(user) user.can? :read, self end ! def pullable_by_legacy?(user) # original code for pullable_by? # collaborators || team || special case end end
32. class Repository def pullable_by?(user) science "ability.repository.pullable-by" do |e| e.context :user => (user.id if user), :repo => id ! e.use { pullable_by_legacy?(user) } e.try { pullable_by_abs?(user) } end end ! def pullable_by_abs?(user) user.can? :read, self end ! def pullable_by_legacy?(user) # original code for pullable_by? # collaborators || team || special case end end
33. class Repository def pullable_by?(user) science "ability.repository.pullable-by" do |e| e.context :user => (user.id if user), :repo => id ! e.use { pullable_by_legacy?(user) } e.try { pullable_by_abs?(user) } end end ! def pullable_by_abs?(user) user.can? :read, self end ! def pullable_by_legacy?(user) # original code for pullable_by? # collaborators || team || special case end end
34. class Repository def pullable_by?(user) science "ability.repository.pullable-by" do |e| e.context :user => (user.id if user), :repo => id ! e.use { pullable_by_legacy?(user) } e.try { pullable_by_abs?(user) } end end ! def pullable_by_abs?(user) user.can? :read, self end ! def pullable_by_legacy?(user) # original code for pullable_by? # collaborators || team || special case end end
35. class Repository def pullable_by?(user) science "ability.repository.pullable-by" do |e| e.context :user => (user.id if user), :repo => id ! e.use { pullable_by_legacy?(user) } e.try { pullable_by_abs?(user) } end end ! def pullable_by_abs?(user) user.can? :read, self end ! def pullable_by_legacy?(user) # original code for pullable_by? # collaborators || team || special case end end
36. class Scientist::Experiment def publish(event, payload) instrument "science.#{event}", payload end end
37. subscribe /^science\./ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.total" timing "#{experiment}.control", payload[:control][:duration] timing "#{experiment}.candidate", payload[:candidate][:duration] end ! subscribe /^science\.mismatch/ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.wrong" Redis.lpush "#{experiment}.mismatch", payload.to_json end
38. subscribe /^science\./ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.total" timing "#{experiment}.control", payload[:control][:duration] timing "#{experiment}.candidate", payload[:candidate][:duration] end ! subscribe /^science\.mismatch/ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.wrong" Redis.lpush "#{experiment}.mismatch", payload.to_json end
39. subscribe /^science\./ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.total" timing "#{experiment}.control", payload[:control][:duration] timing "#{experiment}.candidate", payload[:candidate][:duration] end ! subscribe /^science\.mismatch/ do |name, . . ., payload| name = payload[:experiment] experiment = "science.#{name}" ! increment "#{experiment}.wrong" Redis.lpush "#{experiment}.mismatch", payload.to_json end
40. subscribe /^science\./ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.total" timing "#{experiment}.control", payload[:control][:duration] timing "#{experiment}.candidate", payload[:candidate][:duration] end ! subscribe /^science\.mismatch/ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.wrong" Redis.lpush "#{experiment}.mismatch", payload.to_json end
41. subscribe /^science\./ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.total" timing "#{experiment}.control", payload[:control][:duration] timing "#{experiment}.candidate", payload[:candidate][:duration] end ! subscribe /^science\.mismatch/ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.wrong" Redis.lpush "#{experiment}.mismatch", payload.to_json end
42. subscribe /^science\./ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.total" timing "#{experiment}.control", payload[:control][:duration] timing "#{experiment}.candidate", payload[:candidate][:duration] end ! subscribe /^science\.mismatch/ do |name, . . ., payload| experiment = "science.#{payload[:experiment]}" ! increment "#{experiment}.wrong" Redis.lpush "#{experiment}.mismatch", payload.to_json end
43. ' lessons learned
45. user.can? :read, repository repository.grant user, :write repository.revoke user
46. +------+ | User |-------------- +------+ +------+ | Team |-------------- +------+ +------------+ | Repository |-------- +------------+
47. +------+ | User |---------------- +------+ +------+ 1 | Team |--------+------- +------+ | | +------------+ v | Repository |---------- +------------+
48. +------+ 2 | User |-----------+---- +------+ | | +------+ 1 v | Team |--------+------- +------+ | | +------------+ v | Repository |---------- +------------+
49. +------+ 2 3 | User |-----------+--+- +------+ | . | . +------+ 1 v . | Team |--------+-----.- +------+ | . | . +------------+ v v | Repository |---------- +------------+
50. & wrote the core of Abilities in a few months ' once Abilities was written, wrote migrators to generate Abilities data from legacy data % dark ship writing to Abilities and the legacy tables at the same time
51. science all the things
52. ' ( examining the data
56. –Johnny Appleseed “Type a quote here.” analyzing mismatches
57. irb(main):001> Redis.llen “science.ability.repository.pullable-by.mismatch” => 3283 ! irb(main):002> result = Redis.rpop “science.ability.repository.pullable- by.mismatch” => {"from"=>"GitHub::Jobs::ProcessEvent", "experiment"=>"ability.repository.pullable-by", "user"=>XXXXXXX, “repo"=>XXXXXXX, "timestamp"=>"2014-05-28T18:53:16-07:00", "candidate"=>{"duration"=>0.41368900000000003, "exception"=>nil, "value"=>false}, "control"=>{"duration"=>1.141963, "exception"=>nil, "value"=>true}, "first"=>"control"} !
58. irb(main):001> Redis.llen “science.ability.repository.pullable-by.mismatch” => 3283 ! irb(main):002> result = Redis.rpop “science.ability.repository.pullable- by.mismatch” => {"from"=>"GitHub::Jobs::ProcessEvent", "experiment"=>"ability.repository.pullable-by", "user"=>XXXXXXX, “repo"=>XXXXXXX, "timestamp"=>"2014-05-28T18:53:16-07:00", "candidate"=>{"duration"=>0.41368900000000003, "exception"=>nil, "value"=>false}, "control"=>{"duration"=>1.141963, "exception"=>nil, "value"=>true}, "first"=>"control"} !
59. irb(main):001> Redis.llen “science.ability.repository.pullable-by.mismatch” => 3283 ! irb(main):002> result = Redis.rpop “science.ability.repository.pullable- by.mismatch” => {"from"=>"GitHub::Jobs::ProcessEvent", "experiment"=>"ability.repository.pullable-by", "user"=>XXXXXXX, “repo"=>XXXXXXX, "timestamp"=>"2014-05-28T18:53:16-07:00", "candidate"=>{"duration"=>0.41368900000000003, "exception"=>nil, "value"=>false}, "control"=>{"duration"=>1.141963, "exception"=>nil, "value"=>true}, "first"=>"control"} !
60. irb(main):001> Redis.llen “science.ability.repository.pullable-by.mismatch” => 3283 ! irb(main):002> result = Redis.rpop “science.ability.repository.pullable- by.mismatch” => {"from"=>"GitHub::Jobs::ProcessEvent", "experiment"=>"ability.repository.pullable-by", "user"=>XXXXXXX, “repo"=>XXXXXXX, "timestamp"=>"2014-05-28T18:53:16-07:00", "candidate"=>{"duration"=>0.41368900000000003, "exception"=>nil, "value"=>false}, "control"=>{"duration"=>1.141963, "exception"=>nil, "value"=>true}, "first"=>"control"} !
61. " ' )*
62. ' data quality
63. " ' )*
64. ' ( performance problems
68. rewrite …and refactor …and ) ) …and * # repair …and (
69. ) progress organizations teams repositories
71. ) progress organizations teams repositories
73. ) progress organizations teams repositories
74. ' One Last Data Quality Issue
76. + #
82. ) progress organizations teams repositories
83. –Johnny Appleseed “Type a quote here.” http://github.com/github/scientist + scientist
84. Thank you!