Upgrading GitHub from Ruby 2.6 to 2.7

It's no secret that the upgrade to Ruby 2.7 is difficult — fixing the keyword argument, URI, and other deprecation warnings can feel overwhelming, tedious, and never ending. We experienced this first-hand at GitHub; we fixed over 11k+ warnings, sent patches to 15+ gems, upgraded 30+ gems, and replaced abandoned gems. In this talk we’ll look at our custom monkey patch for capturing warnings, how we divided work among teams, and the keys to a successful Ruby 2.7 upgrade. We’ll explore why upgrading is important and take a dive into Ruby 2.7’s notable performance improvements.

Transcript

1. UPGRADING GITHUB from Ruby 2.6 to 2.7 a Eileen M. Uchitelle | @eileencodes
2. Hello! I’m Eileen M. Uchitelle Principal Engineer at GitHub Rails Core Team Find me: @eileencodes
3. UPGRADING GITHUB from Ruby 2.6 to 2.7 Eileen M Uchitelle | @eileencodes a
4. GitHub is born 2007 2008 2009 Forked Rails & Ruby Public launch
5. a
6. a Why was this Ruby upgrade hard?
7. a Running GitHub Deprecation Free
8. a
9. a Over 11k deprecation warnings!
10. DEPRECATION: Separation of positional and keyword arguments
11. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end DEPRECATION: Separation of arguments
12. MyClass.new({ part_1: "I work fine", part_2: "in Ruby 2.6" }) => "I work fine in Ruby 2.6" DEPRECATION: Separation of arguments
13. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) => "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here DEPRECATION: Separation of arguments
14. DEPRECATION: Separation of arguments MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) => "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here The Caller
15. DEPRECATION: Separation of arguments MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) => "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here The Definition
16. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) MyClass.new( part_1: "I don't throw warnings", part_2: "in 2.7" ) => "I don't throw warnings in 2.7" DEPRECATION: Separation of arguments
17. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) MyClass.new(**{ part_1: "I don't throw warnings", part_2: "in 2.7" }) => "I don't throw warnings in 2.7" DEPRECATION: Separation of arguments
18. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end class AbstractClass def initialize(*args) MyClass.new(*args) end end DEPRECATION: Separation of arguments
19. AbstractClass.new( part_1: "I throw warnings", part_2: "in 2.7" ) DEPRECATION: Separation of arguments
20. AbstractClass.new( part_1: "I throw warnings", part_2: "in 2.7" ) example.rb:9: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here DEPRECATION: Separation of arguments
21. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end class AbstractClass def initialize(**kwargs) MyClass.new(**kwargs) end end DEPRECATION: Separation of arguments
22. class MyClassJob def perform(part_1:, part_2:) puts "#{part_1} #{part_2}" end end DEPRECATION: Separation of arguments
23. module ActiveJob module Enqueuing def perform_later(*args) job_or_instantiate(*args).enqueue end end end DEPRECATION: Separation of arguments
24. module ActiveJob module Enqueuing def perform_later(*args) job_or_instantiate(*args).enqueue end ruby2_keywords(:perform_later) if respond_to?(:ruby2_keywords, true) end end DEPRECATION: Separation of arguments
25. MyClassJob.perform_later({ part_1: "I am a", part_2: "job with kwargs" }) activejob/lib/active_job/execution.rb:48: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call app/jobs/my_job_class_job.rb:2: warning: The called method `perform' is defined here DEPRECATION: Separation of arguments
26. DEPRECATION: URI method deprecations
27. DEPRECATION: URI Methods URI.encode URI.decode URI.escape URI.unescape
28. URI.escape("https://google.com?query_param=1") => "https://google.com?query_param=1" uri_example.rb:1: warning: URI.escape is obsolete DEPRECATION: URI Methods
29. URI.escape("https://google.com?query_param=1") Addressable::URI.escape("https://google.com? query_param=1") => "https://google.com?query_param=1" DEPRECATION: URI Methods
30. URI.escape("https://goog\nle.com?query_param=1") => "https://goog%0Ale.com?query_param=1" Addressable::URI.escape("https://goog\nle.com? query_param=1") => Addressable::URI::InvalidURIError (Invalid character in host: 'goog) le.com' DEPRECATION: URI Methods
31. URI.escape("https://goog\nle.com?query_param=1") => "https://goog%0Ale.com?query_param=1" CGI.escape("https://goog\nle.com?query_param=1") => "https%3A%2F%2Fgoog%0Ale.com%3Fquery_param%3D1" DEPRECATION: URI Methods
32. a How to Fix More Than 11k Deprecations
33. a UPGRADING: Dual-booting our application
34. # config/ruby-version RUBY_NEXT_SHA = "d1ba554" RUBY_SHA = "dcc231c" if ENV["RUBY_NEXT"] print RUBY_NEXT_SHA else print RUBY_SHA end UPGRADING: Dual-booting
35. RUBY_NEXT=1 bin/rails server RUBY_NEXT=1 bin/rails console RUBY_NEXT=1 bin/rails test path/to/test_file.rb UPGRADING: Dual-booting
36. UPGRADING: Dual-booting
37. UPGRADING: Dual-booting
38. a UPGRADING: Monkey patch Warning module
39. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
40. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
41. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
42. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
43. activejob/lib/active_job/execution.rb:48: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call activejob/lib/active_job/execution.rb:48:in `block in perform_now' ... test/integration/a_test.rb:6:in `block in test_something' activesupport/lib/active_support/testing/assertions.rb:34:in `assert_nothing_raised' activejob/lib/active_job/test_helper.rb:591:in `perform_enqueued_jobs' test/integration/my_class_job_test.rb:5:in `test_my_class_job' ... app/jobs/my_job_class_job.rb:2: warning: The called method `perform' is defined here UPGRADING: Warning monkey patch
44. def test_my_class_job perform_enqueued_jobs do MyClassJob.perform_later({ part_1: "I am a", part_2: "job with kwargs" }) end ... end UPGRADING: Warning monkey patch
45. module Warning def self.warn(message) line = caller_locations.find do |location| location.path.end_with?("_test.rb") end WarningsCollector.instance << [message.chomp, line.path] STDERR.print(message) ... UPGRADING: Warning monkey patch
46. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt") File.open(path, "a") do |f| @data.each do |message, origin| f.puts [message, origin].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
47. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt") File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
48. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt") File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
49. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt") File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
50. warnings = {} Dir["tmp/warning*.txt"].each do |file| File.read(file).split("\n").each do |filepath| message, filepath = line.split("*^.^*") warnings[message] ||= Message.new(message) warnings[message].paths << filepath if filepath end end warnings.values.each do |warning| warning.owner ||= CODEOWNERS.for(source_file).keys.first.to_s end UPGRADING: Warning monkey patch
51. - [ ] `test/lib/platform/mutations/ update_mobile_push_notification_schedules_test.rb` - **warnings** - Line 118: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - [ ] `test/test_helpers/newsies/deliver_notifications_job_test_helper.rb` - **warnings** - Line 1282: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - Line 1283: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - **test suites that trigger these warnings** UPGRADING: Warning File Generation
52. UPGRADING: Warning monkey patch
53. UPGRADING: Warning File Generation
54. a UPGRADING: Fixing warnings in gems
56. a UPGRADING Replace abandoned gems
57. a UPGRADING Preventing regressions
58. a Our Deploy Process
59. a DEPLOYING: Make smaller change sets
60. a DEPLOYING: Testing on staging
61. a
62. a DEPLOYING: Slow & incremental rollout
63. 2% Incremental rollout
64. 2% Incremental rollout
65. 2% 0% Incremental rollout
66. 2% 0% 2% Incremental rollout
67. 2% 0% 2% 30% Incremental rollout
68. 2% 0% 2% 30% 60% Incremental rollout
69. 2% 0% 2% 30% 60% 30% Incremental rollout
70. 2% 0% 2% 30% 60% 30% 100% Incremental rollout
71. a DEPLOYING: Strive for boring deploys
72. a Features Worth Upgrading For
73. FEATURE: Performance improvements
74. FEATURES: Boot-time decrease
75. FEATURES: Boot-time decrease
76. FEATURES: Allocations decrease
77. FEATURE: Method#inspect improvements
78. class MyClass ... def my_method(arg, part_1:) end end obj = MyClass.new(part_1: "a", part_2: "b") obj.method(:my_method) => #<Method: MyClass#my_method> FEATURE: Method#inspect improvements
79. class MyClass ... def my_method(arg, part_1:) end end obj = MyClass.new(part_1: "a", part_2: "b") obj.method(:my_method) => #<Method: MyClass#some_method(arg, part_1:) (irb):19> FEATURE: Method#inspect improvements
80. FEATURE: IRB Improvements
81. FEATURE: REPL Improvements
82. FEATURE: Manual GC Compaction
83. GC.compact FEATURE: Method#inspect improvements
84. a Making Ruby (Even) Better
85. If you don't like how Ruby works, change it.
86. UPSTREAM FIX: Warning categories
87. Warning[:deprecated] = false UPSTREAM FIX: Warning categories
88. UPSTREAM FIX: Warning categories
89. module Warning def self.warn(message, category: nil) if category == :deprecated raise message else super end end end UPSTREAM FIX: Warning categories
90. a Why You Should Upgrade (like yesterday)
91. a Nothing makes an upgrade harder than waiting
92. Thank You! Eileen M. Uchitelle Principal Engineer at GitHub Rails Core Team Find me: @eileencodes