Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Upgrading GitHub from Ruby 2.6 to 2.7

Upgrading GitHub from Ruby 2.6 to 2.7

It's no secret that the upgrade to Ruby 2.7 is difficult — fixing the keyword argument, URI, and other deprecation warnings can feel overwhelming, tedious, and never ending. We experienced this first-hand at GitHub; we fixed over 11k+ warnings, sent patches to 15+ gems, upgraded 30+ gems, and replaced abandoned gems. In this talk we’ll look at our custom monkey patch for capturing warnings, how we divided work among teams, and the keys to a successful Ruby 2.7 upgrade. We’ll explore why upgrading is important and take a dive into Ruby 2.7’s notable performance improvements.

C44e1f7e22c3f23cff7bc130871047ef?s=128

Eileen M. Uchitelle

November 19, 2020
Tweet

Transcript

  1. UPGRADING GITHUB from Ruby 2.6 to 2.7 a Eileen M.

    Uchitelle | @eileencodes
  2. Hello! I’m Eileen M. Uchitelle Principal Engineer at GitHub Rails

    Core Team Find me: @eileencodes
  3. UPGRADING GITHUB from Ruby 2.6 to 2.7 Eileen M Uchitelle

    | @eileencodes a
  4. GitHub is born 2007 2008 2009 Forked Rails & Ruby

    Public launch
  5. a

  6. a Why was this Ruby upgrade hard?

  7. a Running GitHub Deprecation Free

  8. a

  9. a Over 11k deprecation warnings!

  10. DEPRECATION: Separation of positional and keyword arguments

  11. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    DEPRECATION: Separation of arguments
  12. MyClass.new({ part_1: "I work fine", part_2: "in Ruby 2.6" })

    => "I work fine in Ruby 2.6" DEPRECATION: Separation of arguments
  13. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) =>

    "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here DEPRECATION: Separation of arguments
  14. DEPRECATION: Separation of arguments MyClass.new({ part_1: "I throw warnings", part_2:

    "in 2.7" }) => "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here The Caller
  15. DEPRECATION: Separation of arguments MyClass.new({ part_1: "I throw warnings", part_2:

    "in 2.7" }) => "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here The Definition
  16. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) MyClass.new(

    part_1: "I don't throw warnings", part_2: "in 2.7" ) => "I don't throw warnings in 2.7" DEPRECATION: Separation of arguments
  17. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) MyClass.new(**{

    part_1: "I don't throw warnings", part_2: "in 2.7" }) => "I don't throw warnings in 2.7" DEPRECATION: Separation of arguments
  18. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    class AbstractClass def initialize(*args) MyClass.new(*args) end end DEPRECATION: Separation of arguments
  19. AbstractClass.new( part_1: "I throw warnings", part_2: "in 2.7" ) DEPRECATION:

    Separation of arguments
  20. AbstractClass.new( part_1: "I throw warnings", part_2: "in 2.7" ) example.rb:9:

    warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here DEPRECATION: Separation of arguments
  21. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    class AbstractClass def initialize(**kwargs) MyClass.new(**kwargs) end end DEPRECATION: Separation of arguments
  22. class MyClassJob def perform(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    DEPRECATION: Separation of arguments
  23. module ActiveJob module Enqueuing def perform_later(*args) job_or_instantiate(*args).enqueue end end end

    DEPRECATION: Separation of arguments
  24. module ActiveJob module Enqueuing def perform_later(*args) job_or_instantiate(*args).enqueue end ruby2_keywords(:perform_later) if

    respond_to?(:ruby2_keywords, true) end end DEPRECATION: Separation of arguments
  25. MyClassJob.perform_later({ part_1: "I am a", part_2: "job with kwargs" })

    activejob/lib/active_job/execution.rb:48: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call app/jobs/my_job_class_job.rb:2: warning: The called method `perform' is defined here DEPRECATION: Separation of arguments
  26. DEPRECATION: URI method deprecations

  27. DEPRECATION: URI Methods URI.encode URI.decode URI.escape URI.unescape

  28. URI.escape("https://google.com?query_param=1") => "https://google.com?query_param=1" uri_example.rb:1: warning: URI.escape is obsolete DEPRECATION: URI

    Methods
  29. URI.escape("https://google.com?query_param=1") Addressable::URI.escape("https://google.com? query_param=1") => "https://google.com?query_param=1" DEPRECATION: URI Methods

  30. URI.escape("https://goog\nle.com?query_param=1") => "https://goog%0Ale.com?query_param=1" Addressable::URI.escape("https://goog\nle.com? query_param=1") => Addressable::URI::InvalidURIError (Invalid character in

    host: 'goog) le.com' DEPRECATION: URI Methods
  31. URI.escape("https://goog\nle.com?query_param=1") => "https://goog%0Ale.com?query_param=1" CGI.escape("https://goog\nle.com?query_param=1") => "https%3A%2F%2Fgoog%0Ale.com%3Fquery_param%3D1" DEPRECATION: URI Methods

  32. a How to Fix More Than 11k Deprecations

  33. a UPGRADING: Dual-booting our application

  34. # config/ruby-version RUBY_NEXT_SHA = "d1ba554" RUBY_SHA = "dcc231c" if ENV["RUBY_NEXT"]

    print RUBY_NEXT_SHA else print RUBY_SHA end UPGRADING: Dual-booting
  35. RUBY_NEXT=1 bin/rails server RUBY_NEXT=1 bin/rails console RUBY_NEXT=1 bin/rails test path/to/test_file.rb

    UPGRADING: Dual-booting
  36. UPGRADING: Dual-booting

  37. UPGRADING: Dual-booting

  38. a UPGRADING: Monkey patch Warning module

  39. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  40. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  41. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  42. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  43. activejob/lib/active_job/execution.rb:48: warning: Using the last argument as keyword parameters is

    deprecated; maybe ** should be added to the call activejob/lib/active_job/execution.rb:48:in `block in perform_now' ... test/integration/a_test.rb:6:in `block in test_something' activesupport/lib/active_support/testing/assertions.rb:34:in `assert_nothing_raised' activejob/lib/active_job/test_helper.rb:591:in `perform_enqueued_jobs' test/integration/my_class_job_test.rb:5:in `test_my_class_job' ... app/jobs/my_job_class_job.rb:2: warning: The called method `perform' is defined here UPGRADING: Warning monkey patch
  44. def test_my_class_job perform_enqueued_jobs do MyClassJob.perform_later({ part_1: "I am a", part_2:

    "job with kwargs" }) end ... end UPGRADING: Warning monkey patch
  45. module Warning def self.warn(message) line = caller_locations.find do |location| location.path.end_with?("_test.rb")

    end WarningsCollector.instance << [message.chomp, line.path] STDERR.print(message) ... UPGRADING: Warning monkey patch
  46. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, origin| f.puts [message, origin].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  47. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  48. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  49. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  50. warnings = {} Dir["tmp/warning*.txt"].each do |file| File.read(file).split("\n").each do |filepath| message,

    filepath = line.split("*^.^*") warnings[message] ||= Message.new(message) warnings[message].paths << filepath if filepath end end warnings.values.each do |warning| warning.owner ||= CODEOWNERS.for(source_file).keys.first.to_s end UPGRADING: Warning monkey patch
  51. - [ ] `test/lib/platform/mutations/ update_mobile_push_notification_schedules_test.rb` - **warnings** - Line 118:

    warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - [ ] `test/test_helpers/newsies/deliver_notifications_job_test_helper.rb` - **warnings** - Line 1282: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - Line 1283: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - **test suites that trigger these warnings** UPGRADING: Warning File Generation
  52. UPGRADING: Warning monkey patch

  53. UPGRADING: Warning File Generation

  54. a UPGRADING: Fixing warnings in gems

  55. None
  56. a UPGRADING Replace abandoned gems

  57. a UPGRADING Preventing regressions

  58. a Our Deploy Process

  59. a DEPLOYING: Make smaller change sets

  60. a DEPLOYING: Testing on staging

  61. a

  62. a DEPLOYING: Slow & incremental rollout

  63. 2% Incremental rollout

  64. 2% Incremental rollout

  65. 2% 0% Incremental rollout

  66. 2% 0% 2% Incremental rollout

  67. 2% 0% 2% 30% Incremental rollout

  68. 2% 0% 2% 30% 60% Incremental rollout

  69. 2% 0% 2% 30% 60% 30% Incremental rollout

  70. 2% 0% 2% 30% 60% 30% 100% Incremental rollout

  71. a DEPLOYING: Strive for boring deploys

  72. a Features Worth Upgrading For

  73. FEATURE: Performance improvements

  74. FEATURES: Boot-time decrease

  75. FEATURES: Boot-time decrease

  76. FEATURES: Allocations decrease

  77. FEATURE: Method#inspect improvements

  78. class MyClass ... def my_method(arg, part_1:) end end obj =

    MyClass.new(part_1: "a", part_2: "b") obj.method(:my_method) => #<Method: MyClass#my_method> FEATURE: Method#inspect improvements
  79. class MyClass ... def my_method(arg, part_1:) end end obj =

    MyClass.new(part_1: "a", part_2: "b") obj.method(:my_method) => #<Method: MyClass#some_method(arg, part_1:) (irb):19> FEATURE: Method#inspect improvements
  80. FEATURE: IRB Improvements

  81. FEATURE: REPL Improvements

  82. FEATURE: Manual GC Compaction

  83. GC.compact FEATURE: Method#inspect improvements

  84. a Making Ruby (Even) Better

  85. If you don't like how Ruby works, change it.

  86. UPSTREAM FIX: Warning categories

  87. Warning[:deprecated] = false UPSTREAM FIX: Warning categories

  88. UPSTREAM FIX: Warning categories

  89. module Warning def self.warn(message, category: nil) if category == :deprecated

    raise message else super end end end UPSTREAM FIX: Warning categories
  90. a Why You Should Upgrade (like yesterday)

  91. a Nothing makes an upgrade harder than waiting

  92. Thank You! Eileen M. Uchitelle Principal Engineer at GitHub Rails

    Core Team Find me: @eileencodes