Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Upgrading GitHub from Ruby 2.6 to 2.7

Upgrading GitHub from Ruby 2.6 to 2.7

It's no secret that the upgrade to Ruby 2.7 is difficult — fixing the keyword argument, URI, and other deprecation warnings can feel overwhelming, tedious, and never ending. We experienced this first-hand at GitHub; we fixed over 11k+ warnings, sent patches to 15+ gems, upgraded 30+ gems, and replaced abandoned gems. In this talk we’ll look at our custom monkey patch for capturing warnings, how we divided work among teams, and the keys to a successful Ruby 2.7 upgrade. We’ll explore why upgrading is important and take a dive into Ruby 2.7’s notable performance improvements.

Eileen M. Uchitelle

November 19, 2020
Tweet

More Decks by Eileen M. Uchitelle

Other Decks in Programming

Transcript

  1. UPGRADING GITHUB from Ruby 2.6 to 2.7 a Eileen M.

    Uchitelle | @eileencodes
  2. Hello! I’m Eileen M. Uchitelle Principal Engineer at GitHub Rails

    Core Team Find me: @eileencodes
  3. UPGRADING GITHUB from Ruby 2.6 to 2.7 Eileen M Uchitelle

    | @eileencodes a
  4. GitHub is born 2007 2008 2009 Forked Rails & Ruby

    Public launch
  5. a

  6. a Why was this Ruby upgrade hard?

  7. a Running GitHub Deprecation Free

  8. a

  9. a Over 11k deprecation warnings!

  10. DEPRECATION: Separation of positional and keyword arguments

  11. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    DEPRECATION: Separation of arguments
  12. MyClass.new({ part_1: "I work fine", part_2: "in Ruby 2.6" })

    => "I work fine in Ruby 2.6" DEPRECATION: Separation of arguments
  13. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) =>

    "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here DEPRECATION: Separation of arguments
  14. DEPRECATION: Separation of arguments MyClass.new({ part_1: "I throw warnings", part_2:

    "in 2.7" }) => "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here The Caller
  15. DEPRECATION: Separation of arguments MyClass.new({ part_1: "I throw warnings", part_2:

    "in 2.7" }) => "I throw warnings in 2.7" example.rb:13: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here The Definition
  16. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) MyClass.new(

    part_1: "I don't throw warnings", part_2: "in 2.7" ) => "I don't throw warnings in 2.7" DEPRECATION: Separation of arguments
  17. MyClass.new({ part_1: "I throw warnings", part_2: "in 2.7" }) MyClass.new(**{

    part_1: "I don't throw warnings", part_2: "in 2.7" }) => "I don't throw warnings in 2.7" DEPRECATION: Separation of arguments
  18. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    class AbstractClass def initialize(*args) MyClass.new(*args) end end DEPRECATION: Separation of arguments
  19. AbstractClass.new( part_1: "I throw warnings", part_2: "in 2.7" ) DEPRECATION:

    Separation of arguments
  20. AbstractClass.new( part_1: "I throw warnings", part_2: "in 2.7" ) example.rb:9:

    warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call example.rb:2: warning: The called method `initialize' is defined here DEPRECATION: Separation of arguments
  21. class MyClass def initialize(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    class AbstractClass def initialize(**kwargs) MyClass.new(**kwargs) end end DEPRECATION: Separation of arguments
  22. class MyClassJob def perform(part_1:, part_2:) puts "#{part_1} #{part_2}" end end

    DEPRECATION: Separation of arguments
  23. module ActiveJob module Enqueuing def perform_later(*args) job_or_instantiate(*args).enqueue end end end

    DEPRECATION: Separation of arguments
  24. module ActiveJob module Enqueuing def perform_later(*args) job_or_instantiate(*args).enqueue end ruby2_keywords(:perform_later) if

    respond_to?(:ruby2_keywords, true) end end DEPRECATION: Separation of arguments
  25. MyClassJob.perform_later({ part_1: "I am a", part_2: "job with kwargs" })

    activejob/lib/active_job/execution.rb:48: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call app/jobs/my_job_class_job.rb:2: warning: The called method `perform' is defined here DEPRECATION: Separation of arguments
  26. DEPRECATION: URI method deprecations

  27. DEPRECATION: URI Methods URI.encode URI.decode URI.escape URI.unescape

  28. URI.escape("https://google.com?query_param=1") => "https://google.com?query_param=1" uri_example.rb:1: warning: URI.escape is obsolete DEPRECATION: URI

    Methods
  29. URI.escape("https://google.com?query_param=1") Addressable::URI.escape("https://google.com? query_param=1") => "https://google.com?query_param=1" DEPRECATION: URI Methods

  30. URI.escape("https://goog\nle.com?query_param=1") => "https://goog%0Ale.com?query_param=1" Addressable::URI.escape("https://goog\nle.com? query_param=1") => Addressable::URI::InvalidURIError (Invalid character in

    host: 'goog) le.com' DEPRECATION: URI Methods
  31. URI.escape("https://goog\nle.com?query_param=1") => "https://goog%0Ale.com?query_param=1" CGI.escape("https://goog\nle.com?query_param=1") => "https%3A%2F%2Fgoog%0Ale.com%3Fquery_param%3D1" DEPRECATION: URI Methods

  32. a How to Fix More Than 11k Deprecations

  33. a UPGRADING: Dual-booting our application

  34. # config/ruby-version RUBY_NEXT_SHA = "d1ba554" RUBY_SHA = "dcc231c" if ENV["RUBY_NEXT"]

    print RUBY_NEXT_SHA else print RUBY_SHA end UPGRADING: Dual-booting
  35. RUBY_NEXT=1 bin/rails server RUBY_NEXT=1 bin/rails console RUBY_NEXT=1 bin/rails test path/to/test_file.rb

    UPGRADING: Dual-booting
  36. UPGRADING: Dual-booting

  37. UPGRADING: Dual-booting

  38. a UPGRADING: Monkey patch Warning module

  39. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  40. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  41. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  42. module Warning def self.warn(message) STDERR.print(message) if ENV["RAISE_ON_WARNINGS"] raise message end

    if ENV["DEBUG_WARNINGS"] STDERR.puts caller end end end UPGRADING: Warning monkey patch
  43. activejob/lib/active_job/execution.rb:48: warning: Using the last argument as keyword parameters is

    deprecated; maybe ** should be added to the call activejob/lib/active_job/execution.rb:48:in `block in perform_now' ... test/integration/a_test.rb:6:in `block in test_something' activesupport/lib/active_support/testing/assertions.rb:34:in `assert_nothing_raised' activejob/lib/active_job/test_helper.rb:591:in `perform_enqueued_jobs' test/integration/my_class_job_test.rb:5:in `test_my_class_job' ... app/jobs/my_job_class_job.rb:2: warning: The called method `perform' is defined here UPGRADING: Warning monkey patch
  44. def test_my_class_job perform_enqueued_jobs do MyClassJob.perform_later({ part_1: "I am a", part_2:

    "job with kwargs" }) end ... end UPGRADING: Warning monkey patch
  45. module Warning def self.warn(message) line = caller_locations.find do |location| location.path.end_with?("_test.rb")

    end WarningsCollector.instance << [message.chomp, line.path] STDERR.print(message) ... UPGRADING: Warning monkey patch
  46. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, origin| f.puts [message, origin].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  47. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  48. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  49. class WarningsCollector < ParallelCollector def process path = File.join("/tmp", "warnings.txt")

    File.open(path, "a") do |f| @data.each do |message, filepath| f.puts [message, filepath].join("*^.^*") end end script = File.absolute_path( "../../../script/process-warnings", __FILE__) system(script, "/tmp") end end UPGRADING: Warning monkey patch
  50. warnings = {} Dir["tmp/warning*.txt"].each do |file| File.read(file).split("\n").each do |filepath| message,

    filepath = line.split("*^.^*") warnings[message] ||= Message.new(message) warnings[message].paths << filepath if filepath end end warnings.values.each do |warning| warning.owner ||= CODEOWNERS.for(source_file).keys.first.to_s end UPGRADING: Warning monkey patch
  51. - [ ] `test/lib/platform/mutations/ update_mobile_push_notification_schedules_test.rb` - **warnings** - Line 118:

    warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - [ ] `test/test_helpers/newsies/deliver_notifications_job_test_helper.rb` - **warnings** - Line 1282: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - Line 1283: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call - **test suites that trigger these warnings** UPGRADING: Warning File Generation
  52. UPGRADING: Warning monkey patch

  53. UPGRADING: Warning File Generation

  54. a UPGRADING: Fixing warnings in gems

  55. None
  56. a UPGRADING Replace abandoned gems

  57. a UPGRADING Preventing regressions

  58. a Our Deploy Process

  59. a DEPLOYING: Make smaller change sets

  60. a DEPLOYING: Testing on staging

  61. a

  62. a DEPLOYING: Slow & incremental rollout

  63. 2% Incremental rollout

  64. 2% Incremental rollout

  65. 2% 0% Incremental rollout

  66. 2% 0% 2% Incremental rollout

  67. 2% 0% 2% 30% Incremental rollout

  68. 2% 0% 2% 30% 60% Incremental rollout

  69. 2% 0% 2% 30% 60% 30% Incremental rollout

  70. 2% 0% 2% 30% 60% 30% 100% Incremental rollout

  71. a DEPLOYING: Strive for boring deploys

  72. a Features Worth Upgrading For

  73. FEATURE: Performance improvements

  74. FEATURES: Boot-time decrease

  75. FEATURES: Boot-time decrease

  76. FEATURES: Allocations decrease

  77. FEATURE: Method#inspect improvements

  78. class MyClass ... def my_method(arg, part_1:) end end obj =

    MyClass.new(part_1: "a", part_2: "b") obj.method(:my_method) => #<Method: MyClass#my_method> FEATURE: Method#inspect improvements
  79. class MyClass ... def my_method(arg, part_1:) end end obj =

    MyClass.new(part_1: "a", part_2: "b") obj.method(:my_method) => #<Method: MyClass#some_method(arg, part_1:) (irb):19> FEATURE: Method#inspect improvements
  80. FEATURE: IRB Improvements

  81. FEATURE: REPL Improvements

  82. FEATURE: Manual GC Compaction

  83. GC.compact FEATURE: Method#inspect improvements

  84. a Making Ruby (Even) Better

  85. If you don't like how Ruby works, change it.

  86. UPSTREAM FIX: Warning categories

  87. Warning[:deprecated] = false UPSTREAM FIX: Warning categories

  88. UPSTREAM FIX: Warning categories

  89. module Warning def self.warn(message, category: nil) if category == :deprecated

    raise message else super end end end UPSTREAM FIX: Warning categories
  90. a Why You Should Upgrade (like yesterday)

  91. a Nothing makes an upgrade harder than waiting

  92. Thank You! Eileen M. Uchitelle Principal Engineer at GitHub Rails

    Core Team Find me: @eileencodes