Upgrading a big application to Rails 5

Upgrading a big application to Rails 5

In this talk we would take a look in different strategies to upgrade Rails application to the newest version taking as example a huge monolithic Rails application. We will learn what were the biggest challenges and how they could be avoided. We will also learn why the changes were made in Rails and how they work.

0525b332aafb83307b32d9747a93de03?s=128

Rafael França

April 26, 2017
Tweet

Transcript

  1. Upgrading a big application to Rails 5

  2. 5

  3. Rafael França rafaelfranca rafaelfranca

  4. Core Member

  5. None
  6. None
  7. Maintainer

  8. 5

  9. Why look at Shopify?

  10. Started around the same time as Rails

  11. None
  12. Never rewritten

  13. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  14. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  15. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  16. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  17. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  18. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  19. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  20. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

    2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
  21. It is a big application

  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. It is in the latest Rails version*

  29. Upgrading to Rails 5

  30. Long-running branch strategy 1. Create a branch 2. Make all

    changes necessary 3. Merge it 4. ?????? 5. Profit!
  31. None
  32. Dual boot strategy

  33. $ BUNDLE_GEMFILE=Gemfile.next bundle install $ BUNDLE_GEMFILE=Gemfile bundle install

  34. Hack to share the same Gemfile

  35. if ENV['RAILS_NEXT'] # monkey patching to support dual booting module

    Bundler::SharedHelpers def default_lockfile=(path) @default_lockfile = path end def default_lockfile @default_lockfile ||= Pathname.new("#{default_gemfile}.lock") end end Bundler::SharedHelpers.default_lockfile = Pathname.new("#{Bundler::SharedHelpers.default_gemfile}_next.lock") # Bundler::Dsl.evaluate already called with an incorrect lockfile ... fix it class Bundler::Dsl # A bit messy, this can be called multiple times by bundler, avoid blowing the stack unless self.method_defined? :to_definition_unpatched alias_method :to_definition_unpatched, :to_definition end def to_definition(bad_lockfile, unlock) to_definition_unpatched(Bundler::SharedHelpers.default_lockfile, unlock) end end end if ENV['RAILS_NEXT'] gem 'rails', github: 'rails/rails', branch: '5-0-stable' else gem 'rails', '~> 4.2.7' end
  36. if ENV['RAILS_NEXT'] gem 'rails', github: 'rails/rails', branch: '5-0-stable' else gem

    'rails', '~> 4.2.7' end
  37. $ RAILS_NEXT=1 bundle install $ bundle install

  38. Better way to share the same Gemfile

  39. Gemfile source 'https://rubygems.org' gem 'rails', '~> 5.0.0' eval_gemfile("Gemfile.shared")

  40. Gemfile.next source 'https://rubygems.org' gem 'rails', '~> 5.1.0.rc2' eval_gemfile("Gemfile.shared")

  41. Gemfile.shared gem 'sqlite3' gem 'puma', '~> 3.7' gem 'sass-rails', '~>

    5.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.2' gem 'turbolinks', '~> 5' gem 'jbuilder', '~> 2.5'
  42. config/boot.rb if ENV['RAILS_NEXT'] ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile.next', __dir__) else ENV['BUNDLE_GEMFILE'] ||=

    File.expand_path('../Gemfile', __dir__) end require 'bundler/setup' # Set up gems listed in the Gemfile.
  43. $ BUNDLE_GEMFILE=Gemfile.next bundle install $ BUNDLE_GEMFILE=Gemfile bundle install

  44. None
  45. Upgrade our dependencies

  46. Contribute to the dependencies that don’t support the latest Rails

    version
  47. https://github.com/rails/actionpack-xml_parser/pull/15

  48. require 'active_support/core_ext/hash/conversions' require 'action_dispatch/http/request' require 'active_support/core_ext/hash/indifferent_access' module ActionDispatch class XmlParamsParser

    def initialize(app) @app = app end def call(env) if params = parse_formatted_parameters(env) env["action_dispatch.request.request_parameters"] = params end @app.call(env) end private def parse_formatted_parameters(env) request = Request.new(env) return false if request.content_length.zero? mime_type = content_type_from_legacy_post_data_format_header(env) || request.content_mime_type if mime_type == Mime[:xml] # Rails 5 removed #deep_munge and replaced it with #normalize_encode_params munger = defined?(Request::Utils) ? Request::Utils : request
  49. if mime_type == Mime[:xml] # Rails 5 removed #deep_munge and

    replaced it with #normalize_encode_params munger = defined?(Request::Utils) ? Request::Utils : request params = Hash.from_xml(request.body.read) || {} data = munger.normalize_encode_params(params) request.body.rewind if request.body.respond_to?(:rewind) data.with_indifferent_access else false end rescue Exception # XML code block errors logger(env).debug "Error occurred while parsing request parameters.\nContents: \n\n#{request.raw_post}" raise ActionDispatch::ParamsParser::ParseError end def content_type_from_legacy_post_data_format_header(env) if env['HTTP_X_POST_DATA_FORMAT'].to_s.downcase == 'xml' Mime[:xml] end end def logger(env) env['action_dispatch.logger'] || ActiveSupport::Logger.new($stderr) end end end
  50. require 'active_support' require 'active_support/core_ext/hash/conversions' require 'action_dispatch' require 'action_dispatch/http/request' module ActionPack

    class XmlParser def self.register original_parsers = ActionDispatch::Request.parameter_parsers ActionDispatch::Request.parameter_parsers = original_parsers.merge(Mime[:xml].symbol => self) end def self.call(raw_post) Hash.from_xml(raw_post) || {} end end end
  51. require 'active_support' require 'active_support/core_ext/hash/conversions' require 'action_dispatch' require 'action_dispatch/http/request' module ActionPack

    class XmlParser def self.register original_parsers = ActionDispatch::Request.parameter_parsers ActionDispatch::Request.parameter_parsers = original_parsers.merge(Mime[:xml].symbol => self) end def self.call(raw_post) Hash.from_xml(raw_post) || {} end end end
  52. Fix all the tests

  53. None
  54. None
  55. Bigest chalenges

  56. protected_attributes

  57. class User < ApplicationRecord attribute :name, :string attribute :password, :string

    attribute :admin, :boolean end
  58. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if

    user.update_attributes(params[:user]) redirect_to root_path else render 'edit' end end end
  59. <%# app/views/admin/users/edit.html.erb %> <%= form_for @user do |f| %> <%=

    f.text_field :name %> <%= f.password_field :password %> <%= f.check_box :admin %> <% end %>
  60. class User < ApplicationRecord attribute :name, :string attribute :password, :string

    attribute :admin, :boolean attr_accessible :name, :password, :admin end
  61. class UsersController < ApplicationController def update @user = current_user if

    user.update_attributes(params[:user]) redirect_to root_path else render 'edit' end end end
  62. <%# app/views/users/edit.html.erb %> <%= form_for @user do |f| %> <%=

    f.text_field :name %> <%= f.password_field :password %> <% end %>
  63. None
  64. put '/ssh_keys', key: hacker_ssh_key, user_id: dhh_user_id

  65. class User < ApplicationRecord attribute :name, :string attribute :password, :string

    attribute :admin, :boolean attr_accessible :name, :password, :admin, as: :admin attr_accessible :name, :password end
  66. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if

    user.update_attributes(params[:user], as: :admin) redirect_to root_path else render 'edit' end end end
  67. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if

    user.update_attributes(params[:user], as: :admin) redirect_to root_path else render 'edit' end end end
  68. Strong Parameters

  69. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if

    user.update_attributes(user_params) redirect_to root_path else render 'edit' end end private def user_params params.require(:user). permit(:name, :password, :admin) end end
  70. None
  71. Missing abstractions?

  72. class MyFormObject self.schema = Dry::Validation.Form do optional(:attributes).maybe(:hash?) optional(:remember_me).maybe(:bool?) optional(:email).maybe(:str?) optional(:note).maybe(:str?)

    end end
  73. Controller Tests

  74. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post:

    { title: 'Title', tags: [] } end end
  75. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post:

    { title: 'Title', tags: [] } # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title' } end end
  76. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post:

    { title: 'Title', tags: [] } # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title' } end end
  77. post :create, post: { title: 'Title', tags: [] } #

    => Not the actual implementation @request = TestRequest.new @request.params = { post: { title: 'Title', tags: [] } } controller = UsersController.new controller.request = request controller.create
  78. post :create, post: { title: 'Title', tags: [] } #

    => Not the actual implementation @request = TestRequest.new @request.params = TestRequest.encode_params( post: { title: 'Title', tags: [] } ) controller = UsersController.new controller.request = request controller.create
  79. post :create, post: { title: 'Title', tags: [] } #

    => @request = TestRequest.new @request.params = TestRequest.encode_params( post: { title: 'Title', tags: [] } ) controller = UsersController.new controller.request = request controller.create
  80. None
  81. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post:

    { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
  82. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post:

    { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
  83. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post:

    { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
  84. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post:

    { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
  85. Parameters

  86. Rails 4 module ActionController class Parameters < Hash end end

  87. Rails 5 module ActionController class Parameters end end

  88. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  89. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  90. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  91. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  92. class MyModel def self.do_someting(params) if params.is_a?(Hash) # Do Something end

    end end
  93. class MyModel def self.do_someting(params) if params.is_a?(Hash) # Do Something end

    end end
  94. class MyController def index MyModel.do_something(my_params.to_h) end private def my_params params.permit(:name,

    :age) end end
  95. class MyController def index MyModel.do_something(my_params.to_h) end private def my_params params.permit(:name,

    :age) end end
  96. class MyModel def self.do_someting(params) if params.is_a?(Hash) || params.is_a?(ActionController::Parameters) # Do

    Something end end end
  97. class MyModel def self.do_someting(params) if params.is_a?(Hash) || params.is_a?(ActionController::Parameters) # Do

    Something end end end
  98. Not an ideal solution

  99. None
  100. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  101. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  102. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  103. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h #

    => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
  104. Road to production

  105. Compatibilities Layers

  106. None
  107. https://github.com/rails/rails/pull/26017

  108. Gradual rollout

  109. None
  110. Benchmarks

  111. None
  112. None
  113. None
  114. December 23th

  115. Cleanup

  116. Remove conditionals

  117. Remove deprecations

  118. Upgrade configurations to match new values

  119. Preparation to the next upgrade

  120. Future

  121. Avoid monkey patches

  122. Keep dependencies number small

  123. Keep the parallel CI

  124. Keep tracking master

  125. Make everyone's concern

  126. Think about backwards compatibility

  127. Give back to the community

  128. Rafael França rafaelfranca rafaelfranca Thank you