Upgrading a big application to Rails 5

In this talk we would take a look in different strategies to upgrade Rails application to the newest version taking as example a huge monolithic Rails application. We will learn what were the biggest challenges and how they could be avoided. We will also learn why the changes were made in Rails and how they work.

Transcript

1. Upgrading a big application to Rails 5
3. Rafael França rafaelfranca rafaelfranca
4. Core Member
7. Maintainer
9. Why look at Shopify?
10. Started around the same time as Rails
12. Never rewritten
13. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
14. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
15. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
16. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
17. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
18. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
19. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
20. 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 1.0 2.0 2.1 2.3 3.0 3.1 3.2 4.0 4.1 4.2 5.0 5.1 2.2 Initial commit Beginning of history 1.1 1.2 2.0 2.1 2.2 2.3 3.0 3.2 4.0 4.1 4.2 5.0 Rails Shopify
21. It is a big application
28. It is in the latest Rails version*
29. Upgrading to Rails 5
30. Long-running branch strategy 1. Create a branch 2. Make all changes necessary 3. Merge it 4. ?????? 5. Profit!
32. Dual boot strategy
33. $ BUNDLE_GEMFILE=Gemfile.next bundle install $ BUNDLE_GEMFILE=Gemfile bundle install
34. Hack to share the same Gemfile
35. if ENV['RAILS_NEXT'] # monkey patching to support dual booting module Bundler::SharedHelpers def default_lockfile=(path) @default_lockfile = path end def default_lockfile @default_lockfile ||= Pathname.new("#{default_gemfile}.lock") end end Bundler::SharedHelpers.default_lockfile = Pathname.new("#{Bundler::SharedHelpers.default_gemfile}_next.lock") # Bundler::Dsl.evaluate already called with an incorrect lockfile ... fix it class Bundler::Dsl # A bit messy, this can be called multiple times by bundler, avoid blowing the stack unless self.method_defined? :to_definition_unpatched alias_method :to_definition_unpatched, :to_definition end def to_definition(bad_lockfile, unlock) to_definition_unpatched(Bundler::SharedHelpers.default_lockfile, unlock) end end end if ENV['RAILS_NEXT'] gem 'rails', github: 'rails/rails', branch: '5-0-stable' else gem 'rails', '~> 4.2.7' end
36. if ENV['RAILS_NEXT'] gem 'rails', github: 'rails/rails', branch: '5-0-stable' else gem 'rails', '~> 4.2.7' end
37. $ RAILS_NEXT=1 bundle install $ bundle install
38. Better way to share the same Gemfile
39. Gemfile source 'https://rubygems.org' gem 'rails', '~> 5.0.0' eval_gemfile("Gemfile.shared")
40. Gemfile.next source 'https://rubygems.org' gem 'rails', '~> 5.1.0.rc2' eval_gemfile("Gemfile.shared")
41. Gemfile.shared gem 'sqlite3' gem 'puma', '~> 3.7' gem 'sass-rails', '~> 5.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.2' gem 'turbolinks', '~> 5' gem 'jbuilder', '~> 2.5'
42. config/boot.rb if ENV['RAILS_NEXT'] ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile.next', __dir__) else ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile', __dir__) end require 'bundler/setup' # Set up gems listed in the Gemfile.
43. $ BUNDLE_GEMFILE=Gemfile.next bundle install $ BUNDLE_GEMFILE=Gemfile bundle install
45. Upgrade our dependencies
46. Contribute to the dependencies that don’t support the latest Rails version
47. https://github.com/rails/actionpack-xml_parser/pull/15
48. require 'active_support/core_ext/hash/conversions' require 'action_dispatch/http/request' require 'active_support/core_ext/hash/indifferent_access' module ActionDispatch class XmlParamsParser def initialize(app) @app = app end def call(env) if params = parse_formatted_parameters(env) env["action_dispatch.request.request_parameters"] = params end @app.call(env) end private def parse_formatted_parameters(env) request = Request.new(env) return false if request.content_length.zero? mime_type = content_type_from_legacy_post_data_format_header(env) || request.content_mime_type if mime_type == Mime[:xml] # Rails 5 removed #deep_munge and replaced it with #normalize_encode_params munger = defined?(Request::Utils) ? Request::Utils : request
49. if mime_type == Mime[:xml] # Rails 5 removed #deep_munge and replaced it with #normalize_encode_params munger = defined?(Request::Utils) ? Request::Utils : request params = Hash.from_xml(request.body.read) || {} data = munger.normalize_encode_params(params) request.body.rewind if request.body.respond_to?(:rewind) data.with_indifferent_access else false end rescue Exception # XML code block errors logger(env).debug "Error occurred while parsing request parameters.\nContents: \n\n#{request.raw_post}" raise ActionDispatch::ParamsParser::ParseError end def content_type_from_legacy_post_data_format_header(env) if env['HTTP_X_POST_DATA_FORMAT'].to_s.downcase == 'xml' Mime[:xml] end end def logger(env) env['action_dispatch.logger'] || ActiveSupport::Logger.new($stderr) end end end
50. require 'active_support' require 'active_support/core_ext/hash/conversions' require 'action_dispatch' require 'action_dispatch/http/request' module ActionPack class XmlParser def self.register original_parsers = ActionDispatch::Request.parameter_parsers ActionDispatch::Request.parameter_parsers = original_parsers.merge(Mime[:xml].symbol => self) end def self.call(raw_post) Hash.from_xml(raw_post) || {} end end end
51. require 'active_support' require 'active_support/core_ext/hash/conversions' require 'action_dispatch' require 'action_dispatch/http/request' module ActionPack class XmlParser def self.register original_parsers = ActionDispatch::Request.parameter_parsers ActionDispatch::Request.parameter_parsers = original_parsers.merge(Mime[:xml].symbol => self) end def self.call(raw_post) Hash.from_xml(raw_post) || {} end end end
52. Fix all the tests
55. Bigest chalenges
56. protected_attributes
57. class User < ApplicationRecord attribute :name, :string attribute :password, :string attribute :admin, :boolean end
58. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if user.update_attributes(params[:user]) redirect_to root_path else render 'edit' end end end
59. <%# app/views/admin/users/edit.html.erb %> <%= form_for @user do |f| %> <%= f.text_field :name %> <%= f.password_field :password %> <%= f.check_box :admin %> <% end %>
60. class User < ApplicationRecord attribute :name, :string attribute :password, :string attribute :admin, :boolean attr_accessible :name, :password, :admin end
61. class UsersController < ApplicationController def update @user = current_user if user.update_attributes(params[:user]) redirect_to root_path else render 'edit' end end end
62. <%# app/views/users/edit.html.erb %> <%= form_for @user do |f| %> <%= f.text_field :name %> <%= f.password_field :password %> <% end %>
64. put '/ssh_keys', key: hacker_ssh_key, user_id: dhh_user_id
65. class User < ApplicationRecord attribute :name, :string attribute :password, :string attribute :admin, :boolean attr_accessible :name, :password, :admin, as: :admin attr_accessible :name, :password end
66. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if user.update_attributes(params[:user], as: :admin) redirect_to root_path else render 'edit' end end end
67. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if user.update_attributes(params[:user], as: :admin) redirect_to root_path else render 'edit' end end end
68. Strong Parameters
69. class Admin::UsersController < ApplicationController def update @user = User.find(params[:id]) if user.update_attributes(user_params) redirect_to root_path else render 'edit' end end private def user_params params.require(:user). permit(:name, :password, :admin) end end
71. Missing abstractions?
72. class MyFormObject self.schema = Dry::Validation.Form do optional(:attributes).maybe(:hash?) optional(:remember_me).maybe(:bool?) optional(:email).maybe(:str?) optional(:note).maybe(:str?) end end
73. Controller Tests
74. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post: { title: 'Title', tags: [] } end end
75. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post: { title: 'Title', tags: [] } # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title' } end end
76. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post: { title: 'Title', tags: [] } # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title' } end end
77. post :create, post: { title: 'Title', tags: [] } # => Not the actual implementation @request = TestRequest.new @request.params = { post: { title: 'Title', tags: [] } } controller = UsersController.new controller.request = request controller.create
78. post :create, post: { title: 'Title', tags: [] } # => Not the actual implementation @request = TestRequest.new @request.params = TestRequest.encode_params( post: { title: 'Title', tags: [] } ) controller = UsersController.new controller.request = request controller.create
79. post :create, post: { title: 'Title', tags: [] } # => @request = TestRequest.new @request.params = TestRequest.encode_params( post: { title: 'Title', tags: [] } ) controller = UsersController.new controller.request = request controller.create
81. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post: { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
82. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post: { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
83. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post: { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
84. class UsersControllerTest < ActionController::TestCase test "#create" do post :create, post: { title: 'Title', tags: [] }, as: :json # Rails 4.2 # => "post" => { "title" => 'Title', "tags" => [] } # Rails 5.0 # => "post" => { "title" => 'Title', "tags" => [] } end end
85. Parameters
86. Rails 4 module ActionController class Parameters < Hash end end
87. Rails 5 module ActionController class Parameters end end
88. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
89. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
90. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
91. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => {} params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
92. class MyModel def self.do_someting(params) if params.is_a?(Hash) # Do Something end end end
93. class MyModel def self.do_someting(params) if params.is_a?(Hash) # Do Something end end end
94. class MyController def index MyModel.do_something(my_params.to_h) end private def my_params params.permit(:name, :age) end end
95. class MyController def index MyModel.do_something(my_params.to_h) end private def my_params params.permit(:name, :age) end end
96. class MyModel def self.do_someting(params) if params.is_a?(Hash) || params.is_a?(ActionController::Parameters) # Do Something end end end
97. class MyModel def self.do_someting(params) if params.is_a?(Hash) || params.is_a?(ActionController::Parameters) # Do Something end end end
98. Not an ideal solution
100. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
101. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
102. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
103. params = ActionController::Parameters.new(name: 'Rafael') params.to_h # => ActionController::UnfilteredParameters params.to_unsafe_h # => { 'name' => 'Rafael' } params.permit(:name).to_h # => { 'name' => 'Rafael' }
104. Road to production
105. Compatibilities Layers
107. https://github.com/rails/rails/pull/26017
108. Gradual rollout
110. Benchmarks
114. December 23th
115. Cleanup
116. Remove conditionals
117. Remove deprecations
118. Upgrade configurations to match new values
119. Preparation to the next upgrade
120. Future
121. Avoid monkey patches
122. Keep dependencies number small
123. Keep the parallel CI
124. Keep tracking master
125. Make everyone's concern
126. Think about backwards compatibility
127. Give back to the community
128. ❤
129. Rafael França rafaelfranca rafaelfranca Thank you