require() bombed my multi-threaded app!

require() bombed my multi-threaded app!

Would you like Circular Dependencies with that? In Rails we take autoloading for granted, but slow test suites and evasive multi-thread bombs too often drive us to upbraid ActiveSupport’s autoloading magic.

To dispel the magic, we will cover nothing less than the entirety of require’s modest origins, the enlightening truth of Ruby execution, a selection of minimal autoload examples, the most interesting ways to break integration specs, and utopian isolation practices. All while speeding up your test suite.

https://github.com/nybblr/ruby-code-loading

27a38e420ceeb97e61f109c4c6a0e9b4?s=128

Jonathan Lee Martin

November 13, 2015
Tweet

Transcript

  1. require() bombed my multi-threaded app!

  2. jonathan martin

  3. @nybblr

  4. Atlanta GA, USA

  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. None
  12. #ruby #autoload

  13. None
  14. None
  15. None
  16. None
  17. None
  18. None
  19. #1. Plain ol’ Ruby eval, load, & require.

  20. Cookery.rb

  21. /home/nybblr/ !"" cookery # !"" ingredient.rb # $"" recipe.rb !""

    cookery.rb $"" main.rb 1 directory, 4 files > tree /home/nybblr
  22. puts to_s > main self.puts self.to_s > main > irb

  23. puts self.class.ancestors > [Object, Kernel, BasicObject] self.eval "puts to_s" >

    main > irb
  24. class Cook def get_binding binding end end eval "puts to_s",

    Cook.new.get_binding > #<Cook:0x007f88918e3110> > irb
  25. puts "Loaded Cookery::Recipe class!" module Cookery class Recipe end end

    cookery/recipe.rb
  26. dir = File.expand_path("..", __FILE__) file = File.join(dir, "cookery/recipe.rb") eval File.read(file),

    binding, "cookery/recipe.rb", 1 > Loaded Cookery::Recipe class! main.rb
  27. load File.join(dir, "cookery/recipe.rb") > Loaded Cookery::Recipe class! main.rb

  28. $LOAD_PATH.unshift(dir) load "cookery/recipe.rb" > Loaded Cookery::Recipe class! main.rb

  29. $LOAD_PATH.unshift(dir) require "cookery/recipe" > Loaded Cookery::Recipe class! p $LOADED_FEATURES.last >

    "/home/nybblr/cookery/recipe.rb" main.rb
  30. require File.join(dir, "cookery/recipe.rb") > Loaded Cookery::Recipe class! require "cookery/recipe" >

    *Eerie silence…* main.rb
  31. # main.rb header from now on! require 'active_support' ROOT =

    File.expand_path("..", __FILE__) $LOAD_PATH.unshift(ROOT) p $LOADED_FEATURES.last > ".../active_support/dependencies.rb" main.rb
  32. #2. Autoloading with ActiveSupport

  33. # ActiveSupport::Dependencies.mechanism = :load ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded

    Cookery::Recipe class! p $LOADED_FEATURES.last > ".../active_support/dependencies.rb" require 'cookery/recipe' > Loaded Cookery::Recipe class! main.rb
  34. ActiveSupport::Dependencies.mechanism = :require ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded Cookery::Recipe

    class! p $LOADED_FEATURES.last > "/home/nybblr/cookery/recipe.rb" require 'cookery/recipe' > *Eerie silence…* main.rb
  35. module Cookery extend ActiveSupport::Autoload autoload :Recipe end cookery.rb

  36. require 'cookery' Cookery::Recipe > Loaded Cookery::Recipe class! p $LOADED_FEATURES.last >

    "/home/nybblr/cookery/recipe.rb" require 'cookery/recipe' > *Eerie silence…* main.rb
  37. #3. Namespaces matter to Autoload.

  38. puts "Loaded ::Recipe class!" class Recipe end cookery/recipe.rb

  39. ActiveSupport::Dependencies.mechanism = :require ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded ::Recipe

    class! > Unable to autoload constant Cookery::Recipe, expected ./cookery/recipe.rb to define it (LoadError) main.rb
  40. ActiveSupport::Dependencies.mechanism = :require ActiveSupport::Dependencies.autoload_paths += ["#{ROOT}/cookery", ROOT] Recipe > Loaded

    ::Recipe class! p $LOADED_FEATURES.last > "/home/nybblr/cookery/recipe.rb" Cookery::Recipe > Unable to autoload constant Cookery::Recipe, expected ./cookery/ recipe.rb to define it (LoadError) main.rb
  41. #4. Constants should not be redefined.

  42. puts "Loaded Cookery::Recipe class!" module Cookery class Recipe CONSTANT =

    "constant".freeze end end cookery/recipe.rb
  43. # ActiveSupport::Dependencies.mechanism = :load ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded

    Cookery::Recipe class! p $LOADED_FEATURES.last > ".../active_support/dependencies.rb" require 'cookery/recipe' > Loaded Cookery::Recipe class! > already initialized constant Cookery::Recipe::CONSTANT main.rb
  44. ActiveSupport::Dependencies.mechanism = :require ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded Cookery::Recipe

    class! p $LOADED_FEATURES.last > "/home/nybblr/cookery.rb" require 'cookery/recipe' > *Eerie silence...* main.rb
  45. #5. Circular Deps require + autoload = fists

  46. puts "Loaded Cookery::Recipe class!" module Cookery class Recipe CONSTANT =

    Ingredient end end cookery/recipe.rb
  47. puts "Loaded Cookery::Ingredient class!" require 'cookery/recipe' module Cookery class Ingredient

    CONSTANT = Recipe end end cookery/ingredient.rb
  48. # ActiveSupport::Dependencies.mechanism = :load ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded

    Cookery::Recipe class! > Loaded Cookery::Ingredient class! > Loaded Cookery::Recipe class! > Circular dependency detected while autoloading constant Cookery::Ingredient (RuntimeError) main.rb
  49. ActiveSupport::Dependencies.mechanism = :require ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded Cookery::Recipe

    class! > Loaded Cookery::Ingredient class! main.rb
  50. puts "Loaded Cookery::Ingredient class!" # require 'cookery/recipe' module Cookery class

    Ingredient CONSTANT = Recipe end end cookery/ingredient.rb
  51. # ActiveSupport::Dependencies.mechanism = :load ActiveSupport::Dependencies.autoload_paths << ROOT Cookery::Recipe > Loaded

    Cookery::Recipe class! > Loaded Cookery::Ingredient class! main.rb
  52. #6. eagerload_paths !== autoload_paths.

  53. module Cookery class Application < Rails::Application config.autoload_paths << "#{config.root}/lib" end

    end config/application.rb
  54. module Cookery class Application < Rails::Application config.eager_load_paths << "#{config.root}/lib" end

    end config/application.rb
  55. module Cookery class Application < Rails::Application config.eager_load_paths += ["#{config.root}/app/models", "#{config.root}/app/models/queries"]

    end end config/application.rb
  56. #7. Isolated Tests use a different spec helper.

  57. ENV["RAILS_ENV"] ||= 'test' if !Object.const_defined?(:Rails) require 'active_support/all' root = File.expand_path('../..',

    __FILE__) ActiveSupport::Dependencies.autoload_paths += Dir["#{root}/app/*/"] end # General config RSpec.configure do |config| config.order = :random end spec/spec_helper.rb
  58. require File.expand_path('../../config/environment', __FILE__) require 'spec_helper' require 'rspec/rails' # Rails specific

    config RSpec.configure do |config| config.use_transactional_fixtures = true config.infer_spec_type_from_file_location! end spec/rails_helper.rb
  59. require 'spec_helper' describe Cookery::Recipe do # That was fast! end

    spec/cookery/recipe_spec.rb
  60. None
  61. AS::Dep.mechanism = :require

  62. autoload + require( ) =

  63. CONSTANT ||= “string”.freeze

  64. autoload_paths <=> AS::Autoload

  65. spec_helper.rb > rails_helper.rb

  66. require_dependency( ) =

  67. bit.ly/lol-ruby

  68. questions? domande?

  69. thanks! @nybblr