require() bombed my multi-threaded app!

Would you like Circular Dependencies with that? In Rails we take autoloading for granted, but slow test suites and evasive multi-thread bombs too often drive us to upbraid ActiveSupport’s autoloading magic.

To dispel the magic, we will cover nothing less than the entirety of require’s modest origins, the enlightening truth of Ruby execution, a selection of minimal autoload examples, the most interesting ways to break integration specs, and utopian isolation practices. All while speeding up your test suite.

https://github.com/nybblr/ruby-code-loading

Transcript

1. require() bombed my
   multi-threaded app!
   

   View Slide

2. jonathan martin
   

   View Slide

3. @nybblr
   

   View Slide

4. Atlanta GA, USA
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. #ruby #autoload
    

    View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. View Slide

18. View Slide

19. #1. Plain ol’ Ruby
    eval, load, & require.
    

    View Slide

20. Cookery.rb
    

    View Slide

21. /home/nybblr/
    !"" cookery
    # !"" ingredient.rb
    # $"" recipe.rb
    !"" cookery.rb
    $"" main.rb
    1 directory, 4 files
    > tree /home/nybblr
    

    View Slide

22. puts to_s
    > main
    self.puts self.to_s
    > main
    > irb
    

    View Slide

23. puts self.class.ancestors
    > [Object, Kernel, BasicObject]
    self.eval "puts to_s"
    > main
    > irb
    

    View Slide

24. class Cook
    def get_binding
    binding
    end
    end
    eval "puts to_s", Cook.new.get_binding
    > #
    > irb
    

    View Slide

25. puts "Loaded Cookery::Recipe class!"
    module Cookery
    class Recipe
    end
    end
    cookery/recipe.rb
    

    View Slide

26. dir = File.expand_path("..", __FILE__)
    file = File.join(dir, "cookery/recipe.rb")
    eval File.read(file), binding, "cookery/recipe.rb", 1
    > Loaded Cookery::Recipe class!
    main.rb
    

    View Slide

27. load File.join(dir, "cookery/recipe.rb")
    > Loaded Cookery::Recipe class!
    main.rb
    

    View Slide

28. $LOAD_PATH.unshift(dir)
    load "cookery/recipe.rb"
    > Loaded Cookery::Recipe class!
    main.rb
    

    View Slide

29. $LOAD_PATH.unshift(dir)
    require "cookery/recipe"
    > Loaded Cookery::Recipe class!
    p $LOADED_FEATURES.last
    > "/home/nybblr/cookery/recipe.rb"
    main.rb
    

    View Slide

30. require File.join(dir, "cookery/recipe.rb")
    > Loaded Cookery::Recipe class!
    require "cookery/recipe"
    > *Eerie silence…*
    main.rb
    

    View Slide

31. # main.rb header from now on!
    require 'active_support'
    ROOT = File.expand_path("..", __FILE__)
    $LOAD_PATH.unshift(ROOT)
    p $LOADED_FEATURES.last
    > ".../active_support/dependencies.rb"
    main.rb
    

    View Slide

32. #2. Autoloading
    with ActiveSupport
    

    View Slide

33. # ActiveSupport::Dependencies.mechanism = :load
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    p $LOADED_FEATURES.last
    > ".../active_support/dependencies.rb"
    require 'cookery/recipe'
    > Loaded Cookery::Recipe class!
    main.rb
    

    View Slide

34. ActiveSupport::Dependencies.mechanism = :require
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    p $LOADED_FEATURES.last
    > "/home/nybblr/cookery/recipe.rb"
    require 'cookery/recipe'
    > *Eerie silence…*
    main.rb
    

    View Slide

35. module Cookery
    extend ActiveSupport::Autoload
    autoload :Recipe
    end
    cookery.rb
    

    View Slide

36. require 'cookery'
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    p $LOADED_FEATURES.last
    > "/home/nybblr/cookery/recipe.rb"
    require 'cookery/recipe'
    > *Eerie silence…*
    main.rb
    

    View Slide

37. #3. Namespaces
    matter to Autoload.
    

    View Slide

38. puts "Loaded ::Recipe class!"
    class Recipe
    end
    cookery/recipe.rb
    

    View Slide

39. ActiveSupport::Dependencies.mechanism = :require
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded ::Recipe class!
    > Unable to autoload constant Cookery::Recipe,
    expected ./cookery/recipe.rb to define it (LoadError)
    main.rb
    

    View Slide

40. ActiveSupport::Dependencies.mechanism = :require
    ActiveSupport::Dependencies.autoload_paths +=
    ["#{ROOT}/cookery", ROOT]
    Recipe
    > Loaded ::Recipe class!
    p $LOADED_FEATURES.last
    > "/home/nybblr/cookery/recipe.rb"
    Cookery::Recipe
    > Unable to autoload constant Cookery::Recipe, expected ./cookery/
    recipe.rb to define it (LoadError)
    main.rb
    

    View Slide

41. #4. Constants
    should not be redefined.
    

    View Slide

42. puts "Loaded Cookery::Recipe class!"
    module Cookery
    class Recipe
    CONSTANT = "constant".freeze
    end
    end
    cookery/recipe.rb
    

    View Slide

43. # ActiveSupport::Dependencies.mechanism = :load
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    p $LOADED_FEATURES.last
    > ".../active_support/dependencies.rb"
    require 'cookery/recipe'
    > Loaded Cookery::Recipe class!
    > already initialized constant Cookery::Recipe::CONSTANT
    main.rb
    

    View Slide

44. ActiveSupport::Dependencies.mechanism = :require
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    p $LOADED_FEATURES.last
    > "/home/nybblr/cookery.rb"
    require 'cookery/recipe'
    > *Eerie silence...*
    main.rb
    

    View Slide

45. #5. Circular Deps
    require + autoload = fists
    

    View Slide

46. puts "Loaded Cookery::Recipe class!"
    module Cookery
    class Recipe
    CONSTANT = Ingredient
    end
    end
    cookery/recipe.rb
    

    View Slide

47. puts "Loaded Cookery::Ingredient class!"
    require 'cookery/recipe'
    module Cookery
    class Ingredient
    CONSTANT = Recipe
    end
    end
    cookery/ingredient.rb
    

    View Slide

48. # ActiveSupport::Dependencies.mechanism = :load
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    > Loaded Cookery::Ingredient class!
    > Loaded Cookery::Recipe class!
    > Circular dependency detected while autoloading
    constant Cookery::Ingredient (RuntimeError)
    main.rb
    

    View Slide

49. ActiveSupport::Dependencies.mechanism = :require
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    > Loaded Cookery::Ingredient class!
    main.rb
    

    View Slide

50. puts "Loaded Cookery::Ingredient class!"
    # require 'cookery/recipe'
    module Cookery
    class Ingredient
    CONSTANT = Recipe
    end
    end
    cookery/ingredient.rb
    

    View Slide

51. # ActiveSupport::Dependencies.mechanism = :load
    ActiveSupport::Dependencies.autoload_paths << ROOT
    Cookery::Recipe
    > Loaded Cookery::Recipe class!
    > Loaded Cookery::Ingredient class!
    main.rb
    

    View Slide

52. #6. eagerload_paths
    !== autoload_paths.
    

    View Slide

53. module Cookery
    class Application < Rails::Application
    config.autoload_paths << "#{config.root}/lib"
    end
    end
    config/application.rb
    

    View Slide

54. module Cookery
    class Application < Rails::Application
    config.eager_load_paths << "#{config.root}/lib"
    end
    end
    config/application.rb
    

    View Slide

55. module Cookery
    class Application < Rails::Application
    config.eager_load_paths +=
    ["#{config.root}/app/models",
    "#{config.root}/app/models/queries"]
    end
    end
    config/application.rb
    

    View Slide

56. #7. Isolated Tests
    use a different spec helper.
    

    View Slide

57. ENV["RAILS_ENV"] ||= 'test'
    if !Object.const_defined?(:Rails)
    require 'active_support/all'
    root = File.expand_path('../..', __FILE__)
    ActiveSupport::Dependencies.autoload_paths += Dir["#{root}/app/*/"]
    end
    # General config
    RSpec.configure do |config|
    config.order = :random
    end
    spec/spec_helper.rb
    

    View Slide

58. require File.expand_path('../../config/environment', __FILE__)
    require 'spec_helper'
    require 'rspec/rails'
    # Rails specific config
    RSpec.configure do |config|
    config.use_transactional_fixtures = true
    config.infer_spec_type_from_file_location!
    end
    spec/rails_helper.rb
    

    View Slide

59. require 'spec_helper'
    describe Cookery::Recipe do
    # That was fast!
    end
    spec/cookery/recipe_spec.rb
    

    View Slide

60. View Slide

61. AS::Dep.mechanism = :require
    

    View Slide

62. autoload + require( ) =
    

    View Slide

63. CONSTANT ||= “string”.freeze
    

    View Slide

64. autoload_paths <=> AS::Autoload
    

    View Slide

65. spec_helper.rb > rails_helper.rb
    

    View Slide

66. require_dependency( ) =
    

    View Slide

67. bit.ly/lol-ruby
    

    View Slide

68. questions?
    domande?
    

    View Slide

69. thanks!
    @nybblr
    

    View Slide