Ruby Nation 2017: Building the New Rails System Test Framework

At the 2014 RailsConf DHH declared system testing would be added to Rails. Three years later, Rails 5.1 makes good on that promise by introducing a new testing framework: ActionDispatch::SystemTestCase. The feature brings system testing to Rails with zero application configuration by adding Capybara integration. After a demonstration of the new framework, we'll walk through what's uniquely involved with building OSS features & how the architecture follows the Rails Doctrine. We'll take a rare look at what it takes to build a major feature for Rails, including goals, design decisions, & roadblocks.

Transcript

1. module ActionDispatch
   class SystemTestCase < IntegrationTest
   include Capybara::DSL
   include Capybara::Minitest::Assertions
   include SystemTesting::TestHelpers::SetupAndTeardown
   include SystemTesting::TestHelpers::ScreenshotHelper
   def initialize(*) # :nodoc:
   super
   self.class.superclass.driver.use
   end
   def self.start_application # :nodoc:
   Capybara.app = Rack::Builder.new do
   map "/" do
   run Rails.application
   end
   end
   SystemTesting::Server.new.run
   BUILDING THE NEW RAILS
   System Tests
   

   View Slide

2. module ActionDispatch
   class SystemTestCase < IntegrationTest
   include Capybara::DSL
   include Capybara::Minitest::Assertions
   include SystemTesting::TestHelpers::SetupAndTeardown
   include SystemTesting::TestHelpers::ScreenshotHelper
   def initialize(*) # :nodoc:
   super
   self.class.superclass.driver.use
   end
   def self.start_application # :nodoc:
   Capybara.app = Rack::Builder.new do
   map "/" do
   run Rails.application
   end
   end
   SystemTesting::Server.new.run
   Hi! I’m
   Eileen M.
   Uchitelle
   

   View Slide

3. module ActionDispatch
   class SystemTestCase < IntegrationTest
   include Capybara::DSL
   include Capybara::Minitest::Assertions
   include SystemTesting::TestHelpers::SetupAndTeardown
   include SystemTesting::TestHelpers::ScreenshotHelper
   def initialize(*) # :nodoc:
   super
   self.class.superclass.driver.use
   end
   def self.start_application # :nodoc:
   Capybara.app = Rack::Builder.new do
   map "/" do
   run Rails.application
   end
   end
   SystemTesting::Server.new.run
   I’m a Systems
   Engineer @
   

   View Slide

4. module ActionDispatch
   class SystemTestCase < IntegrationTest
   include Capybara::DSL
   include Capybara::Minitest::Assertions
   include SystemTesting::TestHelpers::SetupAndTeardown
   include SystemTesting::TestHelpers::ScreenshotHelper
   def initialize(*) # :nodoc:
   super
   self.class.superclass.driver.use
   end
   def self.start_application # :nodoc:
   Capybara.app = Rack::Builder.new do
   map "/" do
   run Rails.application
   end
   end
   SystemTesting::Server.new.run
   I’m on the
   Core Team
   

   View Slide

5. module ActionDispatch
   class SystemTestCase < IntegrationTest
   include Capybara::DSL
   include Capybara::Minitest::Assertions
   include SystemTesting::TestHelpers::SetupAndTeardown
   include SystemTesting::TestHelpers::ScreenshotHelper
   def initialize(*) # :nodoc:
   super
   self.class.superclass.driver.use
   end
   def self.start_application # :nodoc:
   Capybara.app = Rack::Builder.new do
   map "/" do
   run Rails.application
   end
   end
   SystemTesting::Server.new.run
   You can find
   me online
   @eileencodes
   

   View Slide

6. module ActionDispatch
   class SystemTestCase < IntegrationTest
   include Capybara::DSL
   include Capybara::Minitest::Assertions
   include SystemTesting::TestHelpers::SetupAndTeardown
   include SystemTesting::TestHelpers::ScreenshotHelper
   def initialize(*) # :nodoc:
   super
   self.class.superclass.driver.use
   end
   def self.start_application # :nodoc:
   Capybara.app = Rack::Builder.new do
   map "/" do
   run Rails.application
   end
   end
   SystemTesting::Server.new.run
   BUILDING THE NEW RAILS
   System Tests
   

   View Slide

7. At the 2014
   RailsConf…
   

   View Slide

8. View Slide

9. IS DEAD
   

   View Slide

10. Today we do [Rails does] nothing
    to encourage full system tests.
    There's no default answer in the
    stack. That's a mistake we’re
    going to fix.
    “
    —DHH in “TDD is dead. Long live testing.”
    http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html
    

    View Slide

11. 3 years later…
    

    View Slide

12. Zomg, Rails 5.1
    has system
    testing!!1!11!
    

    View Slide

13. Wait, what’s a
    system test?
    

    View Slide

14. Tests your application
    as a system
    

    View Slide

15. gem 'capybara', '> 2.13.0'
    gem 'selenium-webdriver'
    Rails 5.1 Gemfile
    

    View Slide

16. application_system_test_case.rb
    require 'test_helper'
    class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
    driven_by :selenium, using: :chrome, screen_size: [1400, 1400]
    end
    

    View Slide

17. require "application_system_test_case"
    class PostsTest < ApplicationSystemTestCase
    test "visiting the index" do
    visit posts_url
    assert_selector "h1", text: "Post"
    end
    end
    

    View Slide

18. rails test:system
    Running System Tests
    

    View Slide

19. module ActionDispatch
    class SystemTestCase < IntegrationTest
    include Capybara::DSL
    include Capybara::Minitest::Assertions
    include SystemTesting::TestHelpers::SetupAndTeardown
    include SystemTesting::TestHelpers::ScreenshotHelper
    def initialize(*) # :nodoc:
    super
    self.class.superclass.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    SystemTesting::Server.new.run
    SYSTEM TESTING
    Why did it take 3 years?
    

    View Slide

20. Integration tests
    were slowww….
    

    View Slide

21. View Slide

22. View Slide

23. Yo, can you
    make my system
    test promise
    come true?
    

    View Slide

24. I had never used
    Capybara
    before
    

    View Slide

25. module ActionDispatch
    class SystemTestCase < IntegrationTest
    include Capybara::DSL
    include Capybara::Minitest::Assertions
    include SystemTesting::TestHelpers::SetupAndTeardown
    include SystemTesting::TestHelpers::ScreenshotHelper
    def initialize(*) # :nodoc:
    super
    self.class.superclass.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    SystemTesting::Server.new.run
    SYSTEM TESTING
    Guiding Principles
    

    View Slide

26. 1. Optimize for programmer happiness
    2. Convention over configuration
    3. The menu is omakase
    4. No one paradigm
    5. Exalt beautiful code
    6. Provide sharp knives
    7. Value integrated systems
    8. Progress over stability
    9. Push up a big tent
    The Rails Doctrine
    

    View Slide

27. Optimize for programmer
    happiness
    

    View Slide

28. require 'test_helper'
    # set the driver
    Capybara.current_driver = :selenium
    # register the driver & browser
    Capybara.register_driver :selenium do |app|
    Capybara::Selenium::Driver.new(app, browser: :chrome).tap do |driver|
    driver.browser.manage.window.size =
    Selenium::WebDriver::Dimension.new(*[1400, 1400])
    end
    end
    # set the server
    Capybara.server = :puma
    Capybara.always_include_port = true
    # register the server
    Capybara.register_server :puma do |app, port, host|
    Rack::Handler::Puma.run(app, Port: port, Threads: "0:1")
    end
    

    View Slide

29. There is zero configuration
    required to use Capybara
    with Rails 5.1
    

    View Slide

30. require 'test_helper'
    class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
    driven_by :selenium, using: :chrome, screen_size: [1400, 1400]
    end
    

    View Slide

31. Value integrated systems
    

    View Slide

32. Rails as a whole now
    addresses system testing
    

    View Slide

33. Progress over stability
    

    View Slide

34. When system tests were
    merged there were a few
    known bugs
    

    View Slide

35. module ActionDispatch
    class SystemTestCase < IntegrationTest
    include Capybara::DSL
    include Capybara::Minitest::Assertions
    include SystemTesting::TestHelpers::SetupAndTeardown
    include SystemTesting::TestHelpers::ScreenshotHelper
    def initialize(*) # :nodoc:
    super
    self.class.superclass.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    SystemTesting::Server.new.run
    SYSTEM TESTING
    Implementation &
    Architecture
    

    View Slide

36. Config defaults:
    Why Selenium?
    

    View Slide

37. Can see the tests running in
    the browser
    

    View Slide

38. Better for beginners learning
    Rails and Capybara
    

    View Slide

39. driven_by :selenium
    Driver Options
    

    View Slide

40. driven_by :poltergeist
    Driver Options
    

    View Slide

41. Config defaults:
    Why Chrome?
    

    View Slide

42. Chrome is widely used
    

    View Slide

43. For awhile, Firefox was
    broken with Selenium
    

    View Slide

44. driven_by :selenium, using: :chrome
    Driver Options
    

    View Slide

45. driven_by :selenium, using: :firefox
    Driver Options
    

    View Slide

46. Driver Options
    driven_by :selenium, using: :firefox

    screen_size: [1400, 1400]
    

    View Slide

47. Driver Options
    driven_by :selenium, using: :firefox

    options: {
    url: "http://chrome:4444/hub"
    }
    

    View Slide

48. Config defaults:
    Failure Screenshots
    
    

    View Slide

49. def after_teardown
    take_failed_screenshot
    Capybara.reset_sessions!
    super
    end
    Driver Options
    

    View Slide

50. test "the page contains users" do
    visit users_path
    take_screenshot
    assert_selector "h1", text: "Arya"
    end
    Take a Screenshot
    

    View Slide

51. Database Cleaner
    not required
    

    View Slide

52. 
    Test transaction
    opens a
    connection
    

    View Slide

53. 
    Puma opens
    a second
    connection
    Test transaction
    opens a
    connection
    

    View Slide

54. The first connection can’t
    see the data inserted on
    the second connection
    
    

    View Slide

55. View Slide

56. I asked for help from
    @tenderlove & @matthewd
    
    

    View Slide

57. 
    Test transaction
    opens a
    connection
    

    View Slide

58. 
    Puma connects
    on the same
    connection
    Test transaction
    opens a
    connection
    

    View Slide

59. Plumbing
    
    

    View Slide

60. # In actionpack/lib/action_dispatch/system_test_case.rb
    module ActionDispatch
    class SystemTestCase < IntegrationTest
    def initialize(*) # :nodoc:
    super
    self.class.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    

    View Slide

61. # In actionpack/lib/action_dispatch/system_test_case.rb
    module ActionDispatch
    class SystemTestCase < IntegrationTest
    def initialize(*) # :nodoc:
    super
    self.class.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    

    View Slide

62. # In actionpack/lib/action_dispatch/system_test_case.rb
    module ActionDispatch
    class SystemTestCase < IntegrationTest
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    SystemTesting::Server.new.run
    end
    […]
    

    View Slide

63. # In actionpack/lib/action_dispatch/system_test_case.rb
    module ActionDispatch
    class SystemTestCase < IntegrationTest
    def self.driven_by(driver, using: :chrome,
    screen_size: [1400, 1400], options: {})
    @driver = SystemTesting::Driver.new(driver,
    using: using, screen_size: screen_size)
    end
    driven_by :selenium
    end
    SystemTestCase.start_application
    end
    

    View Slide

64. # In action_dispatch/system_testing/driver.rb
    module ActionDispatch
    module SystemTesting
    class Driver # :nodoc:
    def initialize(name, **options)
    @name = name
    @browser = options[:using]
    @screen_size = options[:screen_size]
    @options = options[:options]
    end
    def use
    register if selenium?
    setup
    end
    

    View Slide

65. # In actionpack/lib/action_dispatch/system_test_case.rb
    module ActionDispatch
    class SystemTestCase < IntegrationTest
    def initialize(*) # :nodoc:
    super
    self.class.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    

    View Slide

66. # In action_dispatch/system_testing/driver.rb
    module ActionDispatch
    module SystemTesting
    class Driver # :nodoc:
    def use
    register unless rack_test?
    setup
    end
    private
    def rack_test?
    @name == :rack_test
    end
    […]
    

    View Slide

67. # In action_dispatch/system_testing/driver.rb
    def register
    Capybara.register_driver @name do |app|
    case @name
    when :selenium then register_selenium(app)
    when :poltergeist then register_poltergeist(app)
    when :webkit then register_webkit(app)
    end
    end
    end
    

    View Slide

68. # In action_dispatch/system_testing/driver.rb
    def register
    Capybara.register_driver @name do |app|
    Capybara::Selenium::Driver.new(app,
    { browser: @browser }.merge(@options))
    .tap do |driver|
    driver.browser.manage.window.size =
    Selenium::WebDriver::Dimension.new(*@screen_size)
    end
    end
    end
    

    View Slide

69. # In action_dispatch/system_testing/driver.rb
    module ActionDispatch
    module SystemTesting
    class Driver # :nodoc:
    def use
    register if selenium?
    setup
    end
    private
    def selenium?
    @name == :selenium
    end
    […]
    

    View Slide

70. # In action_dispatch/system_testing/driver.rb
    module ActionDispatch
    module SystemTesting
    class Driver # :nodoc:
    private
    def setup
    Capybara.current_driver = @name
    end
    end
    end
    end
    

    View Slide

71. module ActionDispatch
    class SystemTestCase < IntegrationTest
    include Capybara::DSL
    include Capybara::Minitest::Assertions
    include SystemTesting::TestHelpers::SetupAndTeardown
    include SystemTesting::TestHelpers::ScreenshotHelper
    def initialize(*) # :nodoc:
    super
    self.class.superclass.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    SystemTesting::Server.new.run
    BUILDING
    Open Source Features
    

    View Slide

72. Public Work
    

    View Slide

73. Feels like everyone is
    judging you
    

    View Slide

74. View Slide

75. Stakeholders
    

    View Slide

76. View Slide

77. View Slide

78. $%&'
    ()*+
    ,-./
    

    View Slide

79. $%&'
    ()*+
    ,-./
    70
    

    View Slide

80. $%&'
    ()*+
    ,-./
    

    View Slide

81. $%&'
    ()*+
    ,-./
    Perfect?
    

    View Slide

82. Consensus is the enemy
    of vision
    

    View Slide

83. Dealing with multiple
    stakeholders
    

    View Slide

84. Manage expectations
    

    View Slide

85. Know who you are building
    the feature for
    

    View Slide

86. Be open to change
    

    View Slide

87. Open source doesn’t work
    without contributors
    

    View Slide

88. Build a foundation for others
    to work off of
    

    View Slide

89. Thanks @twapole
    

    View Slide

90. Thanks @robin850
    

    View Slide

91. Thanks @mtsmfm &
    @lucasmazza
    

    View Slide

92. Thanks @renchap
    

    View Slide

93. Open source doesn’t work
    without contributors
    

    View Slide

94. Open source doesn’t work
    without you
    

    View Slide

95. module ActionDispatch
    class SystemTestCase < IntegrationTest
    include Capybara::DSL
    include Capybara::Minitest::Assertions
    include SystemTesting::TestHelpers::SetupAndTeardown
    include SystemTesting::TestHelpers::ScreenshotHelper
    def initialize(*) # :nodoc:
    super
    self.class.superclass.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    SystemTesting::Server.new.run
    Thank you
    Ruby Nation!
    

    View Slide

96. module ActionDispatch
    class SystemTestCase < IntegrationTest
    include Capybara::DSL
    include Capybara::Minitest::Assertions
    include SystemTesting::TestHelpers::SetupAndTeardown
    include SystemTesting::TestHelpers::ScreenshotHelper
    def initialize(*) # :nodoc:
    super
    self.class.superclass.driver.use
    end
    def self.start_application # :nodoc:
    Capybara.app = Rack::Builder.new do
    map "/" do
    run Rails.application
    end
    end
    SystemTesting::Server.new.run
    Find me
    everywhere
    @eileencodes
    

    View Slide