Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Embracing Capybara

Tim Moore
October 31, 2013

Embracing Capybara

I’m going to talk about Embracing Capybara: how to understand Capybara, train it to do new tricks, and discipline it when it misbehaves.

[Slide 4]

My name is Tim. I work at Lonely Planet, on a fairly complex single-page app, and we use Capybara for our tests.

In fact, we use a lot of Capybaras.

Like, a whole herd of Capybaras running together in parallel.

We, too, have been bitten by our Capybaras.

We got through it, learned a lot, and now we can't imagine living without our Capybara.

[Slide 9]

Capybaras are the largest rodents in the world. They are related to guinea pigs, but are the size of a big dog or small pig. They live in South America, and they're pretty damn cute.

Capybara is also the name of a Ruby gem written by this guy, Jonas Nicklas. It helps you write integration tests for web apps by pretending to be a user with a web browser. It takes a black box approach to testing, letting you go to URLs, click links and buttons, fill in forms, and check that the rendered page contains what you think it should have in it.

It doesn't replace test runners like RSpec, Cucumber and Minitest, it works with them. It has built-in integration with the most popular testing frameworks, but is test-framework agnostic.

[Slide 10]

It has three parts:

Front end is a Ruby-based DSL for browsing the web: actions like visiting URLs, clicking links and buttons, interacting with forms, and querying for contents of the page.

Powering this is a flexible selector engine: supports CSS selectors, XPath, finding by content or form input labels.

On the back end, Capybara has pluggable drivers for various browsing engines. Out of the box, it supports RackTest and Selenium WebDriver.

RackTest is a headless, pure Ruby browser simulator. Very fast, no setup required, works everywhere, but doesn’t support JavaScript execution.

Selenium drives a real web browser: Firefox, Chrome, IE, etc. Executes JavaScript. Great for compatibility testing and JavaScript-heavy apps, but slow to run, and asynchronous execution can be difficult to understand.

Other options via gems: e.g., Poltergeist integrates with PhantomJS, a headless WebKit implementation.

Most people use Selenium WebDriver. Why not code directly against the WebDriver API? Capybara is more Ruby-like, more flexible, and makes it easier to handle asynchrony.

[Slide 12]

Part of what makes Capybara so appealing is that it feels like magic. This is also what can make it frustrating.

To work effectively with Capybara, you need to learn to think like a Capybara.

[Slide 13]

Capybara automatically waits for asynchronous operations to complete. When you try to find an element that isn't on the page, it waits and retries until it is there, or a timeout duration elapses. Default is 2 seconds.

That's pretty short, so if the sign-in is slow, you might get timeout errors.

[Slide 14]

You can wrap a block in 'using_wait_time' to give it a longer timeout duration.

[Slide 15]

In Capybara 2.1, you can also pass a 'wait' option to individual methods.

[Slide 16]

Methods that unambiguously identify a fixed number of elements will wait.

This isn't fully comprehensive, but most methods fall into one of these categories.

[Slide 17]

Anything where Capybara can't guess how many elements you’re expecting, or what they are, won't wait. They just return whatever is there when you call the method, and don't accept a wait option.

[Slide 18]

This test may fail when it should pass.

[Slide 19]

We can tell Capybara how many results we're expecting, and it will wait for it to become true, or time out.

[Slide 20]

When something is supposed to be removed from a page after an action, Capybara needs to wait for that, too.

[Slide 21]

If we tell Capybara that we're expecting the text not to be there, it will wait for that to be true.

[Slide 22]

Capybara RSpec matchers are smart, so this works too. It knows whether you said expect/to (or should) or expect/not_to (or should_not).

[Slide 23]

We've seen examples of simple Capybara code written in an imperative, script-like style, but in big apps, it can get unwieldy.

Luckily, Capybara speaks Ruby, so we can use all of the tools of the language—classes, inheritance, delegation, blocks—to teach Capybara abstractions and encapsulate the details of our application.

[Slide 24]

This Cucumber scenario is very tightly coupled to the page it is testing. It’s ugly enough here, but in a big app this style of Cucumber is much worse, with URL paths and CSS selectors duplicated everywhere.

You see this a lot in apps that used an older version of the cucumber-rails gem. It used to generate a file of default Cucumber step definitions that looked just like this.

[Slide 25]

Jonas, the author of Capybara, wrote a blog post called “You’re Cuking it Wrong” explaining why this is bad practice.

Aslak Hellesøy, the author of Cucumber, has since disavowed web_steps.rb, and it is no longer created by default.

[Slide 26]

Here, the underlying URL and page markup details have been encapsulated within step definitions. This is much more readable for non-technical stakeholders.

Still, it is very coupled to specific UI implementation details.

[Slide 27]

Now the Cucumber test concisely describes the business requirements.

[Slide 28]

You may be tempted to simply move the tightly-coupled test code into the step definition, but this is hardly better. It is still boring and brittle.

[Slide 29]

We can refactor to page objects that model the UI of the application, and create an object-oriented testing API that can be reused across step definitions.

[Slide 30]

The application tester object can be added to the Cucumber world. It encapsulates access to the Capybara session, and provides methods that represent pages in the app, or services it provides.

[Slide 31]

Each page encapsulates markup details by providing logical methods for the elements on the page. These return Capybara Node objects, which support methods such as click, fill_in, finding descendent elements, etc. The entire Capybara DSL is available, scoped within that node.

For more complex elements, you can encapsulate them in their own page objects that provide higher-level APIs. For example, a date picker could be wrapped in an object that lets you set a date or choose “Today”.

[Slide 34]

Long selectors like this are extremely brittle.

There's no shame in adding CSS IDs and classes to your markup for your tests. Difficulty writing tests can be a code smell: clean, semantic markup makes for easier testing. Try (A)TDD.

Page Objects help here, too. Navigating through a chain of page elements gives useful errors and stack traces.

[Slide 35]

Sometimes, despite your best efforts, Capybara misbehaves. Here are some tips you can use for getting it back in line.

[Slide 36]

Add this to your Cucumber step definitions.

If a step is failing, throw “And I debug” right before it and watch what happens in the browser. If it looks like it’s working, continue in the debugger. If it passes, you probably have a synchronisation issue, where the step isn't waiting for the operation to complete.

[Slide 37]

Capybara can save screenshots. This is a simple way to save a screenshot after each scenario. You can get more clever with this, such as only saving after failures. Our code is pretty complex, but there is also a gem that makes it automatic.

[Slide 38]

One annoying thing with Firefox: sometimes tests fail when it doesn’t have focus. Firefox doesn’t fire certain events when it’s in the background.

focusmanager.testmode is a hidden Firefox configuration that disables this. Google for it and you’ll find examples of how to set it.

New versions of Firefox often break WebDriver, so keep that gem up to date using “bundle update selenium-webdriver” often.

Also, disable Firefox’s auto-update in CI.

Linux handles events differently than Mac OS and Windows. Keep that in mind when diagnosing failures in CI. Make sure your xvnc or xvfb window is large enough to display the page properly. We had to configure the default geometry in Jenkins.

You’ll get a clean profile in Firefox with no add-ons installed by default. Capybara Firebug is a gem that automatically installs and activates Firebug into Firefox when Capybara runs. Really useful, but slows down your tests. Use a Cucumber profile to activate it.

[Slide 41]

The most important thing to do: prioritise fixing flaky tests as a team.

Flaky tests are ones that sometimes pass, sometimes fail, seemingly at random.

It’s all too easy to get into a situation where one flaky test becomes many, your build is always red, and you start to ignore failures. Your tests lose their meaning, and then one day you ship a bug that the tests would have caught.

That really stinks.

Tim Moore

October 31, 2013
Tweet

More Decks by Tim Moore

Other Decks in Programming

Transcript

  1. Embracing
    Capybara

    View Slide

  2. Who Has Used
    Capybara?

    View Slide

  3. Who Has Been
    Bitten By Capybara?

    View Slide

  4. TimMoore
    tmoore

    View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. What’s a
    Capybara?
    World’s
    Largest Rodent!
    “Makes it easy to
    simulate how a
    user interacts with
    your application”
    http://git.io/capybara

    View Slide

  10. Capybara Anatomy
    Browsing

    DSL
    Matchers/

    Selectors
    Browser
    Engines

    View Slide

  11. visit '/sessions/new'
    within("#session") do
    fill_in 'Login',

    with: '[email protected]'
    fill_in 'Password',

    with: 'password'
    end
    click_link 'Sign in'
    expect(page).to have_content 'Success'

    View Slide

  12. Understanding
    Capybara

    View Slide

  13. Capybara is Patient
    visit '/sessions/new'
    within("#session") do

    fill_in 'Login',

    with: '[email protected]'
    fill_in 'Password',

    with: 'password'

    end
    click_link 'Sign in'
    expect(page).

    to have_content 'Success'
    Waits for page to load

    View Slide

  14. Capybara is Patient
    visit '/sessions/new'
    within("#session") do

    fill_in 'Login',

    with: '[email protected]'
    fill_in 'Password',

    with: 'password'

    end
    click_link 'Sign in'
    Capybara.using_wait_time(60) do

    expect(page).

    to have_content 'Success'

    end
    Waits for page to load

    View Slide

  15. Capybara is Patient
    visit '/sessions/new'
    within("#session") do

    fill_in 'Login',

    with: '[email protected]'
    fill_in 'Password',

    with: 'password'

    end
    click_link 'Sign in'
    expect(page).

    to have_content 'Success',

    wait: 60
    Waits for page to load

    View Slide

  16. Methods that Wait
    • find(selector), find_field, find_link, etc.
    • within(selector) (scoping)
    • has_selector?/has_no_selector? & assertions
    • form & link actions
    • click_link/button
    • fill_in
    • check/uncheck, select, choose, etc.

    View Slide

  17. Methods that Don’t
    • visit(path)
    • current_path
    • all(selector)
    • first(selector) (counter-intuitively)
    • execute_script/evaluate_script
    • simple accessors: text, value, title, etc.

    View Slide

  18. Don’t Do This
    click_link('Search')
    expect(all('.search-result').count).

    to eq 2
    Returns 0
    Does not wait

    View Slide

  19. Good
    click_link('Search')
    expect(page).

    to have_css '.search-result',

    count: 2

    View Slide

  20. Don’t Do This
    click_button('Save')
    expect(find('.dirty-warning').text).

    not_to contain('Unsaved changes')
    Does not wait

    View Slide

  21. Good
    click_button('Save')
    expect(find('.dirty-warning')).

    to have_no_text('Unsaved changes')

    View Slide

  22. Also Good
    click_button('Save')
    expect(find('.dirty-warning')).

    not_to have_text('Unsaved changes')

    View Slide

  23. Training Capybara

    View Slide

  24. Don’t Do This
    Scenario: Searching for cats
    Given I am on "/search"
    When I fill in "input.search_text"
    with "cats"
    And I press "Search"
    Then I should see "grumpy cat"
    within "div.search-results"

    View Slide

  25. Don’t Do This
    Scenario: Searching for cats
    Given I am on "/search"
    When I fill in "input.search_text"
    with "cats"
    And I press "Search"
    Then I should see "grumpy cat"
    within "div.search-results"
    “A step description
    should never contain
    regexen, CSS or
    XPath selectors”
    “web_steps.rb is a terrible, terrible idea”
    - Boring scenarios
    - Brittle scenarios
    - Where is my workflow?

    View Slide

  26. Better
    Scenario: Searching for cats
    Given I am on the search page
    When I fill in "Search" with "cats"
    And I press "Search"
    Then I should see "grumpy cat"
    within the search results

    View Slide

  27. Best
    Scenario: Searching for cats
    When I search for "cats"
    Then I should see "grumpy cat" within
    the search results

    View Slide

  28. Don’t Do This
    When /^I search for "([^"]*)"$/

    do |search_text|
    visit "/search"
    fill_in "input.search_text",

    with: search_text
    click_button "Search"
    end

    View Slide

  29. Better
    When /^I search for "([^"]*)"$/

    do |search_text|
    search_page =

    my_app.search_page
    search_page.search_input.

    fill_in search_text
    search_page.search_button.click
    end

    View Slide

  30. Better
    class MyAppTester
    def initialize
    @session = Capybara::Session.new(:selenium)
    end
    !
    def search_page
    MyAppTester::SearchPage.new(@session)
    end
    end

    View Slide

  31. Better
    class MyAppTester::SearchPage
    def initialize(session)

    session.visit "/search"

    @session = session

    end
    def search_input

    @session.find "input.search_text"

    end
    def search_button

    @session.find "button.search"

    end
    end

    View Slide

  32. Better
    When /^I search for "([^"]*)"$/

    do |search_text|
    search_page =

    my_app.search_page
    search_page.search_input.

    fill_in search_text
    search_page.search_button.click
    end

    View Slide

  33. Best
    When /^I search for "([^"]*)"$/

    do |search_text|
    my_app.search_for search_text
    end

    View Slide

  34. Don’t Do This
    find(:css, "div.main div.sidebar
    div.search div.advanced div.actions
    button.submit").click
    Failure/Error: find(:css, "div.main div.sidebar
    div.search div.advanced div.actions button.submit").click
    Capybara::ElementNotFound:
    Unable to find css "div.main div.sidebar
    div.search div.advanced div.actions button.submit"

    View Slide

  35. Disciplining
    Capybara

    View Slide

  36. Setting Breakpoints
    When /^I debug$/ do
    debugger
    end

    View Slide

  37. Taking Screenshots
    After do |scenario|
    filename =

    scenario.name.gsub(/\W/, '_')
    page.save_screenshot(

    "target/reports/#{filename}.png")
    end
    !
    http://git.io/capybarascreenshot

    View Slide

  38. Firefox
    •focusmanager.testmode = true
    • Update selenium-webdriver gem
    • Disable auto-update in CI
    • Beware platform differences
    • http://git.io/capybarafirebug

    View Slide

  39. Getting Help
    • http://groups.google.com/group/
    ruby-capybara
    • Freenode (IRC): #capybara
    • Stack Overflow

    View Slide

  40. Summary
    • Understand Capybara synchronisation
    • Be explicit with Capybara
    • Use Page Objects for readable,
    flexible tests
    • Keep selectors simple
    • If you get bitten, seek help!

    View Slide

  41. Fix Flaky Tests

    View Slide

  42. Thanks!
    Any Questions?

    View Slide