Embracing Capybara

I’m going to talk about Embracing Capybara: how to understand Capybara, train it to do new tricks, and discipline it when it misbehaves.

[Slide 4]

My name is Tim. I work at Lonely Planet, on a fairly complex single-page app, and we use Capybara for our tests.

In fact, we use a lot of Capybaras.

Like, a whole herd of Capybaras running together in parallel.

We, too, have been bitten by our Capybaras.

We got through it, learned a lot, and now we can't imagine living without our Capybara.

[Slide 9]

Capybaras are the largest rodents in the world. They are related to guinea pigs, but are the size of a big dog or small pig. They live in South America, and they're pretty damn cute.

Capybara is also the name of a Ruby gem written by this guy, Jonas Nicklas. It helps you write integration tests for web apps by pretending to be a user with a web browser. It takes a black box approach to testing, letting you go to URLs, click links and buttons, fill in forms, and check that the rendered page contains what you think it should have in it.

It doesn't replace test runners like RSpec, Cucumber and Minitest, it works with them. It has built-in integration with the most popular testing frameworks, but is test-framework agnostic.

[Slide 10]

It has three parts:

Front end is a Ruby-based DSL for browsing the web: actions like visiting URLs, clicking links and buttons, interacting with forms, and querying for contents of the page.

Powering this is a flexible selector engine: supports CSS selectors, XPath, finding by content or form input labels.

On the back end, Capybara has pluggable drivers for various browsing engines. Out of the box, it supports RackTest and Selenium WebDriver.

RackTest is a headless, pure Ruby browser simulator. Very fast, no setup required, works everywhere, but doesn’t support JavaScript execution.

Selenium drives a real web browser: Firefox, Chrome, IE, etc. Executes JavaScript. Great for compatibility testing and JavaScript-heavy apps, but slow to run, and asynchronous execution can be difficult to understand.

Other options via gems: e.g., Poltergeist integrates with PhantomJS, a headless WebKit implementation.

Most people use Selenium WebDriver. Why not code directly against the WebDriver API? Capybara is more Ruby-like, more flexible, and makes it easier to handle asynchrony.

[Slide 12]

Part of what makes Capybara so appealing is that it feels like magic. This is also what can make it frustrating.

To work effectively with Capybara, you need to learn to think like a Capybara.

[Slide 13]

Capybara automatically waits for asynchronous operations to complete. When you try to find an element that isn't on the page, it waits and retries until it is there, or a timeout duration elapses. Default is 2 seconds.

That's pretty short, so if the sign-in is slow, you might get timeout errors.

[Slide 14]

You can wrap a block in 'using_wait_time' to give it a longer timeout duration.

[Slide 15]

In Capybara 2.1, you can also pass a 'wait' option to individual methods.

[Slide 16]

Methods that unambiguously identify a fixed number of elements will wait.

This isn't fully comprehensive, but most methods fall into one of these categories.

[Slide 17]

Anything where Capybara can't guess how many elements you’re expecting, or what they are, won't wait. They just return whatever is there when you call the method, and don't accept a wait option.

[Slide 18]

This test may fail when it should pass.

[Slide 19]

We can tell Capybara how many results we're expecting, and it will wait for it to become true, or time out.

[Slide 20]

When something is supposed to be removed from a page after an action, Capybara needs to wait for that, too.

[Slide 21]

If we tell Capybara that we're expecting the text not to be there, it will wait for that to be true.

[Slide 22]

Capybara RSpec matchers are smart, so this works too. It knows whether you said expect/to (or should) or expect/not_to (or should_not).

[Slide 23]

We've seen examples of simple Capybara code written in an imperative, script-like style, but in big apps, it can get unwieldy.

Luckily, Capybara speaks Ruby, so we can use all of the tools of the language—classes, inheritance, delegation, blocks—to teach Capybara abstractions and encapsulate the details of our application.

[Slide 24]

This Cucumber scenario is very tightly coupled to the page it is testing. It’s ugly enough here, but in a big app this style of Cucumber is much worse, with URL paths and CSS selectors duplicated everywhere.

You see this a lot in apps that used an older version of the cucumber-rails gem. It used to generate a file of default Cucumber step definitions that looked just like this.

[Slide 25]

Jonas, the author of Capybara, wrote a blog post called “You’re Cuking it Wrong” explaining why this is bad practice.

Aslak Hellesøy, the author of Cucumber, has since disavowed web_steps.rb, and it is no longer created by default.

[Slide 26]

Here, the underlying URL and page markup details have been encapsulated within step definitions. This is much more readable for non-technical stakeholders.

Still, it is very coupled to specific UI implementation details.

[Slide 27]

Now the Cucumber test concisely describes the business requirements.

[Slide 28]

You may be tempted to simply move the tightly-coupled test code into the step definition, but this is hardly better. It is still boring and brittle.

[Slide 29]

We can refactor to page objects that model the UI of the application, and create an object-oriented testing API that can be reused across step definitions.

[Slide 30]

The application tester object can be added to the Cucumber world. It encapsulates access to the Capybara session, and provides methods that represent pages in the app, or services it provides.

[Slide 31]

Each page encapsulates markup details by providing logical methods for the elements on the page. These return Capybara Node objects, which support methods such as click, fill_in, finding descendent elements, etc. The entire Capybara DSL is available, scoped within that node.

For more complex elements, you can encapsulate them in their own page objects that provide higher-level APIs. For example, a date picker could be wrapped in an object that lets you set a date or choose “Today”.

[Slide 34]

Long selectors like this are extremely brittle.

There's no shame in adding CSS IDs and classes to your markup for your tests. Difficulty writing tests can be a code smell: clean, semantic markup makes for easier testing. Try (A)TDD.

Page Objects help here, too. Navigating through a chain of page elements gives useful errors and stack traces.

[Slide 35]

Sometimes, despite your best efforts, Capybara misbehaves. Here are some tips you can use for getting it back in line.

[Slide 36]

Add this to your Cucumber step definitions.

If a step is failing, throw “And I debug” right before it and watch what happens in the browser. If it looks like it’s working, continue in the debugger. If it passes, you probably have a synchronisation issue, where the step isn't waiting for the operation to complete.

[Slide 37]

Capybara can save screenshots. This is a simple way to save a screenshot after each scenario. You can get more clever with this, such as only saving after failures. Our code is pretty complex, but there is also a gem that makes it automatic.

[Slide 38]

One annoying thing with Firefox: sometimes tests fail when it doesn’t have focus. Firefox doesn’t fire certain events when it’s in the background.

focusmanager.testmode is a hidden Firefox configuration that disables this. Google for it and you’ll find examples of how to set it.

New versions of Firefox often break WebDriver, so keep that gem up to date using “bundle update selenium-webdriver” often.

Also, disable Firefox’s auto-update in CI.

Linux handles events differently than Mac OS and Windows. Keep that in mind when diagnosing failures in CI. Make sure your xvnc or xvfb window is large enough to display the page properly. We had to configure the default geometry in Jenkins.

You’ll get a clean profile in Firefox with no add-ons installed by default. Capybara Firebug is a gem that automatically installs and activates Firebug into Firefox when Capybara runs. Really useful, but slows down your tests. Use a Cucumber profile to activate it.

[Slide 41]

The most important thing to do: prioritise fixing flaky tests as a team.

Flaky tests are ones that sometimes pass, sometimes fail, seemingly at random.

It’s all too easy to get into a situation where one flaky test becomes many, your build is always red, and you start to ignore failures. Your tests lose their meaning, and then one day you ship a bug that the tests would have caught.

That really stinks.

Transcript

1. Embracing Capybara
2. Who Has Used Capybara?
3. Who Has Been Bitten By Capybara?
4. TimMoore tmoore
9. What’s a Capybara? World’s Largest Rodent! “Makes it easy to simulate how a user interacts with your application” http://git.io/capybara
10. Capybara Anatomy Browsing
 DSL Matchers/
 Selectors Browser Engines
11. visit '/sessions/new' within("#session") do fill_in 'Login',
 with: 'user@example.com' fill_in 'Password',
 with: 'password' end click_link 'Sign in' expect(page).to have_content 'Success'
12. Understanding Capybara
13. Capybara is Patient visit '/sessions/new' within("#session") do
 fill_in 'Login',
 with: 'user@example.com' fill_in 'Password',
 with: 'password'
 end click_link 'Sign in' expect(page).
 to have_content 'Success' Waits for page to load
14. Capybara is Patient visit '/sessions/new' within("#session") do
 fill_in 'Login',
 with: 'user@example.com' fill_in 'Password',
 with: 'password'
 end click_link 'Sign in' Capybara.using_wait_time(60) do
 expect(page).
 to have_content 'Success'
 end Waits for page to load
15. Capybara is Patient visit '/sessions/new' within("#session") do
 fill_in 'Login',
 with: 'user@example.com' fill_in 'Password',
 with: 'password'
 end click_link 'Sign in' expect(page).
 to have_content 'Success',
 wait: 60 Waits for page to load
16. Methods that Wait • find(selector), find_field, find_link, etc. • within(selector) (scoping) • has_selector?/has_no_selector? & assertions • form & link actions • click_link/button • fill_in • check/uncheck, select, choose, etc.
17. Methods that Don’t • visit(path) • current_path • all(selector) • first(selector) (counter-intuitively) • execute_script/evaluate_script • simple accessors: text, value, title, etc.
18. Don’t Do This click_link('Search') expect(all('.search-result').count).
 to eq 2 Returns 0 Does not wait
19. Good click_link('Search') expect(page).
 to have_css '.search-result',
 count: 2
20. Don’t Do This click_button('Save') expect(find('.dirty-warning').text).
 not_to contain('Unsaved changes') Does not wait
21. Good click_button('Save') expect(find('.dirty-warning')).
 to have_no_text('Unsaved changes')
22. Also Good click_button('Save') expect(find('.dirty-warning')).
 not_to have_text('Unsaved changes')
23. Training Capybara
24. Don’t Do This Scenario: Searching for cats Given I am on "/search" When I fill in "input.search_text" with "cats" And I press "Search" Then I should see "grumpy cat" within "div.search-results"
25. Don’t Do This Scenario: Searching for cats Given I am on "/search" When I fill in "input.search_text" with "cats" And I press "Search" Then I should see "grumpy cat" within "div.search-results" “A step description should never contain regexen, CSS or XPath selectors” “web_steps.rb is a terrible, terrible idea” - Boring scenarios - Brittle scenarios - Where is my workflow?
26. Better Scenario: Searching for cats Given I am on the search page When I fill in "Search" with "cats" And I press "Search" Then I should see "grumpy cat" within the search results
27. Best Scenario: Searching for cats When I search for "cats" Then I should see "grumpy cat" within the search results
28. Don’t Do This When /^I search for "([^"]*)"$/
 do |search_text| visit "/search" fill_in "input.search_text",
 with: search_text click_button "Search" end
29. Better When /^I search for "([^"]*)"$/
 do |search_text| search_page =
 my_app.search_page search_page.search_input.
 fill_in search_text search_page.search_button.click end
30. Better class MyAppTester def initialize @session = Capybara::Session.new(:selenium) end ! def search_page MyAppTester::SearchPage.new(@session) end end
31. Better class MyAppTester::SearchPage def initialize(session)
 session.visit "/search"
 @session = session
 end def search_input
 @session.find "input.search_text"
 end def search_button
 @session.find "button.search"
 end end
32. Better When /^I search for "([^"]*)"$/
 do |search_text| search_page =
 my_app.search_page search_page.search_input.
 fill_in search_text search_page.search_button.click end
33. Best When /^I search for "([^"]*)"$/
 do |search_text| my_app.search_for search_text end
34. Don’t Do This find(:css, "div.main div.sidebar div.search div.advanced div.actions button.submit").click Failure/Error: find(:css, "div.main div.sidebar div.search div.advanced div.actions button.submit").click Capybara::ElementNotFound: Unable to find css "div.main div.sidebar div.search div.advanced div.actions button.submit"
35. Disciplining Capybara
36. Setting Breakpoints When /^I debug$/ do debugger end
37. Taking Screenshots After do |scenario| filename =
 scenario.name.gsub(/\W/, '_') page.save_screenshot(
 "target/reports/#{filename}.png") end ! http://git.io/capybarascreenshot
38. Firefox •focusmanager.testmode = true • Update selenium-webdriver gem • Disable auto-update in CI • Beware platform differences • http://git.io/capybarafirebug
39. Getting Help • http://groups.google.com/group/ ruby-capybara • Freenode (IRC): #capybara • Stack Overflow
40. Summary • Understand Capybara synchronisation • Be explicit with Capybara • Use Page Objects for readable, flexible tests • Keep selectors simple • If you get bitten, seek help!
41. Fix Flaky Tests
42. Thanks!
    Any Questions?