Save 37% off PRO during our Black Friday Sale! »

Cucumber Automation Testing

224884ff0b79e700e516e24fbcae6423?s=47 Nathan Kleyn
November 12, 2013

Cucumber Automation Testing

The basics of Cucumber Automation Testing, and some best practices when it comes to formatting and code style.


Nathan Kleyn

November 12, 2013


  1. Automation Testing

  2. Basics

  3. Basics We write our automation tests using a Ruby based

    tool called “Cucumber”.
  4. Basics Cucumber is a BDD test framework. We use it

    to test that the behaviour of things is as we expect.
  5. Basics The test outline is written in a language called

    “Gherkin”. It uses the “Given”, “When”, “Then” format.
  6. Basics Given we have a release to do And “Chris”

    is asked to deploy it When we go to deploy Then “Chris” should be sad And the release should be “on fire”
  7. Basics You then write step definitions in Ruby, which actually

    execute the steps we wrote in Gherkin.
  8. Basics Given “we have a release to do” do @release

    = end
  9. Basics You can define step definitions using a Regex to

    allow some dynamism.
  10. Basics Given /”([^”]+)” is asked to deploy it/ do |name|

    @deployer = @release.deployer = @deployer end
  11. Basics The Givens should be steps that do some setup.

  12. Basics The When steps are some form of action. These

    actions should depend on the state that was setup when the Given steps were run (if any).
  13. Basics When “we go to deploy” do @release.deploy! end

  14. Basics Finally, the Then steps consist of the assertions to

    make sure everything went as expected.
  15. Basics Then /”([^”]+)” should be “([^”]+)”/ do |name, mood| expect(

    eq(name) expect(@deployer.mood).to eq(mood) end
  16. Basics Then /the release should be “([^”]+)”/ do |status| expect(@release.status).to

    eq(status) end
  17. Basics And that’s it! Cucumber catches the exceptions that expect

    (which comes from the RSpec expectations gem) throws, and turns them into pretty errors that explain where your test failed.
  18. Directory Structure

  19. Directory Structure Our tests are kept in /automation-tests.

  20. Directory Structure . └── cucumber ├── Gemfile ├── Gemfile.lock ├──

    projects │ └── ... └── Rakefile
  21. Directory Structure The Gemfile contains a list of our dependencies,

    which are fetched from The Gemfile.lock contains rules about versions for these, and is auto-generated when a version is updated.
  22. Directory Structure The projects directory contains the sanity directory. This

    is where our tests are kept. /automation-tests/cucumber/projects/sanity
  23. Directory Structure You can run the tests from here: $

    cucumber This will run all of the tests.
  24. Directory Structure . └── features ├── *.feature └── step_definitions ├──

    *.rb └── support └── *.rb
  25. Directory Structure The Gherkin steps go in files called foobar.feature,

    and should be saved into the sanity/features directory.
  26. Directory Structure The step definitions go in files called foobar.rb,

    and should be saved in the sanity/features/step_definitions directory.
  27. Directory Structure The sanity/features/step_definitions/support directory is where test utilities go.

  28. Directory Structure For example, we have api_server.rb, a helper class

    that encapsulates the URLs we need to build when testing across all of our servers.
  29. Directory Structure There is also env.rb, a special Cucumber file

    that gets loaded first and contains the global before/after hooks for the tests. Any global setup goes in here.
  30. Directory Structure support/widgets contains files that define helpers suitable for

    encapsulating the location of certain DOM nodes and popup windows while testing.
  31. Directory Structure Methods you define in support/widgets/base_widget.rb are automatically accessible

    from your step definitions. This is thanks to the World(PageWidgets) call at the bottom of this file.
  32. Structure of Tests

  33. The steps of a single test are referred to as

    a “Scenario”. Structure of Tests
  34. Scenario: Deploying with no internet Given we have no internet

    When we deploy Then the deployment should fail And the error should be “the interwebs is downzorz, oh noes” And there should be a distinct lack of doge and catz vidz Structure of Tests
  35. You can have as many Scenarios in a feature file

    as you want. Structure of Tests
  36. However, each feature file should contain only related Scenarios. For

    example, one feature file for Social Login, and one feature file for Public Sharing. Structure of Tests
  37. Scenario Outlines allow you to DRY up related tests. Structure

    of Tests
  38. Scenario: Social Login With Facebook Given I sign in with

    “facebook” Then I can see the user status Scenario: Social Login With Twitter Given I sign in with “twitter” Then I can see the user status Structure of Tests
  39. Scenario Outline: Social Login When I sign in with “<social_network>”

    Then I can see the user status Examples: | social_network | | facebook | | twitter | Structure of Tests
  40. Tags can be used to group and run tests together.

    Structure of Tests
  41. @foo Scenario: One Given Foo Then Lux Structure of Tests

  42. Tags can be placed above a Scenario (Outline), or above

    the Feature line to apply to all tests in the feature file. @foo @bar Feature: Something Scenario: ... Structure of Tests
  43. Tags should be dash separated. @foo_bar # Bad @foo-bar #

    Good Structure of Tests
  44. Directory Structure You can run just certain tags: $ cucumber

    -t @foo This will run all of the tests tagged with @foo.
  45. Watir

  46. Watir We use Watir to talk to the browser and

    interface with the DOM.
  47. Watir We create a Watir::Browser object and store it at

  48. Watir As an example, to find an <a> with an

    ID: => “foobar”)
  49. Watir To find a <input> with a class: @browser.input(:class =>

  50. Watir To find a <button> with a selector: @browser.element(:css =>

    “body > button:nth-child (2)”).to_subtype
  51. Watir All of these calls will return an element object.

    You can then call methods on these like: #click - Clicks on the element. #hover - Hovers over the element. #present? - Whether the element is in the page. #wait_until_present - Waits until the element is in the page, up to our maximum timeout (currently 30s).
  52. Watir Watir has lots of other methods you can use,

    see the docs for it or look at our existing automation tests for more information.
  53. Manual of Style

  54. Manual of Style No auto-formatting. Do not use the auto-formatting

    tool in IntelliJ yet, it causes conflicts over formatting changes.
  55. Manual of Style Put newlines between tests.

  56. Manual of Style This is bad: Scenario: Foo Given …

    Scenario: Bar Given ...
  57. Manual of Style This is good: Scenario: Foo Given …

    Scenario: Bar Given ...
  58. Manual of Style Newline after the Feature line. Feature: Foo

    bar lux. Background: ...
  59. Manual of Style No newlines between the Scenario (Outline) line,

    and the first step.
  60. Manual of Style This is good: Scenario: Foo Given ...

  61. Manual of Style This is bad: Scenario: Foo Given ...

  62. Manual of Style No newline between Scenario Outline steps and

  63. Manual of Style This is bad: Scenario Outline: Foo Given

    … Examples: | foo | bar | | ... | ... |
  64. Manual of Style This is good: Scenario Outline: Foo Given

    … Examples: | foo | bar | | ... | ... |
  65. Things to Remember

  66. Things to Remember Duplicate step definitions are the achilles heel

    of Cucumber tests.
  67. Things to Remember You must always check whether an existing

    step definition can do what you need without adding a new one.
  68. Things to Remember If you aren’t sure, ask somebody to

  69. Things to Remember When I select the option "<plugin option>"

    When I select social-recommendations plugin When I select the “social-recommendations”
  70. Things to Remember These all did the same thing. They

    are being replaced with simply: When I select the "<plugin option>" plugin from the plugin list
  71. Things to Remember Wording of step definitions is super important.

    When I click on the "<social_network>"
  72. Things to Remember If people can’t find a step definition,

    they will just create a new one, so getting the wording and names of features right is very important.
  73. Things to Remember Other examples of bad names: Social Logging

    # Bad, not the right name Social Login # Good, the right name
  74. Things to Remember At least three names for the same

    thing: IntentHQ Plugin Builder tool # Bad HQ plugin builder # Bad plugin builder page # Bad The plugin builder # Good
  75. Things to Remember Make sure to pay close attention to

    keep your steps doing what their verb says they do: Givens are setup Whens are actions Thens are assertions
  76. Things to Remember If one of your steps does more

    than one of these types of things, split the step up appropriately.
  77. Things to Remember Remember, you are describing a feature, not

    a sequence of actions. That means that it’s okay to record all of your required state in Given steps, even if it doesn’t get applied until a Then executes.
  78. Tricks and Tips

  79. We have three environment variables we look for when we

    run the tests. TEST_BROWSER TEST_SERVER TEST_NO_HEADLESS Tricks and Tips
  80. TEST_BROWSER Sets the browser to run the tests in. Valid

    values are firefox, chrome and safari. Defaults to chrome. Tricks and Tips
  81. TEST_SERVER Sets the server to run the tests against. Valid

    values are local, devn (eg. dev1), staging and live. Defaults to local. Tricks and Tips
  82. TEST_NO_HEADLESS If set, the tests will not run in headless

    mode. This means you’ll be able to see the browser. Tricks and Tips
  83. You can either export these (perhaps in your .{ba,z} shrc:

    $ export TEST_BROWSER=firefox $ export TEST_SERVER=dev2 $ export TEST_NO_HEADLESS=true $ cucumber Tricks and Tips
  84. Or give them inline (they are, after all, just normal

    old environment variables): $ TEST_BROWSER=firefox TEST_SERVER=dev2 TEST_NO_HEADLESS=true cucumber Tricks and Tips
  85. Useful Links

  86. Useful Links • Cucumber: • Cucumber Book: Ask Vamsi!

    • RSpec Expectations: https://www.relishapp. com/rspec/rspec-expectations/docs
  87. Useful Links • Watir: • Watir Cheat Sheet: https://github.

  88. Finito