Cucumber Automation Testing

The basics of Cucumber Automation Testing, and some best practices when it comes to formatting and code style.

Transcript

1. Automation Testing
2. Basics
3. Basics We write our automation tests using a Ruby based tool called “Cucumber”.
4. Basics Cucumber is a BDD test framework. We use it to test that the behaviour of things is as we expect.
5. Basics The test outline is written in a language called “Gherkin”. It uses the “Given”, “When”, “Then” format.
6. Basics Given we have a release to do And “Chris” is asked to deploy it When we go to deploy Then “Chris” should be sad And the release should be “on fire”
7. Basics You then write step definitions in Ruby, which actually execute the steps we wrote in Gherkin.
8. Basics Given “we have a release to do” do @release = Release.new end
9. Basics You can define step definitions using a Regex to allow some dynamism.
10. Basics Given /”([^”]+)” is asked to deploy it/ do |name| @deployer = Person.new(name) @release.deployer = @deployer end
11. Basics The Givens should be steps that do some setup.
12. Basics The When steps are some form of action. These actions should depend on the state that was setup when the Given steps were run (if any).
13. Basics When “we go to deploy” do @release.deploy! end
14. Basics Finally, the Then steps consist of the assertions to make sure everything went as expected.
15. Basics Then /”([^”]+)” should be “([^”]+)”/ do |name, mood| expect(@deployer.name).to eq(name) expect(@deployer.mood).to eq(mood) end
16. Basics Then /the release should be “([^”]+)”/ do |status| expect(@release.status).to eq(status) end
17. Basics And that’s it! Cucumber catches the exceptions that expect (which comes from the RSpec expectations gem) throws, and turns them into pretty errors that explain where your test failed.
18. Directory Structure
19. Directory Structure Our tests are kept in /automation-tests.
20. Directory Structure . └── cucumber ├── Gemfile ├── Gemfile.lock ├── projects │ └── ... └── Rakefile
21. Directory Structure The Gemfile contains a list of our dependencies, which are fetched from RubyGems.org. The Gemfile.lock contains rules about versions for these, and is auto-generated when a version is updated.
22. Directory Structure The projects directory contains the sanity directory. This is where our tests are kept. /automation-tests/cucumber/projects/sanity
23. Directory Structure You can run the tests from here: $ cucumber This will run all of the tests.
24. Directory Structure . └── features ├── *.feature └── step_definitions ├── *.rb └── support └── *.rb
25. Directory Structure The Gherkin steps go in files called foobar.feature, and should be saved into the sanity/features directory.
26. Directory Structure The step definitions go in files called foobar.rb, and should be saved in the sanity/features/step_definitions directory.
27. Directory Structure The sanity/features/step_definitions/support directory is where test utilities go.
28. Directory Structure For example, we have api_server.rb, a helper class that encapsulates the URLs we need to build when testing across all of our servers.
29. Directory Structure There is also env.rb, a special Cucumber file that gets loaded first and contains the global before/after hooks for the tests. Any global setup goes in here.
30. Directory Structure support/widgets contains files that define helpers suitable for encapsulating the location of certain DOM nodes and popup windows while testing.
31. Directory Structure Methods you define in support/widgets/base_widget.rb are automatically accessible from your step definitions. This is thanks to the World(PageWidgets) call at the bottom of this file.
32. Structure of Tests
33. The steps of a single test are referred to as a “Scenario”. Structure of Tests
34. Scenario: Deploying with no internet Given we have no internet When we deploy Then the deployment should fail And the error should be “the interwebs is downzorz, oh noes” And there should be a distinct lack of doge and catz vidz Structure of Tests
35. You can have as many Scenarios in a feature file as you want. Structure of Tests
36. However, each feature file should contain only related Scenarios. For example, one feature file for Social Login, and one feature file for Public Sharing. Structure of Tests
37. Scenario Outlines allow you to DRY up related tests. Structure of Tests
38. Scenario: Social Login With Facebook Given I sign in with “facebook” Then I can see the user status Scenario: Social Login With Twitter Given I sign in with “twitter” Then I can see the user status Structure of Tests
39. Scenario Outline: Social Login When I sign in with “<social_network>” Then I can see the user status Examples: | social_network | | facebook | | twitter | Structure of Tests
40. Tags can be used to group and run tests together. Structure of Tests
41. @foo Scenario: One Given Foo Then Lux Structure of Tests
42. Tags can be placed above a Scenario (Outline), or above the Feature line to apply to all tests in the feature file. @foo @bar Feature: Something Scenario: ... Structure of Tests
43. Tags should be dash separated. @foo_bar # Bad @foo-bar # Good Structure of Tests
44. Directory Structure You can run just certain tags: $ cucumber -t @foo This will run all of the tests tagged with @foo.
45. Watir
46. Watir We use Watir to talk to the browser and interface with the DOM.
47. Watir We create a Watir::Browser object and store it at @browser.
48. Watir As an example, to find an <a> with an ID: @browser.link(:id => “foobar”)
49. Watir To find a <input> with a class: @browser.input(:class => “foobar”)
50. Watir To find a <button> with a selector: @browser.element(:css => “body > button:nth-child (2)”).to_subtype
51. Watir All of these calls will return an element object. You can then call methods on these like: #click - Clicks on the element. #hover - Hovers over the element. #present? - Whether the element is in the page. #wait_until_present - Waits until the element is in the page, up to our maximum timeout (currently 30s).
52. Watir Watir has lots of other methods you can use, see the docs for it or look at our existing automation tests for more information.
53. Manual of Style
54. Manual of Style No auto-formatting. Do not use the auto-formatting tool in IntelliJ yet, it causes conflicts over formatting changes.
55. Manual of Style Put newlines between tests.
56. Manual of Style This is bad: Scenario: Foo Given … Scenario: Bar Given ...
57. Manual of Style This is good: Scenario: Foo Given … Scenario: Bar Given ...
58. Manual of Style Newline after the Feature line. Feature: Foo bar lux. Background: ...
59. Manual of Style No newlines between the Scenario (Outline) line, and the first step.
60. Manual of Style This is good: Scenario: Foo Given ...
61. Manual of Style This is bad: Scenario: Foo Given ...
62. Manual of Style No newline between Scenario Outline steps and Examples.
63. Manual of Style This is bad: Scenario Outline: Foo Given … Examples: | foo | bar | | ... | ... |
64. Manual of Style This is good: Scenario Outline: Foo Given … Examples: | foo | bar | | ... | ... |
65. Things to Remember
66. Things to Remember Duplicate step definitions are the achilles heel of Cucumber tests.
67. Things to Remember You must always check whether an existing step definition can do what you need without adding a new one.
68. Things to Remember If you aren’t sure, ask somebody to confirm.
69. Things to Remember When I select the option "<plugin option>" When I select social-recommendations plugin When I select the “social-recommendations”
70. Things to Remember These all did the same thing. They are being replaced with simply: When I select the "<plugin option>" plugin from the plugin list
71. Things to Remember Wording of step definitions is super important. When I click on the "<social_network>"
72. Things to Remember If people can’t find a step definition, they will just create a new one, so getting the wording and names of features right is very important.
73. Things to Remember Other examples of bad names: Social Logging # Bad, not the right name Social Login # Good, the right name
74. Things to Remember At least three names for the same thing: IntentHQ Plugin Builder tool # Bad HQ plugin builder # Bad plugin builder page # Bad The plugin builder # Good
75. Things to Remember Make sure to pay close attention to keep your steps doing what their verb says they do: Givens are setup Whens are actions Thens are assertions
76. Things to Remember If one of your steps does more than one of these types of things, split the step up appropriately.
77. Things to Remember Remember, you are describing a feature, not a sequence of actions. That means that it’s okay to record all of your required state in Given steps, even if it doesn’t get applied until a Then executes.
78. Tricks and Tips
79. We have three environment variables we look for when we run the tests. TEST_BROWSER TEST_SERVER TEST_NO_HEADLESS Tricks and Tips
80. TEST_BROWSER Sets the browser to run the tests in. Valid values are firefox, chrome and safari. Defaults to chrome. Tricks and Tips
81. TEST_SERVER Sets the server to run the tests against. Valid values are local, devn (eg. dev1), staging and live. Defaults to local. Tricks and Tips
82. TEST_NO_HEADLESS If set, the tests will not run in headless mode. This means you’ll be able to see the browser. Tricks and Tips
83. You can either export these (perhaps in your .{ba,z} shrc: $ export TEST_BROWSER=firefox $ export TEST_SERVER=dev2 $ export TEST_NO_HEADLESS=true $ cucumber Tricks and Tips
84. Or give them inline (they are, after all, just normal old environment variables): $ TEST_BROWSER=firefox TEST_SERVER=dev2 TEST_NO_HEADLESS=true cucumber Tricks and Tips
85. Useful Links
86. Useful Links • Cucumber: http://cukes.info/ • Cucumber Book: Ask Vamsi! • RSpec Expectations: https://www.relishapp. com/rspec/rspec-expectations/docs https://github.com/rspec/rspec-expectations
87. Useful Links • Watir: http://watir.com • Watir Cheat Sheet: https://github. com/watir/watir/wiki/Cheat-Sheet
88. Finito