Basics
We write our automation tests using a Ruby based
tool called “Cucumber”.
Slide 4
Slide 4 text
Basics
Cucumber is a BDD test framework.
We use it to test that the behaviour of things is as we
expect.
Slide 5
Slide 5 text
Basics
The test outline is written in a language called
“Gherkin”.
It uses the “Given”, “When”, “Then” format.
Slide 6
Slide 6 text
Basics
Given we have a release to do
And “Chris” is asked to deploy it
When we go to deploy
Then “Chris” should be sad
And the release should be “on fire”
Slide 7
Slide 7 text
Basics
You then write step definitions in Ruby, which actually
execute the steps we wrote in Gherkin.
Slide 8
Slide 8 text
Basics
Given “we have a release to do” do
@release = Release.new
end
Slide 9
Slide 9 text
Basics
You can define step definitions using a Regex to allow
some dynamism.
Slide 10
Slide 10 text
Basics
Given /”([^”]+)” is asked to deploy it/ do |name|
@deployer = Person.new(name)
@release.deployer = @deployer
end
Slide 11
Slide 11 text
Basics
The Givens should be steps that do some setup.
Slide 12
Slide 12 text
Basics
The When steps are some form of action. These
actions should depend on the state that was setup
when the Given steps were run (if any).
Slide 13
Slide 13 text
Basics
When “we go to deploy” do
@release.deploy!
end
Slide 14
Slide 14 text
Basics
Finally, the Then steps consist of the assertions to
make sure everything went as expected.
Slide 15
Slide 15 text
Basics
Then /”([^”]+)” should be “([^”]+)”/ do |name, mood|
expect(@deployer.name).to eq(name)
expect(@deployer.mood).to eq(mood)
end
Slide 16
Slide 16 text
Basics
Then /the release should be “([^”]+)”/ do |status|
expect(@release.status).to eq(status)
end
Slide 17
Slide 17 text
Basics
And that’s it! Cucumber catches the exceptions that
expect (which comes from the RSpec expectations
gem) throws, and turns them into pretty errors that
explain where your test failed.
Slide 18
Slide 18 text
Directory Structure
Slide 19
Slide 19 text
Directory Structure
Our tests are kept in /automation-tests.
Directory Structure
The Gemfile contains a list of our dependencies,
which are fetched from RubyGems.org.
The Gemfile.lock contains rules about versions for
these, and is auto-generated when a version is
updated.
Slide 22
Slide 22 text
Directory Structure
The projects directory contains the sanity
directory. This is where our tests are kept.
/automation-tests/cucumber/projects/sanity
Slide 23
Slide 23 text
Directory Structure
You can run the tests from here:
$ cucumber
This will run all of the tests.
Slide 24
Slide 24 text
Directory Structure
.
└── features
├── *.feature
└── step_definitions
├── *.rb
└── support
└── *.rb
Slide 25
Slide 25 text
Directory Structure
The Gherkin steps go in files called foobar.feature,
and should be saved into the sanity/features
directory.
Slide 26
Slide 26 text
Directory Structure
The step definitions go in files called foobar.rb, and
should be saved in the
sanity/features/step_definitions directory.
Slide 27
Slide 27 text
Directory Structure
The
sanity/features/step_definitions/support
directory is where test utilities go.
Slide 28
Slide 28 text
Directory Structure
For example, we have api_server.rb, a helper class
that encapsulates the URLs we need to build when
testing across all of our servers.
Slide 29
Slide 29 text
Directory Structure
There is also env.rb, a special Cucumber file that
gets loaded first and contains the global before/after
hooks for the tests.
Any global setup goes in here.
Slide 30
Slide 30 text
Directory Structure
support/widgets contains files that define helpers
suitable for encapsulating the location of certain DOM
nodes and popup windows while testing.
Slide 31
Slide 31 text
Directory Structure
Methods you define in
support/widgets/base_widget.rb are
automatically accessible from your step definitions.
This is thanks to the World(PageWidgets) call at the
bottom of this file.
Slide 32
Slide 32 text
Structure of Tests
Slide 33
Slide 33 text
The steps of a single test are referred to as a
“Scenario”.
Structure of Tests
Slide 34
Slide 34 text
Scenario: Deploying with no internet
Given we have no internet
When we deploy
Then the deployment should fail
And the error should be “the interwebs is
downzorz, oh noes”
And there should be a distinct lack of doge and
catz vidz
Structure of Tests
Slide 35
Slide 35 text
You can have as many Scenarios in a feature file as
you want.
Structure of Tests
Slide 36
Slide 36 text
However, each feature file should contain only related
Scenarios.
For example, one feature file for Social Login, and one
feature file for Public Sharing.
Structure of Tests
Slide 37
Slide 37 text
Scenario Outlines allow you to DRY up related
tests.
Structure of Tests
Slide 38
Slide 38 text
Scenario: Social Login With Facebook
Given I sign in with “facebook”
Then I can see the user status
Scenario: Social Login With Twitter
Given I sign in with “twitter”
Then I can see the user status
Structure of Tests
Slide 39
Slide 39 text
Scenario Outline: Social Login
When I sign in with “”
Then I can see the user status
Examples:
| social_network |
| facebook |
| twitter |
Structure of Tests
Slide 40
Slide 40 text
Tags can be used to group and run tests together.
Structure of Tests
Slide 41
Slide 41 text
@foo
Scenario: One
Given Foo
Then Lux
Structure of Tests
Slide 42
Slide 42 text
Tags can be placed above a Scenario (Outline), or
above the Feature line to apply to all tests in the
feature file.
@foo @bar
Feature: Something
Scenario: ...
Structure of Tests
Slide 43
Slide 43 text
Tags should be dash separated.
@foo_bar # Bad
@foo-bar # Good
Structure of Tests
Slide 44
Slide 44 text
Directory Structure
You can run just certain tags:
$ cucumber -t @foo
This will run all of the tests tagged with @foo.
Slide 45
Slide 45 text
Watir
Slide 46
Slide 46 text
Watir
We use Watir to talk to the browser and interface with
the DOM.
Slide 47
Slide 47 text
Watir
We create a Watir::Browser object and store it at
@browser.
Watir
To find a with a class:
@browser.input(:class => “foobar”)
Slide 50
Slide 50 text
Watir
To find a with a selector:
@browser.element(:css => “body > button:nth-child
(2)”).to_subtype
Slide 51
Slide 51 text
Watir
All of these calls will return an element object. You
can then call methods on these like:
#click - Clicks on the element.
#hover - Hovers over the element.
#present? - Whether the element is in the page.
#wait_until_present - Waits until the element is in the page, up
to our maximum timeout (currently 30s).
Slide 52
Slide 52 text
Watir
Watir has lots of other methods you can use, see the
docs for it or look at our existing automation tests for
more information.
Slide 53
Slide 53 text
Manual of Style
Slide 54
Slide 54 text
Manual of Style
No auto-formatting.
Do not use the auto-formatting tool in IntelliJ yet, it
causes conflicts over formatting changes.
Slide 55
Slide 55 text
Manual of Style
Put newlines between tests.
Slide 56
Slide 56 text
Manual of Style
This is bad:
Scenario: Foo
Given …
Scenario: Bar
Given ...
Slide 57
Slide 57 text
Manual of Style
This is good:
Scenario: Foo
Given …
Scenario: Bar
Given ...
Slide 58
Slide 58 text
Manual of Style
Newline after the Feature line.
Feature: Foo bar lux.
Background:
...
Slide 59
Slide 59 text
Manual of Style
No newlines between the Scenario (Outline) line,
and the first step.
Slide 60
Slide 60 text
Manual of Style
This is good:
Scenario: Foo
Given ...
Slide 61
Slide 61 text
Manual of Style
This is bad:
Scenario: Foo
Given ...
Slide 62
Slide 62 text
Manual of Style
No newline between Scenario Outline steps and
Examples.
Slide 63
Slide 63 text
Manual of Style
This is bad:
Scenario Outline: Foo
Given …
Examples:
| foo | bar |
| ... | ... |
Slide 64
Slide 64 text
Manual of Style
This is good:
Scenario Outline: Foo
Given …
Examples:
| foo | bar |
| ... | ... |
Slide 65
Slide 65 text
Things to Remember
Slide 66
Slide 66 text
Things to Remember
Duplicate step definitions are the achilles heel of
Cucumber tests.
Slide 67
Slide 67 text
Things to Remember
You must always check whether an existing step
definition can do what you need without adding a new
one.
Slide 68
Slide 68 text
Things to Remember
If you aren’t sure, ask somebody to confirm.
Slide 69
Slide 69 text
Things to Remember
When I select the option ""
When I select social-recommendations plugin
When I select the “social-recommendations”
Slide 70
Slide 70 text
Things to Remember
These all did the same thing. They are being replaced
with simply:
When I select the "" plugin from the
plugin list
Slide 71
Slide 71 text
Things to Remember
Wording of step definitions is super important.
When I click on the ""
Slide 72
Slide 72 text
Things to Remember
If people can’t find a step definition, they will just
create a new one, so getting the wording and names of
features right is very important.
Slide 73
Slide 73 text
Things to Remember
Other examples of bad names:
Social Logging # Bad, not the right name
Social Login # Good, the right name
Slide 74
Slide 74 text
Things to Remember
At least three names for the same thing:
IntentHQ Plugin Builder tool # Bad
HQ plugin builder # Bad
plugin builder page # Bad
The plugin builder # Good
Slide 75
Slide 75 text
Things to Remember
Make sure to pay close attention to keep your steps
doing what their verb says they do:
Givens are setup
Whens are actions
Thens are assertions
Slide 76
Slide 76 text
Things to Remember
If one of your steps does more than one of these types
of things, split the step up appropriately.
Slide 77
Slide 77 text
Things to Remember
Remember, you are describing a feature, not a
sequence of actions.
That means that it’s okay to record all of your required
state in Given steps, even if it doesn’t get applied
until a Then executes.
Slide 78
Slide 78 text
Tricks and Tips
Slide 79
Slide 79 text
We have three environment variables we look for when
we run the tests.
TEST_BROWSER
TEST_SERVER
TEST_NO_HEADLESS
Tricks and Tips
Slide 80
Slide 80 text
TEST_BROWSER
Sets the browser to run the tests in. Valid values are
firefox, chrome and safari. Defaults to chrome.
Tricks and Tips
Slide 81
Slide 81 text
TEST_SERVER
Sets the server to run the tests against. Valid values
are local, devn (eg. dev1), staging and live.
Defaults to local.
Tricks and Tips
Slide 82
Slide 82 text
TEST_NO_HEADLESS
If set, the tests will not run in headless mode. This
means you’ll be able to see the browser.
Tricks and Tips
Slide 83
Slide 83 text
You can either export these (perhaps in your .{ba,z}
shrc:
$ export TEST_BROWSER=firefox
$ export TEST_SERVER=dev2
$ export TEST_NO_HEADLESS=true
$ cucumber
Tricks and Tips
Slide 84
Slide 84 text
Or give them inline (they are, after all, just normal old
environment variables):
$ TEST_BROWSER=firefox TEST_SERVER=dev2
TEST_NO_HEADLESS=true cucumber
Tricks and Tips