Acceptance Testing With Cucumber

Cucumber is a software acceptance testing tool that has two main goals:
- Improving communication amongst a software development team and with their stakeholders.
- Providing a structured way of testing an application based on the specification to check it meets the acceptance criteria.

This presentation starts by introducing the notions of unit and acceptance testing and what roles they play in software development. Acceptance testing is then looked at in detail, explaining how the Cucumber tool and surrounding philosophy can be used to improve software quality by integrating testing through all stages in a project.

Transcript

1. ACCEPTANCE TESTING
   with
   

   View Slide

2. Outcomes
   • Explain how we can better integrate testing through all
   stages of a project.
   • Show how Cucumber can be used to improve:
   • Client and inter-team communication.
   • Our acceptance testing process.
   • Explain how everyone has a part to play.
   • Encourage open discussion amongst all team members.
   

   View Slide

3. How We Currently Test
   • Application code
   • Informally by simply using the code developers create.
   • Software systems
   • User Acceptance Testing (UAT) after sprints.
   

   View Slide

4. How Can We Improve?
   • Unit Tests for our application code.
   • Structured methodology to testing classes in our code
   base.
   • Cucumber Acceptance Tests for our systems.
   • Current UAT not directly tied to the software specification.
   

   View Slide

5. Kinds Of Testing
   • Unit tests
   • Test the smallest unit of code; objects in OOP.
   • Express how objects should behave.
   • Acceptance tests
   • Test a system’s functionality; web app / mobile app.
   • Express what the software needs to do in order for the
   stakeholder to find it acceptable.
   

   View Slide

6. Kinds Of Testing
   • Unit tests
   • Test the smallest unit of code; objects in OOP.
   • Express how objects should behave.
   • Acceptance tests
   • Test a system’s functionality; web app / mobile app.
   • Express what the software needs to do in order for the
   stakeholder to find it acceptable.
   

   View Slide

7. Testing Differences
   • Acceptance tests “ensure you build the right thing”.
   • Unit tests “ensure you build the thing right”.
   

   View Slide

8. What Is Cucumber?
   • A tool for running automated tests written in plain text.
   • Describe how software should behave as features.
   • Like user stories in agile.
   • A tool for helping communication.
   • Helps build a bridge between the technical and non-
   technical members of a software team.
   

   View Slide

9. Writing Cucumber Specifications
   • Collaboration between the client and the team.
   • Provides a mechanism for feedback.
   • Ideally written before starting development.
   • Helps eradicate misunderstandings & incorrect
   concepts before they reach the code base.
   • Cheaper & more time efficient in the long run.
   

   View Slide

10. Cucumber Features
    features/user_login.feature
    Feature: User login
    ! So I can use the system
    ! As a user who is not currently logged in
    ! I want to login!
    ! Scenario: Successful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with correct details
    ! ! And I click the Login button
    ! ! Then I should be logged in
    ! ! And I should be on the Dashboard page
    Scenario: Unsuccessful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with incorrect details
    ! ! Then I should still be on the Login page
    ! ! And I should see the incorrect Login message
    

    View Slide

11. Describe the feature
    features/user_login.feature
    Feature: User login
    ! So I can use the system
    ! As a user who is not currently logged in
    ! I want to login!
    ! Scenario: Successful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with correct details
    ! ! And I click the Login button
    ! ! Then I should be logged in
    ! ! And I should be on the Dashboard page
    Scenario: Unsuccessful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with incorrect details
    ! ! Then I should still be on the Login page
    ! ! And I should see the incorrect Login message
    

    View Slide

12. Define scenarios
    features/user_login.feature
    Feature: User login
    ! So I can use the system
    ! As a user who is not currently logged in
    ! I want to login
    !
    ! Scenario: Successful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with correct details
    ! ! And I click the Login button
    ! ! Then I should be logged in
    ! ! And I should be on the Dashboard page
    Scenario: Unsuccessful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with incorrect details
    ! ! Then I should still be on the Login page
    ! ! And I should see the incorrect Login message
    

    View Slide

13. Steps; Givens, Whens and Thens
    features/user_login.feature
    Feature: User login
    ! So I can use the system
    ! As a user who is not currently logged in
    ! I want to login!
    ! Scenario: Successful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with correct details
    ! ! And I click the Login button
    ! ! Then I should be logged in
    ! ! And I should be on the Dashboard page
    Scenario: Unsuccessful User login
    Given there is a valid user in the system
    ! ! When I fill in the Login form with incorrect details
    ! ! Then I should still be on the Login page
    ! ! And I should see the incorrect Login message
    

    View Slide

14. features
    describe
    System
    Cucumber tool
    read by
    step
    definitions
    call
    verify
    Report
    generates
    

    View Slide

15. Cucumber Workflow
    • Write feature files defining the specification/scenarios.
    • Describes how us & client think the system should work.
    • Cucumber tool reads the feature files and executes them.
    • Cucumber calls step definitions.
    • Executes the scenarios using a testing tool.
    • Verifies the system meets the acceptance criteria.
    • Cucumber generates a report.
    

    View Slide

16. Running Tests: Step Definitions
    • Can be written in multiple languages: Ruby, JavaScript, Java...
    • A step definition consists of 2 things:
    • A regular expression that matches the cucumber step
    text and identifies arguments.
    • A block of code to execute when the step is
    encountered.
    

    View Slide

17. Step Definition Example
    Cucumber step from features/user_login.feature:
    When I fill in the Login form with correct details
    Ruby step definition step_definitions/user_login.rb:
    When /^I fill in the Login form with correct details$/ do!
    fill_in('username_id', :with => 'kevinbacon01')
    fill_in('password_id', :with => 'password')
    end
    (Using the Capybara test framework for web applications)
    

    View Slide

18. Step Definition Example
    Cucumber step from features/user_login.feature:
    And I click the Login button
    Ruby step definition step_definitions/user_login.rb:
    And /^I click the Login button$/ do!
    click_button('login_id')
    end
    (Using the Capybara test framework for web applications)
    

    View Slide

19. Step Definition Example
    Cucumber step from features/user_login.feature:
    And I should see the incorrect Login message
    Ruby step definition step_definitions/user_login.rb:
    And /^I should see the incorrect Login message $/ do!
    page.should have_content('Incorrect credentials')
    end
    (Using the Capybara test framework for web applications)
    

    View Slide

20. Running: tests passing
    $ cucumber
    Feature: User login
    So I can use the system
    As a user who is not currently logged in
    I want to login
    Scenario: Successful User login # features/user_login.feature:6
    Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1
    When I fill in the Login form with correct details # features/step_definitions/user_login_steps.rb:5
    And I click the Login button # features/step_definitions/user_login_steps.rb:9
    Then I should be logged in # features/step_definitions/user_login_steps.rb:13
    And I should be on the Dashboard page # features/step_definitions/user_login_steps.rb:17
    Scenario: Unsuccessful User login # features/user_login.feature:13
    Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1
    When I fill in the Login form with incorrect details # features/step_definitions/user_login_steps.rb:21
    Then I should still be on the Login page # features/step_definitions/user_login_steps.rb:25
    And I should see the incorrect Login message # features/step_definitions/user_login_steps.rb:29
    2 scenarios (2 passed)
    9 steps (9 passed)
    0m6.036s
    

    View Slide

21. Running: tests failing
    $ cucumber
    Feature: User login
    So I can use the system
    As a user who is not currently logged in
    I want to login
    Scenario: Successful User login # features/user_login.feature:6
    Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1
    When I fill in the Login form with correct details # features/step_definitions/user_login_steps.rb:5
    And I click the Login button # features/step_definitions/user_login_steps.rb:9
    Then I should be logged in # features/step_definitions/user_login_steps.rb:13
    And I should be on the Dashboard page # features/step_definitions/user_login_steps.rb:17
    Scenario: Unsuccessful User login # features/user_login.feature:13
    Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1
    When I fill in the Login form with incorrect details # features/step_definitions/user_login_steps.rb:21
    Then I should still be on the Login page # features/step_definitions/user_login_steps.rb:25
    And I should see the incorrect Login message # features/step_definitions/user_login_steps.rb:29
    expected: true value
    got: false (RSpec::Expectations::ExpectationNotMetError)
    ./features/step_definitions/user_login_steps.rb:31:in `/^I should see the incorrect Login message$/'
    features/user_login.feature:17:in `And I should see the incorrect Login message'
    Failing Scenarios:
    cucumber features/user_login.feature:13 # Scenario: Unsuccessful User login
    2 scenarios (1 failed, 1 passed)
    9 steps (1 failed, 8 passed)
    0m5.909s
    

    View Slide

22. Advantages Of Cucumber
    • Generated report directly maps to the specification.
    • Features are a true representation of what the system
    does/does not do.
    • Up to date documentation.
    • Same business features for different platforms.
    • No need for duplicate specifications for web, mobile etc.
    • Simply swap out the automation tool.
    

    View Slide

23. Final Thoughts:
    How Much Testing?
    • Testing can never be (truly) complete.
    • Test the most important/risky components first, then the
    next most risky, then the next most risky, etc.
    • Continue until the amount of risk remaining isn’t worth
    spending time & money addressing.
    • End goal: the customer can see that the software does what it
    ought to, and therefore is worth paying for.
    

    View Slide

24. More Information...
    http://cukes.info https://github.com/cucumber/cucumber
    https://github.com/jnicklas/capybara
    tryruby.org/
    

    View Slide

25. Questions?
    

    View Slide