Acceptance Testing With Cucumber

Cucumber is a software acceptance testing tool that has two main goals:
- Improving communication amongst a software development team and with their stakeholders.
- Providing a structured way of testing an application based on the specification to check it meets the acceptance criteria.

This presentation starts by introducing the notions of unit and acceptance testing and what roles they play in software development. Acceptance testing is then looked at in detail, explaining how the Cucumber tool and surrounding philosophy can be used to improve software quality by integrating testing through all stages in a project.

Transcript

1. ACCEPTANCE TESTING with
2. Outcomes • Explain how we can better integrate testing through all stages of a project. • Show how Cucumber can be used to improve: • Client and inter-team communication. • Our acceptance testing process. • Explain how everyone has a part to play. • Encourage open discussion amongst all team members.
3. How We Currently Test • Application code • Informally by simply using the code developers create. • Software systems • User Acceptance Testing (UAT) after sprints.
4. How Can We Improve? • Unit Tests for our application code. • Structured methodology to testing classes in our code base. • Cucumber Acceptance Tests for our systems. • Current UAT not directly tied to the software specification.
5. Kinds Of Testing • Unit tests • Test the smallest unit of code; objects in OOP. • Express how objects should behave. • Acceptance tests • Test a system’s functionality; web app / mobile app. • Express what the software needs to do in order for the stakeholder to find it acceptable.
6. Kinds Of Testing • Unit tests • Test the smallest unit of code; objects in OOP. • Express how objects should behave. • Acceptance tests • Test a system’s functionality; web app / mobile app. • Express what the software needs to do in order for the stakeholder to find it acceptable.
7. Testing Differences • Acceptance tests “ensure you build the right thing”. • Unit tests “ensure you build the thing right”.
8. What Is Cucumber? • A tool for running automated tests written in plain text. • Describe how software should behave as features. • Like user stories in agile. • A tool for helping communication. • Helps build a bridge between the technical and non- technical members of a software team.
9. Writing Cucumber Specifications • Collaboration between the client and the team. • Provides a mechanism for feedback. • Ideally written before starting development. • Helps eradicate misunderstandings & incorrect concepts before they reach the code base. • Cheaper & more time efficient in the long run.
10. Cucumber Features features/user_login.feature Feature: User login ! So I can use the system ! As a user who is not currently logged in ! I want to login! ! Scenario: Successful User login Given there is a valid user in the system ! ! When I fill in the Login form with correct details ! ! And I click the Login button ! ! Then I should be logged in ! ! And I should be on the Dashboard page Scenario: Unsuccessful User login Given there is a valid user in the system ! ! When I fill in the Login form with incorrect details ! ! Then I should still be on the Login page ! ! And I should see the incorrect Login message
11. Describe the feature features/user_login.feature Feature: User login ! So I can use the system ! As a user who is not currently logged in ! I want to login! ! Scenario: Successful User login Given there is a valid user in the system ! ! When I fill in the Login form with correct details ! ! And I click the Login button ! ! Then I should be logged in ! ! And I should be on the Dashboard page Scenario: Unsuccessful User login Given there is a valid user in the system ! ! When I fill in the Login form with incorrect details ! ! Then I should still be on the Login page ! ! And I should see the incorrect Login message
12. Define scenarios features/user_login.feature Feature: User login ! So I can use the system ! As a user who is not currently logged in ! I want to login ! ! Scenario: Successful User login Given there is a valid user in the system ! ! When I fill in the Login form with correct details ! ! And I click the Login button ! ! Then I should be logged in ! ! And I should be on the Dashboard page Scenario: Unsuccessful User login Given there is a valid user in the system ! ! When I fill in the Login form with incorrect details ! ! Then I should still be on the Login page ! ! And I should see the incorrect Login message
13. Steps; Givens, Whens and Thens features/user_login.feature Feature: User login ! So I can use the system ! As a user who is not currently logged in ! I want to login! ! Scenario: Successful User login Given there is a valid user in the system ! ! When I fill in the Login form with correct details ! ! And I click the Login button ! ! Then I should be logged in ! ! And I should be on the Dashboard page Scenario: Unsuccessful User login Given there is a valid user in the system ! ! When I fill in the Login form with incorrect details ! ! Then I should still be on the Login page ! ! And I should see the incorrect Login message
14. features describe System Cucumber tool read by step definitions call verify Report generates
15. Cucumber Workflow • Write feature files defining the specification/scenarios. • Describes how us & client think the system should work. • Cucumber tool reads the feature files and executes them. • Cucumber calls step definitions. • Executes the scenarios using a testing tool. • Verifies the system meets the acceptance criteria. • Cucumber generates a report.
16. Running Tests: Step Definitions • Can be written in multiple languages: Ruby, JavaScript, Java... • A step definition consists of 2 things: • A regular expression that matches the cucumber step text and identifies arguments. • A block of code to execute when the step is encountered.
17. Step Definition Example Cucumber step from features/user_login.feature: When I fill in the Login form with correct details Ruby step definition step_definitions/user_login.rb: When /^I fill in the Login form with correct details$/ do! fill_in('username_id', :with => 'kevinbacon01') fill_in('password_id', :with => 'password') end (Using the Capybara test framework for web applications)
18. Step Definition Example Cucumber step from features/user_login.feature: And I click the Login button Ruby step definition step_definitions/user_login.rb: And /^I click the Login button$/ do! click_button('login_id') end (Using the Capybara test framework for web applications)
19. Step Definition Example Cucumber step from features/user_login.feature: And I should see the incorrect Login message Ruby step definition step_definitions/user_login.rb: And /^I should see the incorrect Login message $/ do! page.should have_content('Incorrect credentials') end (Using the Capybara test framework for web applications)
20. Running: tests passing $ cucumber Feature: User login So I can use the system As a user who is not currently logged in I want to login Scenario: Successful User login # features/user_login.feature:6 Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1 When I fill in the Login form with correct details # features/step_definitions/user_login_steps.rb:5 And I click the Login button # features/step_definitions/user_login_steps.rb:9 Then I should be logged in # features/step_definitions/user_login_steps.rb:13 And I should be on the Dashboard page # features/step_definitions/user_login_steps.rb:17 Scenario: Unsuccessful User login # features/user_login.feature:13 Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1 When I fill in the Login form with incorrect details # features/step_definitions/user_login_steps.rb:21 Then I should still be on the Login page # features/step_definitions/user_login_steps.rb:25 And I should see the incorrect Login message # features/step_definitions/user_login_steps.rb:29 2 scenarios (2 passed) 9 steps (9 passed) 0m6.036s
21. Running: tests failing $ cucumber Feature: User login So I can use the system As a user who is not currently logged in I want to login Scenario: Successful User login # features/user_login.feature:6 Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1 When I fill in the Login form with correct details # features/step_definitions/user_login_steps.rb:5 And I click the Login button # features/step_definitions/user_login_steps.rb:9 Then I should be logged in # features/step_definitions/user_login_steps.rb:13 And I should be on the Dashboard page # features/step_definitions/user_login_steps.rb:17 Scenario: Unsuccessful User login # features/user_login.feature:13 Given there is a valid user in the system # features/step_definitions/user_login_steps.rb:1 When I fill in the Login form with incorrect details # features/step_definitions/user_login_steps.rb:21 Then I should still be on the Login page # features/step_definitions/user_login_steps.rb:25 And I should see the incorrect Login message # features/step_definitions/user_login_steps.rb:29 expected: true value got: false (RSpec::Expectations::ExpectationNotMetError) ./features/step_definitions/user_login_steps.rb:31:in `/^I should see the incorrect Login message$/' features/user_login.feature:17:in `And I should see the incorrect Login message' Failing Scenarios: cucumber features/user_login.feature:13 # Scenario: Unsuccessful User login 2 scenarios (1 failed, 1 passed) 9 steps (1 failed, 8 passed) 0m5.909s
22. Advantages Of Cucumber • Generated report directly maps to the specification. • Features are a true representation of what the system does/does not do. • Up to date documentation. • Same business features for different platforms. • No need for duplicate specifications for web, mobile etc. • Simply swap out the automation tool.
23. Final Thoughts: How Much Testing? • Testing can never be (truly) complete. • Test the most important/risky components first, then the next most risky, then the next most risky, etc. • Continue until the amount of risk remaining isn’t worth spending time & money addressing. • End goal: the customer can see that the software does what it ought to, and therefore is worth paying for.
24. More Information... http://cukes.info https://github.com/cucumber/cucumber https://github.com/jnicklas/capybara tryruby.org/
25. Questions?