Automated Acceptance Testing - with Cucumber and Espresso

There's no doubt, Google did a pretty nice job and testing finally has become an integrated part of Android development. A lot of material exists on HOW developers could test their mobile applications using a variety of tools.

However, working on large scale enterprise projects requires to think one step further. (Domain) knowledge might be spread across the team and even if developers are now aware of HOW to test their code, sometimes guidance is required WHAT should be tested in particular.

This talk presents the popular (BDD) tool Cucumber and showcases how to use it for Android development. The talk is motivated by the aforementioned demands and provides a rough introduction to automated acceptance testing on Android using Cucumber in conjunction with Espresso. One conclusion is that automated acceptance testing provides a very quick test coverage of an application. It helps furthermore to keep documentation in sync and allows a dedicated test-team to write tests which must not necessarily be part of the actual project team.

Transcript

1. Automated Acceptance Testing with Cucumber and Espresso
2. Example Do you know what to test? prefill last logged-in username notify if username not set notify if password not set notify if invalid credentials display indicator while validating notify if too many attempts show “Forgot password?” en-/disable text fields and button navigate to start screen show timer when blocked after too many attempts … …
3. What? Acceptance testing “Build the right thing” Unit testing “Build the thing right” vs BDD TDD
4. Why? Motivation • having end-to-end tests − identify issues not covered by our unit tests − perform architectural-level refactorings • having a long-term test specification − due to evolution in Android testing we had to rewrite our tests a couple of times (Robotium → Robolectric → Espresso) − a high domain specific knowledge is required to write tests
5. Remember? Testing Quality “eliminating bugs”
6. Remember? Testing Design Quality “avoiding bugs”
7. How? Cucumber • tool for executable specifications • available for many major languages • tests readable/writeable for everyone • popular and vibrant community • …
8. How? Gherkin • a business readable, domain specific language • used for documentation and automated testing • understood by Cucumber • line oriented and structured by indentation • …
9. Example Gherkin Feature: Authentication In order to protect sensitive data, users have to login with their name and password. Scenario: Invalid credentials Given I entered my credentials When I click login Then I should see “invalid credentials”
10. Example Gherkin Feature: Authentication In order to protect sensitive data, users have to login with their name and password. Scenario: Invalid credentials Given I entered “andre” as username And I entered “secret” as password When I click login Then I should see “invalid credentials”
11. Example Gherkin Feature: Authentication In order to protect sensitive data, users have to login with their name and password. Scenario: Invalid credentials Given I entered “andre” as username And I entered “secret” as password When I click login Then I should see “invalid credentials” Scenario: Valid credentials … Scenario: No username … Scenario: No password …
12. Example Gherkin Feature: Authentication In order to protect sensitive data, users have to login with their name and password. Scenario Outline: Validate credentials Given I entered “<user>” as username And I entered “<pass>” as password When I click login Then I should see “<msg>” Examples: |user |pass |msg | |andre |wrong |Invalid password.| |andre |secret |Welcome. | | |pass |No username. | |user | |No password. |
13. Gherkin? Keywords Feature Background Scenario Scenario Outline Given When Then And But * Examples
14. Gherkin? Features vs. Steps Features: • specify what to test • are written in Gherkin Steps: • specify how to test • are written in any language supported by Cucumber
15. Example Steps Feature: Authentication In order to protect sensitive data, users have to login with their name and password. Scenario Outline: Validate credentials Given I entered “<user>” as username And I entered “<pass>” as password When I click login Then I should see “<msg>” Examples: |user |pass |msg | |andre |wrong |Invalid password.| |andre |secret |Welcome. | | |pass |No username. | |user | |No password. | @Given("^I entered \"(.+)\" as (.+)$") public void I_entered_(String text, String as) { //perform action on UI } @Given("^I click (.+)$") public void I_click_on_(String text) { //perform action on UI } @Given("^I should see \"(.+)\"$") public void I_should_see_(String text) { //perform assertion on UI }
16. How? Espresso
17. How? Espresso
18. How? Espresso
19. Example Steps Feature: Authentication In order to protect sensitive data, users have to login with their name and password. Scenario Outline: Validate credentials Given I entered “<user>” as username And I entered “<pass>” as password When I click login Then I should see “<msg>” Examples: |user |pass |msg | |andre |wrong |Invalid password.| |andre |secret |Welcome. | | |pass |No username. | |user | |No password. | @Given("^I entered \"(.+)\" as (.+)$") public void I_entered_(String text, String as) { onView(withHint(as)).perform(typeText(text)); } @Given("^I click (.+)$") public void I_click_on_(String text) { onView(withText(text)).perform(click()); } @Given("^I should see \"(.+)\"$") public void I_should_see_(String text) { onView(withText(text)).check( matches(isDisplayed())); }
20. DEMO http://github.com/a11n/android-cucumber-espresso
21. BDD? Ideal vs Pragmatic • write features with customer • single source of truth • living documentation • … • specify what to test • focus on important use cases OR start each screen at least once • incorporate into build mgmt. • …
22. #ProTip Dry-Run • parses *.feature files without executing tests • discovers syntax issues • prints method stubs write features → dry-run → copy stubs → implement steps
23. Start? Resources • Web: − http://cucumber.io − https://github.com/cucumber/cucumber- jvm/tree/master/examples • Print: − The Cucumber for Java Book − Cucumber Recipes
24. Conclusion? Key aspects • Cucumber is a popular tool for BDD and automated acceptance testing • Gherkin is a business readable, domain specific language • Features specify what to test – steps specify how to test • Acceptance tests allow a quick and comprehensive test coverage