Gherkin* and Cucumber*

Gherkin* and Cucumber*: a new test case path composition approach to testing Ruby on Rails web applications.

Transcript

1. Gherkin* and Cucumber* A new test case path composition approach
 to testing Ruby on Rails web applications Trento, 15 July 2014 Graduant: Roberto Zen Supervisor: Prof. Maurizio Marchese University of Trento Department of Information Engineering and Computer Science Research carried on at ICT4G
 Bruno Kessler Foundation Co-Supervisor: Prof. Adolfo Villafiorita
2. Introducing BDD in BringTheFood
3. Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “user@example.com” Then I should see “You are now logged in” Given /^I am on the login page$/ do
 visit login_page end ! When /^I sign in as ([\w]*@example.com)$/ do |email|
 fill_in “e-mail” with: email fill_in “password” with: “password” click_on “Sign in” end Step definitions Test cases in Gherkin Feature file 1
 2 3 4 5 6 1
 2 3 4 5 6 7 8 9 step
4. Test cases execution flow Token ::= A | B A ::= aA | B B ::= nil Ragel lexer.rl lexer.rb while true do ! when 0 then … ! if … then goto 9; ! end transition_table.txt + feature file Given /^I am on .. $/ do end ! When /^ I sign in/$ do end The Gherkin gem Output The Cucumber gem Feature step definitions Feature: Login Scenario: Login failed Given I am on .. When I sign in .. Then I should see … parser.rb lexer.scan( transition_table.txt)
5. Limits of tests in BDD Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “user@example.com” Then I should see “Available Offers” ! Scenario: Logout Given I am logged in as “user@example.com” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “user@example.com” And I sign out Then I should be redirected to the login page ! Scenario: A user publishes a new donation Given I am logged in as “user@example.com” When I publish a new donation And I sign out Then I should be redirected to the login page 1
 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
6. Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “user@example.com” Then I should see “Available Offers” ! Scenario: Logout Given I am logged in as “user@example.com” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “user@example.com” And I sign out Then I should be redirected to the login page ! Scenario: A user publishes a new donation Given I am logged in as “user@example.com” When I publish a new donation And I sign out Then I should be redirected to the login page 1
 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Limits of tests in BDD
7. Aim of the thesis Limit the number of duplicate steps following the DRY principle. Generate automatically new test cases. Improve the validation of web applications.
8. Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “user@example.com” Then I should see “Available Offers” Next_Scenario: Logout ! Scenario: Logout Given I am logged in as “user@example.com” When I sign out Then I should be redirected to the login page Test cases in Gherkin* 1
 2 3 4 5 6 7 8 9 10 11 12
9. Graph generation Feature: Example ! Scenario: Login Given I am on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1
 2 3 4 5 6 7 8 9 10 11 12 Scenario: Login
10. Graph generation Scenario: Login Scenario: New Donation Feature: Example ! Scenario: Login Given I am on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1
 2 3 4 5 6 7 8 9 10 11 12
11. Graph generation Scenario: Login Scenario: Logout Feature: Example ! Scenario: Login Given I am on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1
 2 3 4 5 6 7 8 9 10 11 12 Scenario: New Donation
12. Test case path detections Scenario: Login Scenario: Logout Scenario: s1 Scenario: s2 Scenario: s3
13. Scenario: Login Scenario: Logout Total number of scenarios: |scenarios| + |P| Test case path detections P = {! Login→s1→Logout! Login→s2→Logout Login→s3→Logout Login→s2→s1→Logout Login→s2→s3→Logout
 ! } Scenario: s1 Scenario: s2 Scenario: s3
14. The AST built by Cucumber* Scenario: Login Scenario: Logout Scenario: Login Logout Feature: Example Stepi … … Stepn Stepk … … Stepm {Stepi … … Stepn} ∪ {Stepk … … Stepm}
15. Results and Remarks Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “user@example.com” Then I should see “Available Offers” Next_Scenario: New donation Next_Scenario: Logout ! Scenario: New donation Given I am logged in as “user@example.com” When I publish a new donation Then I should see it in the list of donations Next_Scenario: Logout ! Scenario: Logout Given I am logged in as “user@example.com” When I sign out Then I should be redirected to the login page Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “user@example.com” Then I should see “Available Offers” ! Scenario: New donation Given I am logged in as “user@example.com” When I publish a new donation And I sign out Then I should be redirected to the login page ! Scenario: Logout Given I am logged in as “user@example.com” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “user@example.com” And I sign out Then I should be redirected to the login page Scenarios:
 |scenarios| + |P| = 3 + 2 = 5 Scenarios: 4 1
 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
16. Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “user@example.com” Then I should see “Available Offers” ! Scenario: New donation Given I am logged in as “user@example.com” When I publish a new donation Then I should see it in the list of donations ! Scenario: Logout Given I am logged in as “user@example.com” When I sign out Then I should be redirected to the login page ! Scenario: Login→New Donation→Logout! ! ! Given I am on the login page! ! ! When I sign in as “user@example.com”! ! ! Then I should see “Available Offers”! ! ! Given I am on the login page! ! ! When I publish a new donation! ! ! Then I should see it in the list of donations! ! ! Given I am on the login page! ! ! When I sign out! ! ! Then I should be redirected to the login page! ! ! Scenario: Login→Logout! ! ! Given I am on the login page! ! ! When I sign in as “user@example.com”! ! ! Then I should see “Available Offers”! ! ! Given I am on the login page! ! ! When I sign out! ! ! Then I should be redirected to the login page Results and Remarks
17. • A new keyword in the Gherkin’s language. • Three new phases in Cucumber:
 • Graph generation • Test case paths detection • Scenario generation Conclusion
18. • Analyze the performance of Cucumber*. • Find a way to limit multiple Given-When-Then sentences in output. • Overcome the limit of infeasible paths in the generated graph. Future work
19. Thank you