The Framework That Knows Its Bugs

E3076b26dd2cd384939694fa5a00948e?s=47 apetkova
April 03, 2019

The Framework That Knows Its Bugs

Integrating Java TestNG framework with Atlassian Jira. Presented @TestCon Moscow 2019

E3076b26dd2cd384939694fa5a00948e?s=128

apetkova

April 03, 2019
Tweet

Transcript

  1. The framework that knows its bugs

  2. HELLO! I am Aneta Petkova QA Chapter Lead @SumUp +

    Coding + Agile Development + Investigation ♡ Dogs ♡ 2
  3. 3 A better way to get paid.

  4. 4

  5. 5

  6. A Problem 6 If a test fails, the build fails.

    As it should be! But what happens if a bug is minor? We fix it? In ideal world – maybe; In reality, we prioritize and compromise.
  7. A Problem What people usually do: disable failing tests in

    the code. Like, commenting. No test, no problem. And then? Then we forget to enable it, of course! 7
  8. A Problem So what if… … your framework knew a

    test is going to fail again? … it could find out if you consider it important? … and act accordingly? 8
  9. 9

  10. A Problem But we don’t really need that! ‘Cause we

    already have all we need - Our bug tracker 10
  11. Build test case – issue link Filter defects Import defects

    Execute test set Report back* Solution: The concept 11
  12. Solution: Java tech stack 12 TestNG framework RestAssured Jackson Atlassian

    JIRA Rest API
  13. Solution: Java Tech Stack 13 TestNG framework RestAssured Jackson Atlassian

    JIRA Rest API
  14. Implementation: Test Case ←→ Issues Link 14 Jira custom field:

    Create a custom field to hold the affected tests
  15. Implementation: Test Case ←→ Issues Link 15 Jira Admin rights

    are required for adding custom fields Jira custom field: Create a custom field to hold the affected tests
  16. Implementation 16 Filter Custom field Lower prio Active status Filter

    defects Import defects Execute test set Report back*
  17. Implementation 17 Filter defects Import defects Execute test set Report

    back* Jira JQL: Use saved Jira filter. This will: Ø Allow people with no programming skills work on the filter Ø Allow filter update without new test code build Ø Spare you from building complex JQL queries in your code
  18. Implementation 18 Filter defects Import defects Execute test set Report

    back* The filter needs to have proper permissions
  19. Implementation 19 Filter defects Import defects Execute test set Report

    back* JIRA REST API: Endpoint used: GET /rest/api/2/search/{queryParams} Query Parameters: Ø JQL expression Ø Fields to be returned for each issue
  20. Implementation 20 Filter defects Import defects Execute test set Report

    back* Authentication: Ø Basic Ø Have a separate user created for your framework
  21. Implementation 21 Filter defects Import defects Execute test set Report

    back* RestAssured: Make the API call response = given().queryParam("jql", "filter=10000") .queryParam("fields", "summary", “status”, …) .auth().preemptive().basic(username, password) .contentType(ContentType.JSON) .accept(ContentType.JSON) .when().get(jiraUrl);
  22. Implementation 22 Filter defects Import defects Execute test set Report

    back* Jackson: Unmarshall the response ObjectMapper mapper = new ObjectMapper(); JsonNode node = mapper.readTree(responseBody); issues = mapper.readValue( node.findValue("issues").toString(), JiraIssue[].class);
  23. Implementation 23 Filter defects Import defects Execute test set Report

    back* TestNG IAnnotationTransformer: Disable certain tests before execution start @Override public void transform(… , Method testMethod){ … if (disable(testId)){ annotation.setEnabled(false);
  24. Recap 24 1. Create custom field in Jira, fill in

    TestIds 2. Create filter in Jira 3. Pull issues into test framework 4. Disable compromised test cases 5. Get healthy build
  25. DEMO TEST DISABLING 25

  26. Implementation 26 Filter defects Import defects Execute test set Report

    back* TestNG: Methods that allow actions once a test’s execution is finished: Ø onTestFailure() ØonTestSuccess() from ITestListener
  27. Implementation 27 Filter defects Import defects Execute test set Report

    back* JIRA: Filter defects resolved within last day Pull the issues before tests are run: ISuiteListener
  28. Implementation 28 Filter defects Import defects Execute test set Report

    back* JIRA endpoints: Ø Leave a comment with execution status uPOST /rest/api/2/issue/{issueIdOrKey}/comment ØUpdate the issue status uPOST /rest/api/2/issue/{issueIdOrKey}/transitions
  29. DEMO JIRA COMMENT 29

  30. Potential Hazards Ø Allow unhealthy builds Ø Overrun your bug

    tracker with comments Ø Open/close defects 30
  31. Actions 31 Ø Allow unhealthy builds Ø Overrun your bug

    tracker with comments Ø Open/close defects Ø Prioritize defects with care Ø Check for pre- existing comments Ø Eliminate flaky tests
  32. 32 Ø Create a task with failures for investigation Ø

    Raise a flag for bugs remained open for too long Ø Analise TestId frequency Ø Improve test quality Ø Do we automate what matters? Ø Is single responsibility principle followed? NEXT?
  33. None
  34. 34

  35. Thank you! Q&A /in/apetkova aneta.petkova@sumup.com All product names, logos, and

    images are property of their respective owners Presentation template by http://slidescarnival.com Slides 4, 5, 12, 13 – images by drawing.llamas Slide 32 – image by https://mary147.deviantart.com/ Graphics: https://freepik.com @anetavp CREDITS