The Framework That Knows Its Bugs

Integrating Java TestNG framework with Atlassian Jira. Presented @TestCon Moscow 2019

Transcript

1. The framework that knows its bugs
2. HELLO! I am Aneta Petkova QA Chapter Lead @SumUp + Coding + Agile Development + Investigation ♡ Dogs ♡ 2
3. 3 A better way to get paid.
6. A Problem 6 If a test fails, the build fails. As it should be! But what happens if a bug is minor? We fix it? In ideal world – maybe; In reality, we prioritize and compromise.
7. A Problem What people usually do: disable failing tests in the code. Like, commenting. No test, no problem. And then? Then we forget to enable it, of course! 7
8. A Problem So what if… … your framework knew a test is going to fail again? … it could find out if you consider it important? … and act accordingly? 8
10. A Problem But we don’t really need that! ‘Cause we already have all we need - Our bug tracker 10
11. Build test case – issue link Filter defects Import defects Execute test set Report back* Solution: The concept 11
12. Solution: Java tech stack 12 TestNG framework RestAssured Jackson Atlassian JIRA Rest API
13. Solution: Java Tech Stack 13 TestNG framework RestAssured Jackson Atlassian JIRA Rest API
14. Implementation: Test Case ←→ Issues Link 14 Jira custom field: Create a custom field to hold the affected tests
15. Implementation: Test Case ←→ Issues Link 15 Jira Admin rights are required for adding custom fields Jira custom field: Create a custom field to hold the affected tests
16. Implementation 16 Filter Custom field Lower prio Active status Filter defects Import defects Execute test set Report back*
17. Implementation 17 Filter defects Import defects Execute test set Report back* Jira JQL: Use saved Jira filter. This will: Ø Allow people with no programming skills work on the filter Ø Allow filter update without new test code build Ø Spare you from building complex JQL queries in your code
18. Implementation 18 Filter defects Import defects Execute test set Report back* The filter needs to have proper permissions
19. Implementation 19 Filter defects Import defects Execute test set Report back* JIRA REST API: Endpoint used: GET /rest/api/2/search/{queryParams} Query Parameters: Ø JQL expression Ø Fields to be returned for each issue
20. Implementation 20 Filter defects Import defects Execute test set Report back* Authentication: Ø Basic Ø Have a separate user created for your framework
21. Implementation 21 Filter defects Import defects Execute test set Report back* RestAssured: Make the API call response = given().queryParam("jql", "filter=10000") .queryParam("fields", "summary", “status”, …) .auth().preemptive().basic(username, password) .contentType(ContentType.JSON) .accept(ContentType.JSON) .when().get(jiraUrl);
22. Implementation 22 Filter defects Import defects Execute test set Report back* Jackson: Unmarshall the response ObjectMapper mapper = new ObjectMapper(); JsonNode node = mapper.readTree(responseBody); issues = mapper.readValue( node.findValue("issues").toString(), JiraIssue[].class);
23. Implementation 23 Filter defects Import defects Execute test set Report back* TestNG IAnnotationTransformer: Disable certain tests before execution start @Override public void transform(… , Method testMethod){ … if (disable(testId)){ annotation.setEnabled(false);
24. Recap 24 1. Create custom field in Jira, fill in TestIds 2. Create filter in Jira 3. Pull issues into test framework 4. Disable compromised test cases 5. Get healthy build
25. DEMO TEST DISABLING 25
26. Implementation 26 Filter defects Import defects Execute test set Report back* TestNG: Methods that allow actions once a test’s execution is finished: Ø onTestFailure() ØonTestSuccess() from ITestListener
27. Implementation 27 Filter defects Import defects Execute test set Report back* JIRA: Filter defects resolved within last day Pull the issues before tests are run: ISuiteListener
28. Implementation 28 Filter defects Import defects Execute test set Report back* JIRA endpoints: Ø Leave a comment with execution status uPOST /rest/api/2/issue/{issueIdOrKey}/comment ØUpdate the issue status uPOST /rest/api/2/issue/{issueIdOrKey}/transitions
29. DEMO JIRA COMMENT 29
30. Potential Hazards Ø Allow unhealthy builds Ø Overrun your bug tracker with comments Ø Open/close defects 30
31. Actions 31 Ø Allow unhealthy builds Ø Overrun your bug tracker with comments Ø Open/close defects Ø Prioritize defects with care Ø Check for pre- existing comments Ø Eliminate flaky tests
32. 32 Ø Create a task with failures for investigation Ø Raise a flag for bugs remained open for too long Ø Analise TestId frequency Ø Improve test quality Ø Do we automate what matters? Ø Is single responsibility principle followed? NEXT?
35. Thank you! Q&A /in/apetkova aneta.petkova@sumup.com All product names, logos, and images are property of their respective owners Presentation template by http://slidescarnival.com Slides 4, 5, 12, 13 – images by drawing.llamas Slide 32 – image by https://mary147.deviantart.com/ Graphics: https://freepik.com @anetavp CREDITS