Tweet
Tweet

Slide 1

Slide 1 text

The framework that knows its bugs

Slide 2

Slide 2 text

HELLO! I am Aneta Petkova QA Chapter Lead @SumUp + Coding + Agile Development + Investigation ♡ Dogs ♡ 2

Slide 3

Slide 3 text

3 A better way to get paid.

Slide 4

Slide 4 text

4

Slide 5

Slide 5 text

5

Slide 6

Slide 6 text

A Problem 6 If a test fails, the build fails. As it should be! But what happens if a bug is minor? We fix it? In ideal world – maybe; In reality, we prioritize and compromise.

Slide 7

Slide 7 text

A Problem What people usually do: disable failing tests in the code. Like, commenting. No test, no problem. And then? Then we forget to enable it, of course! 7

Slide 8

Slide 8 text

A Problem So what if… … your framework knew a test is going to fail again? … it could find out if you consider it important? … and act accordingly? 8

Slide 9

Slide 9 text

9

Slide 10

Slide 10 text

A Problem But we don’t really need that! ‘Cause we already have all we need - Our bug tracker 10

Slide 11

Slide 11 text

Build test case – issue link Filter defects Import defects Execute test set Report back* Solution: The concept 11

Slide 12

Slide 12 text

Solution: Java tech stack 12 TestNG framework RestAssured Jackson Atlassian JIRA Rest API

Slide 13

Slide 13 text

Solution: Java Tech Stack 13 TestNG framework RestAssured Jackson Atlassian JIRA Rest API

Slide 14

Slide 14 text

Implementation: Test Case ←→ Issues Link 14 Jira custom field: Create a custom field to hold the affected tests

Slide 15

Slide 15 text

Implementation: Test Case ←→ Issues Link 15 Jira Admin rights are required for adding custom fields Jira custom field: Create a custom field to hold the affected tests

Slide 16

Slide 16 text

Implementation 16 Filter Custom field Lower prio Active status Filter defects Import defects Execute test set Report back*

Slide 17

Slide 17 text

Implementation 17 Filter defects Import defects Execute test set Report back* Jira JQL: Use saved Jira filter. This will: Ø Allow people with no programming skills work on the filter Ø Allow filter update without new test code build Ø Spare you from building complex JQL queries in your code

Slide 18

Slide 18 text

Implementation 18 Filter defects Import defects Execute test set Report back* The filter needs to have proper permissions

Slide 19

Slide 19 text

Implementation 19 Filter defects Import defects Execute test set Report back* JIRA REST API: Endpoint used: GET /rest/api/2/search/{queryParams} Query Parameters: Ø JQL expression Ø Fields to be returned for each issue

Slide 20

Slide 20 text

Implementation 20 Filter defects Import defects Execute test set Report back* Authentication: Ø Basic Ø Have a separate user created for your framework

Slide 21

Slide 21 text

Implementation 21 Filter defects Import defects Execute test set Report back* RestAssured: Make the API call response = given().queryParam("jql", "filter=10000") .queryParam("fields", "summary", “status”, …) .auth().preemptive().basic(username, password) .contentType(ContentType.JSON) .accept(ContentType.JSON) .when().get(jiraUrl);

Slide 22

Slide 22 text

Implementation 22 Filter defects Import defects Execute test set Report back* Jackson: Unmarshall the response ObjectMapper mapper = new ObjectMapper(); JsonNode node = mapper.readTree(responseBody); issues = mapper.readValue( node.findValue("issues").toString(), JiraIssue[].class);

Slide 23

Slide 23 text

Implementation 23 Filter defects Import defects Execute test set Report back* TestNG IAnnotationTransformer: Disable certain tests before execution start @Override public void transform(… , Method testMethod){ … if (disable(testId)){ annotation.setEnabled(false);

Slide 24

Slide 24 text

Recap 24 1. Create custom field in Jira, fill in TestIds 2. Create filter in Jira 3. Pull issues into test framework 4. Disable compromised test cases 5. Get healthy build

Slide 25

Slide 25 text

DEMO TEST DISABLING 25

Slide 26

Slide 26 text

Implementation 26 Filter defects Import defects Execute test set Report back* TestNG: Methods that allow actions once a test’s execution is finished: Ø onTestFailure() ØonTestSuccess() from ITestListener

Slide 27

Slide 27 text

Implementation 27 Filter defects Import defects Execute test set Report back* JIRA: Filter defects resolved within last day Pull the issues before tests are run: ISuiteListener

Slide 28

Slide 28 text

Implementation 28 Filter defects Import defects Execute test set Report back* JIRA endpoints: Ø Leave a comment with execution status uPOST /rest/api/2/issue/{issueIdOrKey}/comment ØUpdate the issue status uPOST /rest/api/2/issue/{issueIdOrKey}/transitions

Slide 29

Slide 29 text

DEMO JIRA COMMENT 29

Slide 30

Slide 30 text

Potential Hazards Ø Allow unhealthy builds Ø Overrun your bug tracker with comments Ø Open/close defects 30

Slide 31

Slide 31 text

Actions 31 Ø Allow unhealthy builds Ø Overrun your bug tracker with comments Ø Open/close defects Ø Prioritize defects with care Ø Check for pre- existing comments Ø Eliminate flaky tests

Slide 32

Slide 32 text

32 Ø Create a task with failures for investigation Ø Raise a flag for bugs remained open for too long Ø Analise TestId frequency Ø Improve test quality Ø Do we automate what matters? Ø Is single responsibility principle followed? NEXT?

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

34

Slide 35

Slide 35 text

Thank you! Q&A /in/apetkova aneta.petkova@sumup.com All product names, logos, and images are property of their respective owners Presentation template by http://slidescarnival.com Slides 4, 5, 12, 13 – images by drawing.llamas Slide 32 – image by https://mary147.deviantart.com/ Graphics: https://freepik.com @anetavp CREDITS