$30 off During Our Annual Pro Sale. View Details »

The Framework That Knows Its Bugs

apetkova
April 03, 2019

The Framework That Knows Its Bugs

Integrating Java TestNG framework with Atlassian Jira. Presented @TestCon Moscow 2019

apetkova

April 03, 2019
Tweet

More Decks by apetkova

Other Decks in Technology

Transcript

  1. The framework
    that knows its bugs

    View Slide

  2. HELLO!
    I am Aneta Petkova
    QA Chapter Lead @SumUp
    + Coding
    + Agile Development
    + Investigation
    ♡ Dogs ♡
    2

    View Slide

  3. 3
    A better way to get paid.

    View Slide

  4. 4

    View Slide

  5. 5

    View Slide

  6. A Problem
    6
    If a test fails, the build fails.
    As it should be!
    But what happens if a bug is minor? We fix it?
    In ideal world – maybe;
    In reality, we prioritize and compromise.

    View Slide

  7. A Problem
    What people usually do: disable failing tests in the code.
    Like, commenting.
    No test, no problem.
    And then? Then we forget to enable it, of course!
    7

    View Slide

  8. A Problem
    So what if…
    … your framework knew a test is going to fail again?
    … it could find out if you consider it important?
    … and act accordingly?
    8

    View Slide

  9. 9

    View Slide

  10. A Problem
    But we don’t really need that!
    ‘Cause we already have all we need -
    Our bug tracker
    10

    View Slide

  11. Build test case – issue link
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    Solution: The concept
    11

    View Slide

  12. Solution: Java tech stack
    12
    TestNG framework
    RestAssured
    Jackson
    Atlassian JIRA
    Rest API

    View Slide

  13. Solution: Java Tech Stack
    13
    TestNG framework
    RestAssured
    Jackson
    Atlassian JIRA
    Rest API

    View Slide

  14. Implementation: Test Case ←→ Issues Link
    14
    Jira custom field: Create a custom field to hold the affected tests

    View Slide

  15. Implementation: Test Case ←→ Issues Link
    15
    Jira Admin rights are required for adding
    custom fields
    Jira custom field: Create a custom field to hold the
    affected tests

    View Slide

  16. Implementation
    16
    Filter
    Custom
    field
    Lower
    prio
    Active
    status
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*

    View Slide

  17. Implementation
    17
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    Jira JQL: Use saved Jira filter. This will:
    Ø Allow people with no programming skills work on the filter
    Ø Allow filter update without new test code build
    Ø Spare you from building complex JQL queries in your code

    View Slide

  18. Implementation
    18
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    The filter needs to have proper
    permissions

    View Slide

  19. Implementation
    19
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    JIRA REST API: Endpoint used:
    GET /rest/api/2/search/{queryParams}
    Query Parameters:
    Ø JQL expression
    Ø Fields to be returned for each issue

    View Slide

  20. Implementation
    20
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    Authentication:
    Ø Basic
    Ø Have a separate user created for your
    framework

    View Slide

  21. Implementation
    21
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    RestAssured: Make the API call
    response = given().queryParam("jql", "filter=10000")
    .queryParam("fields", "summary", “status”, …)
    .auth().preemptive().basic(username, password)
    .contentType(ContentType.JSON)
    .accept(ContentType.JSON)
    .when().get(jiraUrl);

    View Slide

  22. Implementation
    22
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    Jackson: Unmarshall the response
    ObjectMapper mapper = new ObjectMapper();
    JsonNode node = mapper.readTree(responseBody);
    issues = mapper.readValue(
    node.findValue("issues").toString(),
    JiraIssue[].class);

    View Slide

  23. Implementation
    23
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    TestNG IAnnotationTransformer: Disable certain tests before
    execution start
    @Override
    public void transform(… , Method testMethod){

    if (disable(testId)){
    annotation.setEnabled(false);

    View Slide

  24. Recap
    24
    1. Create custom field in Jira, fill in
    TestIds
    2. Create filter in Jira
    3. Pull issues into test framework
    4. Disable compromised test cases
    5. Get healthy build

    View Slide

  25. DEMO
    TEST DISABLING
    25

    View Slide

  26. Implementation
    26
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    TestNG: Methods that allow actions once a test’s
    execution is finished:
    Ø onTestFailure()
    ØonTestSuccess() from ITestListener

    View Slide

  27. Implementation
    27
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    JIRA: Filter defects resolved within last day
    Pull the issues before tests are run: ISuiteListener

    View Slide

  28. Implementation
    28
    Filter
    defects
    Import
    defects
    Execute
    test set
    Report
    back*
    JIRA endpoints:
    Ø Leave a comment with execution status
    uPOST /rest/api/2/issue/{issueIdOrKey}/comment
    ØUpdate the issue status
    uPOST /rest/api/2/issue/{issueIdOrKey}/transitions

    View Slide

  29. DEMO
    JIRA COMMENT
    29

    View Slide

  30. Potential Hazards
    Ø Allow unhealthy
    builds
    Ø Overrun your bug
    tracker with
    comments
    Ø Open/close defects
    30

    View Slide

  31. Actions
    31
    Ø Allow unhealthy
    builds
    Ø Overrun your bug
    tracker with
    comments
    Ø Open/close defects
    Ø Prioritize defects
    with care
    Ø Check for pre-
    existing comments
    Ø Eliminate flaky
    tests

    View Slide

  32. 32
    Ø Create a task with failures for investigation
    Ø Raise a flag for bugs remained open for too long
    Ø Analise TestId frequency
    Ø Improve test quality
    Ø Do we automate what matters?
    Ø Is single responsibility principle followed?
    NEXT?

    View Slide

  33. View Slide

  34. 34

    View Slide

  35. Thank you!
    Q&A
    /in/apetkova
    [email protected]
    All product names, logos, and images are property of their respective owners
    Presentation template by http://slidescarnival.com
    Slides 4, 5, 12, 13 – images by drawing.llamas
    Slide 32 – image by https://mary147.deviantart.com/
    Graphics: https://freepik.com
    @anetavp
    CREDITS

    View Slide