$30 off During Our Annual Pro Sale. View Details »

Deep Testing

Deep Testing

Filipe Freire

June 27, 2019
Tweet

More Decks by Filipe Freire

Other Decks in Technology

Transcript

  1. FILIPE FREIRE – JUNE 2019
    ADIDAS TECH SUMMIT
    DEEP TESTING

    View Slide

  2. ABOUT ME
    JUN-19
    DEEP TESTING 2
    Filipe Freire – Software Tester.
    Developer. Husband. Learner. Open-
    source software lover and contributor.
    Adidas projects: worked in .com’s
    Embellishment & Customization, Mobile
    App Backend APIs and currently Hype 2.0
    Visitmypersonal blog: filfreire.com

    View Slide

  3. “WE ARE IN THE BUSINESS
    OF WINNING WARS (…)”
    KELSEY GRAMMER AS “GEN. PARTRIDGE” -“THE PENTAGON WARS” 1998

    View Slide

  4. Testing a laser-guided bomb:
    place BOMB above the target

    View Slide

  5. Testing heat seeking missile:
    GLUE tons of hotplates the vehicle

    View Slide

  6. THE PENTAGON WARS
    JUN-19
    DEEP TESTING 6
    “(I wanted the Army to run realistic tests) to see what
    would happen to it (the Bradley) and the people inside
    when it was hit by weapons we could expect on the
    battlefield. The army did not want to run these tests.”
    – James Burton
    Shallow testing
    example

    View Slide

  7. DEEPSEA CHALLENGER
    JUN-19
    DEEP TESTING 7
    Ron Allum:“Our solutions draw upon
    thousands of hours of practical
    experience at sea and at depth.”
    Deep testing
    example

    View Slide

  8. “DEEP SEA” ATTITUDE
    JUN-19
    DEEP TESTING 8
    deep testing: whatever it is I’m testing I
    expect to find every single important bug
    that is out there to find

    View Slide

  9. JUN-19
    DEEP TESTING 9
    “If you don't pursue the questions
    beneath the surface, you generally
    won't obtain your money's worth out of
    your investment in testing.”
    - Gerald M. Weinberg

    View Slide

  10. GOING DEEP
    JUN-19
    DEEP TESTING 10
    IDENTITY
    POSTURE
    OWNERSHIP
    NO
    BULL$%1t

    View Slide

  11. IDENTITY
    JUN-19
    DEEP TESTING 11
    We trust doctors, mechanics,
    plumbers…
    We don’t tell them what/how
    to do.
    As a tester…
    do I own my identity?
    or I’m owned by it?

    View Slide

  12. PRECONCEIVED IDENTITY
    JUN-19
    DEEP TESTING 12
    Imposed/preconceived
    - Gatekeeper
    - I create tons of test
    cases or “Automate all
    the testing and test
    cases”
    - I decide and “give the
    go” for the
    manager/owner
    Self-imposed/self-owned
    - Trustworthy
    - I hunt for meaningful
    risks and bugs, using any
    means necessary
    - I inform the
    manager/owner who is
    paid to decide “go”

    View Slide

  13. POSTURE
    JUN-19
    DEEP TESTING 13
    Am I ready to test…
    Anything?
    Anytime?
    Right now?
    Complex stuff?
    Using any tools?

    View Slide

  14. OWNERSHIP
    JUN-19
    DEEP TESTING 14
    “I am the go-to tester to talk
    about testing done in the project
    I’m with.”
    Do I own: the tools? Envs? Code
    reviews? Bug backlog? …
    Example heuristic:
    “I get logs to help Devs with
    debug info?”

    View Slide

  15. NO BULL$%1T
    JUN-19
    DEEP TESTING 15
    Do I use: the process, the bureaucracy,
    the test cases, the time (or lack of it), the
    environment…
    As an excuse to take zero action?
    Example heuristic:
    Can I prepare, without previous notice, a
    load test for my entire API in under 3 hours?

    View Slide

  16. “WHAT BLINDS MY SIGHT?”
    JUN-19
    DEEP TESTING 16
    - I don’t have test cases! Is that bad?
    - I don’t have all the information! So?
    - I don’t have time! Why?
    - …
    I’m blocked until… I 1) understand and 2) do
    something about it!

    View Slide

  17. JUN-19
    DEEP TESTING 17
    “Sound decisions cannot be made in an
    environment where truth is hard to
    find.”
    - James G. Burton

    View Slide

  18. EATING MY DOG‘S FOOD
    JUN-19
    DEEP TESTING 18
    Q1 (& Q2) 2019
    . Hardcore project and
    timelines.
    . Complex architecture &
    business logic (tons of
    malicious checks).
    . 25 business weeks.
    . Every second counts.
    test strategy: mindmap
    test plan: 5 paragraphs
    0 test cases
    >100 meaningful bugs reported
    >400 avg http req/day
    2-4 merges to “master“ / day
    7 diff api services to test
    Release to prod (few hours)= $ >6 figures

    View Slide

  19. NUMBER OF BUGS?
    JUN-19
    DEEP TESTING 19
    “Counting beans = pointless”
    Every single bug tells a story
    through time. “Bugs are not beans”
    Our prio (testing perspective): find
    and report the most important
    ones in the time given…
    How can I direct my testing to do
    this?
    Eating my
    Dog‘s food

    View Slide

  20. JUN-19
    DEEP TESTING 20
    “(Be careful about) falling victim of
    equating information quality with data
    quantity (bug counts, pages of test
    reports, number of test cases run, and
    so on)”
    - Gerald M. Weinberg

    View Slide

  21. CONTEXT MATTERS!
    JUN-19
    DEEP TESTING 21
    - More critical exploit paths than
    regular purchase flow;
    - Persistence tech changed halfway;
    - Architecture changed halfway;
    - Lost documentation at one point;
    - Constant changes and
    improvements;
    Eating my
    Dog‘s food
    Underlying need to be adaptable.
    (Tools: postman, curl, wget, kubectl, aws cli, jenkins, docker, bash, charles, sublime, git…
    … and our “minds”)

    View Slide

  22. Meaningful
    Testing
    WRAPPING UP
    JUN-19
    DEEP TESTING 22
    NO BULL$%1T
    IDENTITY
    POSTURE
    OWNERSHIP
    Structured
    approach to testing
    Context-adapted
    approach to testing

    View Slide

  23. THANK YOU!
    QUESTIONS?

    View Slide