$30 off During Our Annual Pro Sale. View Details »

Seems Good Enough to Me: Working with Testers to Derisk Upgrades

Seems Good Enough to Me: Working with Testers to Derisk Upgrades

It’s that time again! You have a legacy app running business critical functions and it’s on an old version of Elixir. You need to upgrade to stay safe, have access to the most modern features of the ecosystem, and be current with your community. But a full regression of the app will take days if not weeks and no one can find the time. What’s a team to do?

Jenny Bramble suggests leaning on the deep magic of test methodology and collaboration to determine the minimum viable tests to run in order to ensure your system is preforming as expected. She’ll talk about working through release notes with your testers as well as applying test methodologies to the types of problems we see in upgrades and regressions. You’ll walk away from this talk with a stronger understanding of the ways we can simplify and derisk upgrades and a structure for working with test to make sure that we’re doing the right amount of testing.

Jenny Bramble

September 01, 2022
Tweet

More Decks by Jenny Bramble

Other Decks in Programming

Transcript

  1. Seems Good
    Enough to Me
    @jennydoesthings
    Working With Testers to
    Derisk Elixir Upgrades

    View Slide

  2. • Director of Quality Engineering, Pap
    a

    • Tester by nature and nurtur
    e

    • Pronouns are she/he
    r

    • I have stickers!
    Jenny Bramble
    @jennydoesthings - #ElixirConf2022

    View Slide

  3. @jennydoesthings
    @jennydoesthings
    I don’t think I need to
    convince anyone in the room
    that keeping our ecosystem
    up to date is a good thing.

    View Slide

  4. @jennydoesthings
    @jennydoesthings
    What problem do we
    want to solve?

    View Slide

  5. What problem do we
    want to solve?
    • Upgrades are risky but necessary
    .

    • Most parts of our teams don’t understand
    how risky upgrades are or are not
    .

    • Regression testing takes forever and uses
    a lot of our resources
    .

    • If we mess up, it could be real bad.
    @jennydoesthings

    View Slide

  6. @jennydoesthings

    View Slide

  7. @jennydoesthings
    @jennydoesthings
    How do we make
    upgrades less risky?

    View Slide

  8. @jennydoesthings
    @jennydoesthings
    I have an easy answer.

    View Slide

  9. @jennydoesthings
    @jennydoesthings
    Do less testing.

    View Slide

  10. @jennydoesthings

    View Slide

  11. @jennydoesthings
    @jennydoesthings
    How do we make
    upgrades less risky?

    View Slide

  12. @jennydoesthings
    @jennydoesthings
    I believe we derisk upgrades by
    testing the smallest set of features
    that give us confidence to move
    forward.

    View Slide

  13. @jennydoesthings
    @jennydoesthings
    That seems
    obvious, right?

    View Slide

  14. @jennydoesthings
    @jennydoesthings
    Let’s talk about ‘good
    enough testing’.

    View Slide

  15. Good Enough Testing
    • Thoughtfully and intentionally craft test
    scenario
    s

    • Detail what we aren’t testing and wh
    y

    • Don’t thras
    h

    • Don’t toi
    l

    • Prioritizes whitewater kayaking and hiking
    @jennydoesthings

    View Slide

  16. @jennydoesthings

    View Slide

  17. Good Enough Testing
    • Thoughtfully and intentionally craft test
    scenario
    s

    • Detail what we aren’t testing and wh
    y

    • Don’t thras
    h

    • Don’t toi
    l

    • Prioritizes work-life balance
    @jennydoesthings

    View Slide

  18. @jennydoesthings
    @jennydoesthings
    Testers hold the mental model
    of the system in our minds.

    View Slide

  19. @jennydoesthings
    @jennydoesthings
    How do we express
    that in upgrades?

    View Slide

  20. @jennydoesthings
    @jennydoesthings
    Spoiler alert: this isn’t
    going to be a radical shift.

    View Slide

  21. @jennydoesthings
    @jennydoesthings
    Start with the changelog.

    View Slide

  22. @jennydoesthings
    @jennydoesthings
    Now communicate it.

    View Slide

  23. @jennydoesthings
    @jennydoesthings
    For each thing we think will affect
    us, determine its risk profile.

    View Slide

  24. @jennydoesthings

    View Slide

  25. @jennydoesthings
    @jennydoesthings
    What if we get it right?

    View Slide

  26. @jennydoesthings
    @jennydoesthings
    How would this express
    if we missed something?

    View Slide

  27. @jennydoesthings
    @jennydoesthings
    Do we care?

    View Slide

  28. @jennydoesthings
    @jennydoesthings
    How will we monitor and
    observe?

    View Slide

  29. @jennydoesthings
    @jennydoesthings
    Stop throwing things
    over the wall…

    View Slide

  30. @jennydoesthings
    @jennydoesthings
    ….to production.

    View Slide

  31. @jennydoesthings
    @jennydoesthings
    How will we monitor and
    observe?

    View Slide

  32. @jennydoesthings
    @jennydoesthings
    Taadaa, risk profile!

    View Slide

  33. @jennydoesthings
    @jennydoesthings
    Craft scenarios.

    View Slide

  34. @jennydoesthings
    @jennydoesthings
    Defend yourselves.

    View Slide

  35. @jennydoesthings

    View Slide

  36. @jennydoesthings
    @jennydoesthings
    Defend yourselves.

    View Slide

  37. @jennydoesthings - #ElixirConf2022

    View Slide

  38. @jennydoesthings
    @jennydoesthings
    This is how we introduce
    nuance to our testing.

    View Slide

  39. @jennydoesthings
    @jennydoesthings
    This is how we derisk
    upgrades.

    View Slide

  40. @jennydoesthings
    @jennydoesthings
    To recap:

    View Slide

  41. Do less testing.
    • Craft thoughtful, intentional scenarios
    .

    • Describe what we aren’t testing and why
    .

    • Embrace minimalism.
    @jennydoesthings

    View Slide

  42. Communicate more.
    • Share your plans
    .

    • Defend your plans
    .

    • Give others a reason to trust you.
    @jennydoesthings

    View Slide

  43. Questions?
    @jennydoesthings
    [email protected]
    • Twitter: @jennydoesthings

    View Slide

  44. Credits
    This is where you give credit to the ones who are part of this
    project
    .

    ● Presentation template by Slidesgo
    ● Icons by Flaticon
    ● Infographics by Freepik
    ● Images created Freepi
    k

    ● Author introduction slide photo created by Freepi
    k

    ● Text & Image slide photo created by Freepik.com

    View Slide