Seems Good Enough to Me: Working with Testers to Derisk Upgrades

Slide 1

Slide 1 text

Seems Good Enough to Me @jennydoesthings Working With Testers to Derisk Elixir Upgrades

Slide 2

Slide 2 text

• Director of Quality Engineering, Pap a • Tester by nature and nurtur e • Pronouns are she/he r • I have stickers! Jenny Bramble @jennydoesthings - #ElixirConf2022

Slide 3

Slide 3 text

@jennydoesthings @jennydoesthings I don’t think I need to convince anyone in the room that keeping our ecosystem up to date is a good thing.

Slide 4

Slide 4 text

@jennydoesthings @jennydoesthings What problem do we want to solve?

Slide 5

Slide 5 text

What problem do we want to solve? • Upgrades are risky but necessary . • Most parts of our teams don’t understand how risky upgrades are or are not . • Regression testing takes forever and uses a lot of our resources . • If we mess up, it could be real bad. @jennydoesthings

Slide 6

Slide 6 text

@jennydoesthings

Slide 7

Slide 7 text

@jennydoesthings @jennydoesthings How do we make upgrades less risky?

Slide 8

Slide 8 text

@jennydoesthings @jennydoesthings I have an easy answer.

Slide 9

Slide 9 text

@jennydoesthings @jennydoesthings Do less testing.

Slide 10

Slide 10 text

@jennydoesthings

Slide 11

Slide 11 text

@jennydoesthings @jennydoesthings How do we make upgrades less risky?

Slide 12

Slide 12 text

@jennydoesthings @jennydoesthings I believe we derisk upgrades by testing the smallest set of features that give us confidence to move forward.

Slide 13

Slide 13 text

@jennydoesthings @jennydoesthings That seems obvious, right?

Slide 14

Slide 14 text

@jennydoesthings @jennydoesthings Let’s talk about ‘good enough testing’.

Slide 15

Slide 15 text

Good Enough Testing • Thoughtfully and intentionally craft test scenario s • Detail what we aren’t testing and wh y • Don’t thras h • Don’t toi l • Prioritizes whitewater kayaking and hiking @jennydoesthings

Slide 16

Slide 16 text

@jennydoesthings

Slide 17

Slide 17 text

Good Enough Testing • Thoughtfully and intentionally craft test scenario s • Detail what we aren’t testing and wh y • Don’t thras h • Don’t toi l • Prioritizes work-life balance @jennydoesthings

Slide 18

Slide 18 text

@jennydoesthings @jennydoesthings Testers hold the mental model of the system in our minds.

Slide 19

Slide 19 text

@jennydoesthings @jennydoesthings How do we express that in upgrades?

Slide 20

Slide 20 text

@jennydoesthings @jennydoesthings Spoiler alert: this isn’t going to be a radical shift.

Slide 21

Slide 21 text

@jennydoesthings @jennydoesthings Start with the changelog.

Slide 22

Slide 22 text

@jennydoesthings @jennydoesthings Now communicate it.

Slide 23

Slide 23 text

@jennydoesthings @jennydoesthings For each thing we think will affect us, determine its risk profile.

Slide 24

Slide 24 text

@jennydoesthings

Slide 25

Slide 25 text

@jennydoesthings @jennydoesthings What if we get it right?

Slide 26

Slide 26 text

@jennydoesthings @jennydoesthings How would this express if we missed something?

Slide 27

Slide 27 text

@jennydoesthings @jennydoesthings Do we care?

Slide 28

Slide 28 text

@jennydoesthings @jennydoesthings How will we monitor and observe?

Slide 29

Slide 29 text

@jennydoesthings @jennydoesthings Stop throwing things over the wall…

Slide 30

Slide 30 text

@jennydoesthings @jennydoesthings ….to production.

Slide 31

Slide 31 text

@jennydoesthings @jennydoesthings How will we monitor and observe?

Slide 32

Slide 32 text

@jennydoesthings @jennydoesthings Taadaa, risk profile!

Slide 33

Slide 33 text

@jennydoesthings @jennydoesthings Craft scenarios.

Slide 34

Slide 34 text

@jennydoesthings @jennydoesthings Defend yourselves.

Slide 35

Slide 35 text

@jennydoesthings

Slide 36

Slide 36 text

@jennydoesthings @jennydoesthings Defend yourselves.

Slide 37

Slide 37 text

@jennydoesthings - #ElixirConf2022

Slide 38

Slide 38 text

@jennydoesthings @jennydoesthings This is how we introduce nuance to our testing.

Slide 39

Slide 39 text

@jennydoesthings @jennydoesthings This is how we derisk upgrades.

Slide 40

Slide 40 text

@jennydoesthings @jennydoesthings To recap:

Slide 41

Slide 41 text

Do less testing. • Craft thoughtful, intentional scenarios . • Describe what we aren’t testing and why . • Embrace minimalism. @jennydoesthings

Slide 42

Slide 42 text

Communicate more. • Share your plans . • Defend your plans . • Give others a reason to trust you. @jennydoesthings

Slide 43

Slide 43 text

Questions? @jennydoesthings • [email protected] • Twitter: @jennydoesthings

Slide 44

Slide 44 text

Credits This is where you give credit to the ones who are part of this project . ● Presentation template by Slidesgo ● Icons by Flaticon ● Infographics by Freepik ● Images created Freepi k ● Author introduction slide photo created by Freepi k ● Text & Image slide photo created by Freepik.com