Slide 1

Slide 1 text

@angelariggs_ TSQA | 2020 A Beginner’s Guide to the Test Automation Galaxy

Slide 2

Slide 2 text

@angelariggs_ TSQA | 2020

Slide 3

Slide 3 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams Centralized QAE QA Manager at Instrument

Slide 4

Slide 4 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams QA Manager at Instrument Centralized QAE

Slide 5

Slide 5 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams QA Manager at Instrument Centralized QAE

Slide 6

Slide 6 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams QA Manager at Instrument Centralized QAE

Slide 7

Slide 7 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams QA Manager at Instrument Centralized QAE

Slide 8

Slide 8 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams QA Manager at Instrument Centralized QAE

Slide 9

Slide 9 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams QA Manager at Instrument Centralized QAE

Slide 10

Slide 10 text

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams QA Manager at Instrument Centralized QAE

Slide 11

Slide 11 text

@angelariggs_ TSQA | 2020 Itinerary

Slide 12

Slide 12 text

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in Choosing a framework Writing your tests Running your tests Value

Slide 13

Slide 13 text

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in Choosing a framework Writing your tests Running your tests Value

Slide 14

Slide 14 text

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in Choosing a framework Writing your tests Running your tests Value

Slide 15

Slide 15 text

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in Choosing a framework Writing your tests Running your tests Value

Slide 16

Slide 16 text

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in Choosing a framework Writing your tests Running your tests Value

Slide 17

Slide 17 text

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in Choosing a framework Writing your tests Running your tests Value

Slide 18

Slide 18 text

@angelariggs_ TSQA | 2020 This talk isn’t intended to give you all the answers. It’s to help you understand the questions you need to ask, so you can find the answers that work for you.

Slide 19

Slide 19 text

@angelariggs_ TSQA | 2020

Slide 20

Slide 20 text

@angelariggs_ TSQA | 2020

Slide 21

Slide 21 text

@angelariggs_ TSQA | 2020

Slide 22

Slide 22 text

@angelariggs_ TSQA | 2020 How do I choose a framework? Who should write the tests? Should the tests run in our pipeline? What are some best practices? ❓

Slide 23

Slide 23 text

@angelariggs_ TSQA | 2020 Which tests should we automate?

Slide 24

Slide 24 text

@angelariggs_ TSQA | 2020 Prevent defects Maintain quality Meet users’ needs Increase confidence Understand our product Why do we test?

Slide 25

Slide 25 text

@angelariggs_ TSQA | 2020 Why do we automate? Run tests faster Better coverage Increase reliability Reusable / repeatable More frequent testing

Slide 26

Slide 26 text

@angelariggs_ TSQA | 2020

Slide 27

Slide 27 text

@angelariggs_ TSQA | 2020 Automation is not the goal. The goal is to have tests that enhance your ability to achieve and maintain quality.

Slide 28

Slide 28 text

@angelariggs_ TSQA | 2020 Which tests should we automate?

Slide 29

Slide 29 text

@angelariggs_ TSQA | 2020

Slide 30

Slide 30 text

@angelariggs_ TSQA | 2020 Low-risk or stable functionality

Slide 31

Slide 31 text

@angelariggs_ TSQA | 2020 Smoke tests Low-level test to make sure nothing’s on fire

Slide 32

Slide 32 text

@angelariggs_ TSQA | 2020 UI workflows Avoids tedium of manual testing or “selective attention” mistakes

Slide 33

Slide 33 text

@angelariggs_ TSQA | 2020 Visual validation Compares before and after snapshots across the site or product

Slide 34

Slide 34 text

@angelariggs_ TSQA | 2020 Use automation for low-risk work so you can focus on high-risk or high-priority functionality.

Slide 35

Slide 35 text

@angelariggs_ TSQA | 2020 High-risk items that need consistent testing

Slide 36

Slide 36 text

@angelariggs_ TSQA | 2020 API tests Are your APIs handling data the right way?

Slide 37

Slide 37 text

@angelariggs_ TSQA | 2020 Business-critical workflows Validate the UIs that are high-impact, or fall under regulatory compliance

Slide 38

Slide 38 text

@angelariggs_ TSQA | 2020 Performance testing Understand your product’s stability, speed, and scalability so you can future-proof

Slide 39

Slide 39 text

@angelariggs_ TSQA | 2020

Slide 40

Slide 40 text

@angelariggs_ TSQA | 2020 Getting Buy-In

Slide 41

Slide 41 text

@angelariggs_ TSQA | 2020 Automation is valid, meaningful work.

Slide 42

Slide 42 text

@angelariggs_ TSQA | 2020 What do you want to accomplish?

Slide 43

Slide 43 text

@angelariggs_ TSQA | 2020 How does this benefit them?

Slide 44

Slide 44 text

@angelariggs_ TSQA | 2020 Catch regressions Sprint goals Prevent defects Team confidence Stakeholder confidence Benefits for buy-in

Slide 45

Slide 45 text

@angelariggs_ TSQA | 2020 Improve your testing skills

Slide 46

Slide 46 text

@angelariggs_ TSQA | 2020 The goal of getting buy-in is to make sure your automation efforts are supported by the team and business, so you can build automation that supports them.

Slide 47

Slide 47 text

@angelariggs_ TSQA | 2020 Choosing a Framework

Slide 48

Slide 48 text

@angelariggs_ TSQA | 2020 What is a test automation framework?

Slide 49

Slide 49 text

@angelariggs_ TSQA | 2020 The test automation framework is made up of the tools and libraries that you use to write the tests, as well as execute, verify, and report on those tests. Angie Jones!

Slide 50

Slide 50 text

@angelariggs_ TSQA | 2020 Adding assertions Boilerplate or helpers Reporting Framework libraries

Slide 51

Slide 51 text

@angelariggs_ TSQA | 2020 There is no framework to rule them all. The right tools are the ones that you’ll actually use.

Slide 52

Slide 52 text

@angelariggs_ TSQA | 2020 Code-based or codeless? Open-source or paid?

Slide 53

Slide 53 text

@angelariggs_ TSQA | 2020 Who’s writing the tests?

Slide 54

Slide 54 text

@angelariggs_ TSQA | 2020 How are you running your tests?

Slide 55

Slide 55 text

@angelariggs_ TSQA | 2020 Is there a budget?

Slide 56

Slide 56 text

@angelariggs_ TSQA | 2020 Is there a “freemium” level?

Slide 57

Slide 57 text

@angelariggs_ TSQA | 2020 Is there a trial period?

Slide 58

Slide 58 text

@angelariggs_ TSQA | 2020 What’s the documentation like?

Slide 59

Slide 59 text

@angelariggs_ TSQA | 2020 How actively is the framework maintained?

Slide 60

Slide 60 text

@angelariggs_ TSQA | 2020

Slide 61

Slide 61 text

@angelariggs_ TSQA | 2020

Slide 62

Slide 62 text

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in Choosing a framework Writing your tests Running your tests Recap You are here.

Slide 63

Slide 63 text

@angelariggs_ TSQA | 2020

Slide 64

Slide 64 text

@angelariggs_ TSQA | 2020 Writing Your Tests

Slide 65

Slide 65 text

@angelariggs_ TSQA | 2020 The goal of automation is reliable, anti-fragile tests that are simple to maintain, update, and hand off.

Slide 66

Slide 66 text

@angelariggs_ TSQA | 2020 Test automation is code!

Slide 67

Slide 67 text

@angelariggs_ TSQA | 2020 Security

Slide 68

Slide 68 text

@angelariggs_ TSQA | 2020 Security

Slide 69

Slide 69 text

@angelariggs_ TSQA | 2020

Slide 70

Slide 70 text

@angelariggs_ TSQA | 2020 Clarity

Slide 71

Slide 71 text

@angelariggs_ TSQA | 2020 Write tests for Future You

Slide 72

Slide 72 text

@angelariggs_ TSQA | 2020 sleep(10)

Slide 73

Slide 73 text

@angelariggs_ TSQA | 2020 waitForElement(#footer)

Slide 74

Slide 74 text

@angelariggs_ TSQA | 2020 Independent tests

Slide 75

Slide 75 text

@angelariggs_ TSQA | 2020 Create reservation ✦ Enter location ✦ Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates

Slide 76

Slide 76 text

@angelariggs_ TSQA | 2020 Create reservation ✦ Enter location ✦ Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates

Slide 77

Slide 77 text

@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✦ Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates

Slide 78

Slide 78 text

@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅ Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates

Slide 79

Slide 79 text

@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅ Select dates ✅ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates

Slide 80

Slide 80 text

@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅ Select dates ✅ Select unit ❌ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates

Slide 81

Slide 81 text

@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅ Select dates ✅ Select unit ❌ Book vacation Modify reservation ❌ Look up reservation ❌ Select Edit option ❌ Change dates ❌ Confirm new dates

Slide 82

Slide 82 text

@angelariggs_ TSQA | 2020

Slide 83

Slide 83 text

@angelariggs_ TSQA | 2020 Independent tests reduce the fragility of your test automation.

Slide 84

Slide 84 text

@angelariggs_ TSQA | 2020 Parallel testing

Slide 85

Slide 85 text

@angelariggs_ TSQA | 2020 Parallel testing means running multiple automated tests concurrently instead of consecutively.

Slide 86

Slide 86 text

@angelariggs_ TSQA | 2020 Minutes 0 1 2 3 4 5 6 Test 1 Test 2 Test 3 Test 4 Test 5 Test 6

Slide 87

Slide 87 text

@angelariggs_ TSQA | 2020 Minutes 0 1 2 3 4 5 6 Process 1 Process 2 Test 1 Test 3 Test 5 Test 2 Test 4 Test 6

Slide 88

Slide 88 text

@angelariggs_ TSQA | 2020 Parallel testing is faster testing, and allows you to scale your automation more effectively.

Slide 89

Slide 89 text

@angelariggs_ TSQA | 2020 Granularity

Slide 90

Slide 90 text

@angelariggs_ TSQA | 2020

Slide 91

Slide 91 text

@angelariggs_ TSQA | 2020

Slide 92

Slide 92 text

@angelariggs_ TSQA | 2020 { Difficult to refactor } Lack of granularity → dependent tests Hard to read & maintain ❌

Slide 93

Slide 93 text

@angelariggs_ TSQA | 2020 Refactored into granular tests ↑ Independence, ↓ flakiness Outline tests before writing ✅

Slide 94

Slide 94 text

@angelariggs_ TSQA | 2020 Think of your tests as narrating your product’s functionality. You want short, declarative sentences rather than long stream-of-consciousness narration.

Slide 95

Slide 95 text

@angelariggs_ TSQA | 2020 Failures

Slide 96

Slide 96 text

@angelariggs_ TSQA | 2020

Slide 97

Slide 97 text

@angelariggs_ TSQA | 2020 The goal of automation isn’t having tests that pass. The goal of automation is having tests that accurately catch failures.

Slide 98

Slide 98 text

@angelariggs_ TSQA | 2020 If a test never fails, is it providing value?

Slide 99

Slide 99 text

@angelariggs_ TSQA | 2020 Planning for failure means writing your tests in a way that make those failures useful.

Slide 100

Slide 100 text

@angelariggs_ TSQA | 2020 “it 404’ed”

Slide 101

Slide 101 text

@angelariggs_ TSQA | 2020 “Ticket page sends a 404 if support file does not exist”

Slide 102

Slide 102 text

@angelariggs_ TSQA | 2020 Running Your Tests

Slide 103

Slide 103 text

@angelariggs_ TSQA | 2020 Manual kick-off & reporting Scheduled runs during the day Integrated into a CI pipeline Running your tests

Slide 104

Slide 104 text

@angelariggs_ TSQA | 2020 Manual kick-off & reporting

Slide 105

Slide 105 text

@angelariggs_ TSQA | 2020 Someone is responsible for the tests Report gets reviewed & prioritized Failures don’t block the process Pros

Slide 106

Slide 106 text

@angelariggs_ TSQA | 2020 Slower and less reliable
 Reduces visibility into the report Silos knowledge of the automation Cons

Slide 107

Slide 107 text

@angelariggs_ TSQA | 2020 Scheduled runs during the day

Slide 108

Slide 108 text

@angelariggs_ TSQA | 2020 Increased visibility into reporting Fast, frequent feedback Pros

Slide 109

Slide 109 text

@angelariggs_ TSQA | 2020 Triaging the report Maintaining a separate system Cons

Slide 110

Slide 110 text

@angelariggs_ TSQA | 2020 Integrated into a CI pipeline

Slide 111

Slide 111 text

@angelariggs_ TSQA | 2020 Consistent feedback for developers Prevents breaking deploys Pros

Slide 112

Slide 112 text

@angelariggs_ TSQA | 2020 Failures block deploy process May need cross-team work to update Cons

Slide 113

Slide 113 text

@angelariggs_ TSQA | 2020 Having tests is better than not having tests. Pick a method, but be willing to iterate and adjust if something isn’t working.

Slide 114

Slide 114 text

@angelariggs_ TSQA | 2020 Showing Value

Slide 115

Slide 115 text

@angelariggs_ TSQA | 2020 How can you show that the work has been worth the effort?

Slide 116

Slide 116 text

@angelariggs_ TSQA | 2020 What benefits were you aiming for?

Slide 117

Slide 117 text

@angelariggs_ TSQA | 2020 Reduced number of regressions Speed up development time Team morale Client confidence Showing value

Slide 118

Slide 118 text

@angelariggs_ TSQA | 2020 Test automation helps you mitigate risk, validate functionality, and prioritize your testing efforts.

Slide 119

Slide 119 text

@angelariggs_ TSQA | 2020

Slide 120

Slide 120 text

@angelariggs_ TSQA | 2020 Understand what questions to ask, so you can find the answers that work for you

Slide 121

Slide 121 text

@angelariggs_ TSQA | 2020 Understand the benefits of automation, so you know when to use it

Slide 122

Slide 122 text

@angelariggs_ TSQA | 2020 Understand how to get buy-in for prioritizing automation work

Slide 123

Slide 123 text

@angelariggs_ TSQA | 2020 Understand how to research frameworks, so you can find the one that meets your needs

Slide 124

Slide 124 text

@angelariggs_ TSQA | 2020 Understand your product’s fault lines, so you can add meaningful tests

Slide 125

Slide 125 text

@angelariggs_ TSQA | 2020 Understand how new work integrates with existing features

Slide 126

Slide 126 text

@angelariggs_ TSQA | 2020 Understand your stakeholders’ priorities, so your tests can build confidence

Slide 127

Slide 127 text

@angelariggs_ TSQA | 2020 Understand automation best practices, so you can create anti-fragile tests

Slide 128

Slide 128 text

@angelariggs_ TSQA | 2020 Understand the benefits and trade-offs between methods for running your tests

Slide 129

Slide 129 text

@angelariggs_ TSQA | 2020 Understand the value of test automation

Slide 130

Slide 130 text

@angelariggs_ TSQA | 2020

Slide 131

Slide 131 text

@angelariggs_ TSQA | 2020 Thank you!

Slide 132

Slide 132 text

@angelariggs_ TSQA | 2020 Resources

Slide 133

Slide 133 text

@angelariggs_ TSQA | 2020 Lucy Wyman & Zach Reichert Test Automation University Are Our Tests Any Good? http://bit.ly/are-our-tests-good Online automation courses http://bit.ly/test-automation-uni