Beginner's Guide to the Test Automation Galaxy

@angelariggs_ TSQA | 2020 A Beginner’s Guide to the Test
Automation Galaxy

@angelariggs_ TSQA | 2020

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
Centralized QAE QA Manager at Instrument

@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE

@angelariggs_ TSQA | 2020 Itinerary

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Value

@angelariggs_ TSQA | 2020 This talk isn’t intended to give
you all the answers. It’s to help you understand the questions you need to ask, so you can ﬁnd the answers that work for you.

@angelariggs_ TSQA | 2020 How do I choose a framework?
Who should write the tests? Should the tests run in our pipeline? What are some best practices? ❓

@angelariggs_ TSQA | 2020 Which tests should we automate?

@angelariggs_ TSQA | 2020 Prevent defects Maintain quality Meet users’
needs Increase conﬁdence Understand our product Why do we test?

@angelariggs_ TSQA | 2020 Why do we automate? Run tests
faster Better coverage Increase reliability Reusable / repeatable More frequent testing

@angelariggs_ TSQA | 2020 Automation is not the goal. The
goal is to have tests that enhance your ability to achieve and maintain quality.

@angelariggs_ TSQA | 2020 Which tests should we automate?

@angelariggs_ TSQA | 2020 Low-risk or stable functionality

@angelariggs_ TSQA | 2020 Smoke tests Low-level test to make
sure nothing’s on ﬁre

@angelariggs_ TSQA | 2020 UI workﬂows Avoids tedium of manual
testing or “selective attention” mistakes

@angelariggs_ TSQA | 2020 Visual validation Compares before and after
snapshots across the site or product

@angelariggs_ TSQA | 2020 Use automation for low-risk work so
you can focus on high-risk or high-priority functionality.

@angelariggs_ TSQA | 2020 High-risk items that need consistent testing

@angelariggs_ TSQA | 2020 API tests Are your APIs handling
data the right way?

@angelariggs_ TSQA | 2020 Business-critical workﬂows Validate the UIs that
are high-impact, or fall under regulatory compliance

@angelariggs_ TSQA | 2020 Performance testing Understand your product’s stability,
speed, and scalability so you can future-proof

@angelariggs_ TSQA | 2020 Getting Buy-In

@angelariggs_ TSQA | 2020 Automation is valid, meaningful work.

@angelariggs_ TSQA | 2020 What do you want to accomplish?

@angelariggs_ TSQA | 2020 How does this beneﬁt them?

@angelariggs_ TSQA | 2020 Catch regressions Sprint goals Prevent defects
Team confidence Stakeholder confidence Benefits for buy-in

@angelariggs_ TSQA | 2020 Improve your testing skills

@angelariggs_ TSQA | 2020 The goal of getting buy-in is
to make sure your automation eﬀorts are supported by the team and business, so you can build automation that supports them.

@angelariggs_ TSQA | 2020 Choosing a Framework

@angelariggs_ TSQA | 2020 What is a test automation framework?

@angelariggs_ TSQA | 2020 The test automation framework is made
up of the tools and libraries that you use to write the tests, as well as execute, verify, and report on those tests. Angie Jones!

@angelariggs_ TSQA | 2020 Adding assertions Boilerplate or helpers Reporting
Framework libraries

@angelariggs_ TSQA | 2020 There is no framework to rule
them all. The right tools are the ones that you’ll actually use.

@angelariggs_ TSQA | 2020 Code-based or codeless? Open-source or paid?

@angelariggs_ TSQA | 2020 Who’s writing the tests?

@angelariggs_ TSQA | 2020 How are you running your tests?

@angelariggs_ TSQA | 2020 Is there a budget?

@angelariggs_ TSQA | 2020 Is there a “freemium” level?

@angelariggs_ TSQA | 2020 Is there a trial period?

@angelariggs_ TSQA | 2020 What’s the documentation like?

@angelariggs_ TSQA | 2020 How actively is the framework maintained?

@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Recap You are here.

@angelariggs_ TSQA | 2020 Writing Your Tests

@angelariggs_ TSQA | 2020 The goal of automation is reliable,
anti-fragile tests that are simple to maintain, update, and hand oﬀ.

@angelariggs_ TSQA | 2020 Test automation is code!

@angelariggs_ TSQA | 2020 Security

@angelariggs_ TSQA | 2020 Clarity

@angelariggs_ TSQA | 2020 Write tests for Future You

@angelariggs_ TSQA | 2020 sleep(10)

@angelariggs_ TSQA | 2020 waitForElement(#footer)

@angelariggs_ TSQA | 2020 Independent tests

@angelariggs_ TSQA | 2020 Create reservation ✦ Enter location ✦
Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Conﬁrm new dates

@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✦

@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅

Select dates ✅ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Conﬁrm new dates

Select dates ✅ Select unit ❌ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Conﬁrm new dates

Select dates ✅ Select unit ❌ Book vacation Modify reservation ❌ Look up reservation ❌ Select Edit option ❌ Change dates ❌ Conﬁrm new dates

@angelariggs_ TSQA | 2020 Independent tests reduce the fragility of
your test automation.

@angelariggs_ TSQA | 2020 Parallel testing

@angelariggs_ TSQA | 2020 Parallel testing means running multiple automated
tests concurrently instead of consecutively.

@angelariggs_ TSQA | 2020 Minutes 0 1 2 3 4
5 6 Test 1 Test 2 Test 3 Test 4 Test 5 Test 6

@angelariggs_ TSQA | 2020 Minutes 0 1 2 3 4
5 6 Process 1 Process 2 Test 1 Test 3 Test 5 Test 2 Test 4 Test 6

@angelariggs_ TSQA | 2020 Parallel testing is faster testing, and
allows you to scale your automation more eﬀectively.

@angelariggs_ TSQA | 2020 Granularity

@angelariggs_ TSQA | 2020 { Diﬃcult to refactor } Lack
of granularity → dependent tests Hard to read & maintain ❌

@angelariggs_ TSQA | 2020 Refactored into granular tests ↑ Independence,
↓ ﬂakiness Outline tests before writing ✅

@angelariggs_ TSQA | 2020 Think of your tests as narrating
your product’s functionality. You want short, declarative sentences rather than long stream-of-consciousness narration.

@angelariggs_ TSQA | 2020 Failures

@angelariggs_ TSQA | 2020 The goal of automation isn’t having
tests that pass. The goal of automation is having tests that accurately catch failures.

@angelariggs_ TSQA | 2020 If a test never fails, is
it providing value?

@angelariggs_ TSQA | 2020 Planning for failure means writing your
tests in a way that make those failures useful.

@angelariggs_ TSQA | 2020 “it 404’ed”

@angelariggs_ TSQA | 2020 “Ticket page sends a 404 if
support ﬁle does not exist”

@angelariggs_ TSQA | 2020 Running Your Tests

@angelariggs_ TSQA | 2020 Manual kick-oﬀ & reporting Scheduled runs
during the day Integrated into a CI pipeline Running your tests

@angelariggs_ TSQA | 2020 Manual kick-oﬀ & reporting

@angelariggs_ TSQA | 2020 Someone is responsible for the tests
Report gets reviewed & prioritized Failures don’t block the process Pros

@angelariggs_ TSQA | 2020 Slower and less reliable  Reduces visibility
into the report Silos knowledge of the automation Cons

@angelariggs_ TSQA | 2020 Scheduled runs during the day

@angelariggs_ TSQA | 2020 Increased visibility into reporting Fast, frequent
feedback Pros

@angelariggs_ TSQA | 2020 Triaging the report Maintaining a separate
system Cons

@angelariggs_ TSQA | 2020 Integrated into a CI pipeline

@angelariggs_ TSQA | 2020 Consistent feedback for developers Prevents breaking
deploys Pros

@angelariggs_ TSQA | 2020 Failures block deploy process May need
cross-team work to update Cons

@angelariggs_ TSQA | 2020 Having tests is better than not
having tests. Pick a method, but be willing to iterate and adjust if something isn’t working.

@angelariggs_ TSQA | 2020 Showing Value

@angelariggs_ TSQA | 2020 How can you show that the
work has been worth the eﬀort?

@angelariggs_ TSQA | 2020 What beneﬁts were you aiming for?

@angelariggs_ TSQA | 2020 Reduced number of regressions Speed up
development time Team morale Client conﬁdence Showing value

@angelariggs_ TSQA | 2020 Test automation helps you mitigate risk,
validate functionality, and prioritize your testing eﬀorts.

@angelariggs_ TSQA | 2020 Understand what questions to ask, so
you can ﬁnd the answers that work for you

@angelariggs_ TSQA | 2020 Understand the beneﬁts of automation, so
you know when to use it

@angelariggs_ TSQA | 2020 Understand how to get buy-in for
prioritizing automation work

@angelariggs_ TSQA | 2020 Understand how to research frameworks, so
you can ﬁnd the one that meets your needs

@angelariggs_ TSQA | 2020 Understand your product’s fault lines, so
you can add meaningful tests

@angelariggs_ TSQA | 2020 Understand how new work integrates with
existing features

@angelariggs_ TSQA | 2020 Understand your stakeholders’ priorities, so your
tests can build conﬁdence

@angelariggs_ TSQA | 2020 Understand automation best practices, so you
can create anti-fragile tests

@angelariggs_ TSQA | 2020 Understand the beneﬁts and trade-oﬀs between
methods for running your tests

@angelariggs_ TSQA | 2020 Understand the value of test automation

@angelariggs_ TSQA | 2020 Thank you!

@angelariggs_ TSQA | 2020 Resources

@angelariggs_ TSQA | 2020 Lucy Wyman & Zach Reichert Test
Automation University Are Our Tests Any Good? http://bit.ly/are-our-tests-good Online automation courses http://bit.ly/test-automation-uni

Beginner's Guide to the Test Automation Galaxy

Beginner's Guide to the Test Automation Galaxy

More Decks by Angela Riggs

Other Decks in Technology

Featured

Transcript