Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Beginner's Guide to the Test Automation Galaxy
Search
Angela Riggs
February 27, 2020
Technology
0
110
Beginner's Guide to the Test Automation Galaxy
TSQA 2020!
Angela Riggs
February 27, 2020
Tweet
Share
More Decks by Angela Riggs
See All by Angela Riggs
STPCon | Why is There a Marble in Your Nose?
angelariggs
0
290
Why is There a Marble in Your Nose?
angelariggs
0
420
A Beginner's Guide to Test Automation
angelariggs
1
460
Why is There a Marble in Your Nose?
angelariggs
1
580
Creating a Culture of Quality
angelariggs
2
260
Creating a Culture of Quality (TestBash SF)
angelariggs
2
490
Automating Visual Regression Testing with BackstopJS
angelariggs
1
410
Other Decks in Technology
See All in Technology
【若手エンジニア応援LT会】AWSで繋がり、共に成長! ~コミュニティ活動と新人教育への挑戦~
kazushi_ohata
0
150
pandasはPolarsに性能面で追いつき追い越せるのか
vaaaaanquish
4
2.7k
新R25、乃木坂46 Mobileなどのファンビジネスを支えるマルチテナンシーなプラットフォームの全体像 / cam-multi-cloud
cyberagentdevelopers
PRO
1
120
サイロ化した金融システムを、packwerk を利用して無事故でリファクタリングした話
coincheck_recruit
3
3.6k
生成AIとAWS CDKで実現! 自社ブログレビューの効率化
ymae
2
290
AWSコンテナ本出版から3年経った今、もし改めて執筆し直すなら / If I revise our container book
iselegant
13
3.7k
Hotwire光の道とStimulus
nay3
6
2.4k
使えそうで使われないCloudHSM
maikamibayashi
0
150
小規模に始めるデータメッシュとデータガバナンスの実践
kimujun
3
510
Emacs x Nostr
hakkadaikon
1
140
Java x Spring Boot Warm up
kazu_kichi_67
2
470
顧客が本当に必要だったもの - パフォーマンス改善編 / Make what is needed
soudai
23
6.5k
Featured
See All Featured
A Tale of Four Properties
chriscoyier
156
23k
A designer walks into a library…
pauljervisheath
202
24k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
7
150
RailsConf 2023
tenderlove
29
880
Rebuilding a faster, lazier Slack
samanthasiow
79
8.6k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5k
Making the Leap to Tech Lead
cromwellryan
132
8.9k
It's Worth the Effort
3n
183
27k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
Writing Fast Ruby
sferik
626
60k
Navigating Team Friction
lara
183
14k
Automating Front-end Workflow
addyosmani
1365
200k
Transcript
@angelariggs_ TSQA | 2020 A Beginner’s Guide to the Test
Automation Galaxy
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
Centralized QAE QA Manager at Instrument
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE
@angelariggs_ TSQA | 2020 Embedded QAE QAE for multiple teams
QA Manager at Instrument Centralized QAE
@angelariggs_ TSQA | 2020 Itinerary
@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Value
@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Value
@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Value
@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Value
@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Value
@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Value
@angelariggs_ TSQA | 2020 This talk isn’t intended to give
you all the answers. It’s to help you understand the questions you need to ask, so you can find the answers that work for you.
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 How do I choose a framework?
Who should write the tests? Should the tests run in our pipeline? What are some best practices? ❓
@angelariggs_ TSQA | 2020 Which tests should we automate?
@angelariggs_ TSQA | 2020 Prevent defects Maintain quality Meet users’
needs Increase confidence Understand our product Why do we test?
@angelariggs_ TSQA | 2020 Why do we automate? Run tests
faster Better coverage Increase reliability Reusable / repeatable More frequent testing
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Automation is not the goal. The
goal is to have tests that enhance your ability to achieve and maintain quality.
@angelariggs_ TSQA | 2020 Which tests should we automate?
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Low-risk or stable functionality
@angelariggs_ TSQA | 2020 Smoke tests Low-level test to make
sure nothing’s on fire
@angelariggs_ TSQA | 2020 UI workflows Avoids tedium of manual
testing or “selective attention” mistakes
@angelariggs_ TSQA | 2020 Visual validation Compares before and after
snapshots across the site or product
@angelariggs_ TSQA | 2020 Use automation for low-risk work so
you can focus on high-risk or high-priority functionality.
@angelariggs_ TSQA | 2020 High-risk items that need consistent testing
@angelariggs_ TSQA | 2020 API tests Are your APIs handling
data the right way?
@angelariggs_ TSQA | 2020 Business-critical workflows Validate the UIs that
are high-impact, or fall under regulatory compliance
@angelariggs_ TSQA | 2020 Performance testing Understand your product’s stability,
speed, and scalability so you can future-proof
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Getting Buy-In
@angelariggs_ TSQA | 2020 Automation is valid, meaningful work.
@angelariggs_ TSQA | 2020 What do you want to accomplish?
@angelariggs_ TSQA | 2020 How does this benefit them?
@angelariggs_ TSQA | 2020 Catch regressions Sprint goals Prevent defects
Team confidence Stakeholder confidence Benefits for buy-in
@angelariggs_ TSQA | 2020 Improve your testing skills
@angelariggs_ TSQA | 2020 The goal of getting buy-in is
to make sure your automation efforts are supported by the team and business, so you can build automation that supports them.
@angelariggs_ TSQA | 2020 Choosing a Framework
@angelariggs_ TSQA | 2020 What is a test automation framework?
@angelariggs_ TSQA | 2020 The test automation framework is made
up of the tools and libraries that you use to write the tests, as well as execute, verify, and report on those tests. Angie Jones!
@angelariggs_ TSQA | 2020 Adding assertions Boilerplate or helpers Reporting
Framework libraries
@angelariggs_ TSQA | 2020 There is no framework to rule
them all. The right tools are the ones that you’ll actually use.
@angelariggs_ TSQA | 2020 Code-based or codeless? Open-source or paid?
@angelariggs_ TSQA | 2020 Who’s writing the tests?
@angelariggs_ TSQA | 2020 How are you running your tests?
@angelariggs_ TSQA | 2020 Is there a budget?
@angelariggs_ TSQA | 2020 Is there a “freemium” level?
@angelariggs_ TSQA | 2020 Is there a trial period?
@angelariggs_ TSQA | 2020 What’s the documentation like?
@angelariggs_ TSQA | 2020 How actively is the framework maintained?
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Which tests to automate Getting buy-in
Choosing a framework Writing your tests Running your tests Recap You are here.
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Writing Your Tests
@angelariggs_ TSQA | 2020 The goal of automation is reliable,
anti-fragile tests that are simple to maintain, update, and hand off.
@angelariggs_ TSQA | 2020 Test automation is code!
@angelariggs_ TSQA | 2020 Security
@angelariggs_ TSQA | 2020 Security
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Clarity
@angelariggs_ TSQA | 2020 Write tests for Future You
@angelariggs_ TSQA | 2020 sleep(10)
@angelariggs_ TSQA | 2020 waitForElement(#footer)
@angelariggs_ TSQA | 2020 Independent tests
@angelariggs_ TSQA | 2020 Create reservation ✦ Enter location ✦
Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates
@angelariggs_ TSQA | 2020 Create reservation ✦ Enter location ✦
Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates
@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✦
Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates
@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅
Select dates ✦ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates
@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅
Select dates ✅ Select unit ✦ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates
@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅
Select dates ✅ Select unit ❌ Book vacation Modify reservation ✦ Look up reservation ✦ Select Edit option ✦ Change dates ✦ Confirm new dates
@angelariggs_ TSQA | 2020 Create reservation ✅ Enter location ✅
Select dates ✅ Select unit ❌ Book vacation Modify reservation ❌ Look up reservation ❌ Select Edit option ❌ Change dates ❌ Confirm new dates
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Independent tests reduce the fragility of
your test automation.
@angelariggs_ TSQA | 2020 Parallel testing
@angelariggs_ TSQA | 2020 Parallel testing means running multiple automated
tests concurrently instead of consecutively.
@angelariggs_ TSQA | 2020 Minutes 0 1 2 3 4
5 6 Test 1 Test 2 Test 3 Test 4 Test 5 Test 6
@angelariggs_ TSQA | 2020 Minutes 0 1 2 3 4
5 6 Process 1 Process 2 Test 1 Test 3 Test 5 Test 2 Test 4 Test 6
@angelariggs_ TSQA | 2020 Parallel testing is faster testing, and
allows you to scale your automation more effectively.
@angelariggs_ TSQA | 2020 Granularity
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 { Difficult to refactor } Lack
of granularity → dependent tests Hard to read & maintain ❌
@angelariggs_ TSQA | 2020 Refactored into granular tests ↑ Independence,
↓ flakiness Outline tests before writing ✅
@angelariggs_ TSQA | 2020 Think of your tests as narrating
your product’s functionality. You want short, declarative sentences rather than long stream-of-consciousness narration.
@angelariggs_ TSQA | 2020 Failures
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 The goal of automation isn’t having
tests that pass. The goal of automation is having tests that accurately catch failures.
@angelariggs_ TSQA | 2020 If a test never fails, is
it providing value?
@angelariggs_ TSQA | 2020 Planning for failure means writing your
tests in a way that make those failures useful.
@angelariggs_ TSQA | 2020 “it 404’ed”
@angelariggs_ TSQA | 2020 “Ticket page sends a 404 if
support file does not exist”
@angelariggs_ TSQA | 2020 Running Your Tests
@angelariggs_ TSQA | 2020 Manual kick-off & reporting Scheduled runs
during the day Integrated into a CI pipeline Running your tests
@angelariggs_ TSQA | 2020 Manual kick-off & reporting
@angelariggs_ TSQA | 2020 Someone is responsible for the tests
Report gets reviewed & prioritized Failures don’t block the process Pros
@angelariggs_ TSQA | 2020 Slower and less reliable Reduces visibility
into the report Silos knowledge of the automation Cons
@angelariggs_ TSQA | 2020 Scheduled runs during the day
@angelariggs_ TSQA | 2020 Increased visibility into reporting Fast, frequent
feedback Pros
@angelariggs_ TSQA | 2020 Triaging the report Maintaining a separate
system Cons
@angelariggs_ TSQA | 2020 Integrated into a CI pipeline
@angelariggs_ TSQA | 2020 Consistent feedback for developers Prevents breaking
deploys Pros
@angelariggs_ TSQA | 2020 Failures block deploy process May need
cross-team work to update Cons
@angelariggs_ TSQA | 2020 Having tests is better than not
having tests. Pick a method, but be willing to iterate and adjust if something isn’t working.
@angelariggs_ TSQA | 2020 Showing Value
@angelariggs_ TSQA | 2020 How can you show that the
work has been worth the effort?
@angelariggs_ TSQA | 2020 What benefits were you aiming for?
@angelariggs_ TSQA | 2020 Reduced number of regressions Speed up
development time Team morale Client confidence Showing value
@angelariggs_ TSQA | 2020 Test automation helps you mitigate risk,
validate functionality, and prioritize your testing efforts.
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Understand what questions to ask, so
you can find the answers that work for you
@angelariggs_ TSQA | 2020 Understand the benefits of automation, so
you know when to use it
@angelariggs_ TSQA | 2020 Understand how to get buy-in for
prioritizing automation work
@angelariggs_ TSQA | 2020 Understand how to research frameworks, so
you can find the one that meets your needs
@angelariggs_ TSQA | 2020 Understand your product’s fault lines, so
you can add meaningful tests
@angelariggs_ TSQA | 2020 Understand how new work integrates with
existing features
@angelariggs_ TSQA | 2020 Understand your stakeholders’ priorities, so your
tests can build confidence
@angelariggs_ TSQA | 2020 Understand automation best practices, so you
can create anti-fragile tests
@angelariggs_ TSQA | 2020 Understand the benefits and trade-offs between
methods for running your tests
@angelariggs_ TSQA | 2020 Understand the value of test automation
@angelariggs_ TSQA | 2020
@angelariggs_ TSQA | 2020 Thank you!
@angelariggs_ TSQA | 2020 Resources
@angelariggs_ TSQA | 2020 Lucy Wyman & Zach Reichert Test
Automation University Are Our Tests Any Good? http://bit.ly/are-our-tests-good Online automation courses http://bit.ly/test-automation-uni