rainforest @rainforestqa
Your QA options
(Wait… there’s more than one?)
Selenium = only way most people test apps.
Bad for a lot of reasons. The most obvious is mobile.
Slide 2
Slide 2 text
@rainforestqa
rainforest
Paul Burt
QA Evangelist
How did you get started in QA?
What do you do at Rainforest?
Slide 3
Slide 3 text
@rainforestqa
rainforest
Paul Burt
Dev Evangelist, but won’t discuss Jesus.
Slide 4
Slide 4 text
@rainforestqa
rainforest
#1: so, what is QA?
The purpose of QA = discover unknowns.
QA won’t fix quality issues, just Identifies.
Slide 5
Slide 5 text
@rainforestqa
rainforest
#1.1 how I think about QA
QA is not a solvable problem.
Two main components: Exploratory + Regressions.
Slide 6
Slide 6 text
@rainforestqa
rainforest
In QA, you have limited moves.
Spend them wisely.
Slide 7
Slide 7 text
@rainforestqa
rainforest
#2: current landscape
Let’s look at the current options.
Slide 8
Slide 8 text
@rainforestqa
rainforest
#2.1 manual
Comes free.
Half of QA is finding another set of eyes.
Slide 9
Slide 9 text
@rainforestqa
rainforest
Problem: manual testing is tedious.
Slide 10
Slide 10 text
@rainforestqa
rainforest
You can outsource,
or hire a service.
Save time by hiring more people.
Scaling is hard.
Try outside services.
Slide 11
Slide 11 text
@rainforestqa
rainforest
Problem: You won’t sleep easy while manually testing.
Business Owners want scalability + reliability.
Slide 12
Slide 12 text
@rainforestqa
rainforest
Manual testing is inconsistent.
Exploratory testing makes manual a necessity.
Regression testing is what you automate.
Slide 13
Slide 13 text
@rainforestqa
rainforest
#2.2 automation
Customer service + PMs can help do manual.
Not true w/ automation. Requires engineers.
Slide 14
Slide 14 text
@rainforestqa
rainforest
What makes a good
automated test?
Fast.
Accurate.
Only automate if it saves you time.
Slide 15
Slide 15 text
@rainforestqa
rainforest
The problem with automation
Observer effect.
Greater distance from manual = Greater chance false pos/neg.
Slide 16
Slide 16 text
@rainforestqa
rainforest
The problem with automation
Good thing
^
Saves Time.
Only worth automating a test if it saves time.
Slide 17
Slide 17 text
@rainforestqa
rainforest
#2.2.1 selenium
The most popular option.
You’re probably familiar with it.
Slide 18
Slide 18 text
@rainforestqa
rainforest
Easy to write.
Accurate.
Slide 19
Slide 19 text
@rainforestqa
rainforest
Test in Selenium are Flakey.
Slide 20
Slide 20 text
@rainforestqa
rainforest
Infrastructure
+ Solid Test Environments
+ Wrappers
How to fix? Better test environments.
Easy to say, hard to do in reality.
Slide 21
Slide 21 text
@rainforestqa
rainforest
It’s synchronous, and slow.
Slowness complicates CI.
Slide 22
Slide 22 text
@rainforestqa
rainforest
Selenium Grid + $$$
Fix slowness with Selenium Grid.
It’s a hack, not a real fix.
Slide 23
Slide 23 text
@rainforestqa
rainforest
#2.2.2 headless
Slide 24
Slide 24 text
@rainforestqa
rainforest
PhantomJS, fast but ugly.
Often written in JS, but there are wrappers in ruby, etc.
Slide 25
Slide 25 text
@rainforestqa
rainforest
Headless browsers are fast.
Means devs will actually run it.
Slide 26
Slide 26 text
@rainforestqa
rainforest
Headless is Inaccurate.
When accuracy is needed, use Selenium or Manual.
Slide 27
Slide 27 text
@rainforestqa
rainforest
PhantomJS
ZombieJS
SimpleBrowser
Popular browsers are PhantomJS and ZombieJS.
Selenium can be hacked to run Headless.
Slide 28
Slide 28 text
@rainforestqa
rainforest
#3: rainforest
Human powered QA.
Slide 29
Slide 29 text
@rainforestqa
rainforest
Inbetween
manual & automation
Great for flakey tests, or for prototyping new tests.
Not so great at checking your Robots.txt file.
Slide 30
Slide 30 text
@rainforestqa
rainforest
#4 which option is best?
Which one should you use?
Slide 31
Slide 31 text
@rainforestqa
rainforest
ALL OF THEM.
Silly to lock yourself into one solution.
Slide 32
Slide 32 text
@rainforestqa
rainforest
You will always need
Exploratory testing
Need more exploratory help?
Use a service or hire.
Slide 33
Slide 33 text
@rainforestqa
rainforest
You will always need
Regression testing
Need more regressions?
Use the right tool for the right job.
Slide 34
Slide 34 text
@rainforestqa
rainforest
There will always be fires to fight.
On both sides.