Use the Right Tool

A lot of companies use one framework and only one for their testing. Why do that, when headless browsers are such a beautiful compliment to the traditional approach?

Transcript

1. rainforest @rainforestqa Your QA options (Wait… there’s more than one?) Selenium = only way most people test apps. Bad for a lot of reasons. The most obvious is mobile.
2. @rainforestqa rainforest Paul Burt QA Evangelist How did you get started in QA? What do you do at Rainforest?
3. @rainforestqa rainforest Paul Burt Dev Evangelist, but won’t discuss Jesus.
4. @rainforestqa rainforest #1: so, what is QA? The purpose of QA = discover unknowns. QA won’t fix quality issues, just Identifies.
5. @rainforestqa rainforest #1.1 how I think about QA QA is not a solvable problem. Two main components: Exploratory + Regressions.
6. @rainforestqa rainforest In QA, you have limited moves. Spend them wisely.
7. @rainforestqa rainforest #2: current landscape Let’s look at the current options.
8. @rainforestqa rainforest #2.1 manual Comes free. Half of QA is finding another set of eyes.
9. @rainforestqa rainforest Problem: manual testing is tedious.
10. @rainforestqa rainforest You can outsource, or hire a service. Save time by hiring more people. Scaling is hard. Try outside services.
11. @rainforestqa rainforest Problem: You won’t sleep easy while manually testing. Business Owners want scalability + reliability.
12. @rainforestqa rainforest Manual testing is inconsistent. Exploratory testing makes manual a necessity. Regression testing is what you automate.
13. @rainforestqa rainforest #2.2 automation Customer service + PMs can help do manual. Not true w/ automation. Requires engineers.
14. @rainforestqa rainforest What makes a good automated test? Fast. Accurate. Only automate if it saves you time.
15. @rainforestqa rainforest The problem with automation Observer effect. Greater distance from manual = Greater chance false pos/neg.
16. @rainforestqa rainforest The problem with automation Good thing ^ Saves Time. Only worth automating a test if it saves time.
17. @rainforestqa rainforest #2.2.1 selenium The most popular option. You’re probably familiar with it.
18. @rainforestqa rainforest Easy to write. Accurate.
19. @rainforestqa rainforest Test in Selenium are Flakey.
20. @rainforestqa rainforest Infrastructure + Solid Test Environments + Wrappers How to fix? Better test environments. Easy to say, hard to do in reality.
21. @rainforestqa rainforest It’s synchronous, and slow. Slowness complicates CI.
22. @rainforestqa rainforest Selenium Grid + $$$ Fix slowness with Selenium Grid. It’s a hack, not a real fix.
23. @rainforestqa rainforest #2.2.2 headless
24. @rainforestqa rainforest PhantomJS, fast but ugly. Often written in JS, but there are wrappers in ruby, etc.
25. @rainforestqa rainforest Headless browsers are fast. Means devs will actually run it.
26. @rainforestqa rainforest Headless is Inaccurate. When accuracy is needed, use Selenium or Manual.
27. @rainforestqa rainforest PhantomJS ZombieJS SimpleBrowser Popular browsers are PhantomJS and ZombieJS. Selenium can be hacked to run Headless.
28. @rainforestqa rainforest #3: rainforest Human powered QA.
29. @rainforestqa rainforest Inbetween manual & automation Great for flakey tests, or for prototyping new tests. Not so great at checking your Robots.txt file.
30. @rainforestqa rainforest #4 which option is best? Which one should you use?
31. @rainforestqa rainforest ALL OF THEM. Silly to lock yourself into one solution.
32. @rainforestqa rainforest You will always need Exploratory testing Need more exploratory help? Use a service or hire.
33. @rainforestqa rainforest You will always need Regression testing Need more regressions? Use the right tool for the right job.
34. @rainforestqa rainforest There will always be fires to fight. On both sides.
35. rainforest @rainforestqa Questions? @rainforestqa @thatmightbepaul