Use the Right Tool

A lot of companies use one framework and only one for their testing. Why do that, when headless browsers are such a beautiful compliment to the traditional approach?

Transcript

1. rainforest @rainforestqa
   Your QA options
   (Wait… there’s more than one?)
   Selenium = only way most people test apps.
   Bad for a lot of reasons. The most obvious is mobile.
   

   View Slide

2. @rainforestqa
   rainforest
   Paul Burt
   QA Evangelist
   How did you get started in QA?
   What do you do at Rainforest?
   

   View Slide

3. @rainforestqa
   rainforest
   Paul Burt
   Dev Evangelist, but won’t discuss Jesus.
   

   View Slide

4. @rainforestqa
   rainforest
   #1: so, what is QA?
   The purpose of QA = discover unknowns.
   QA won’t fix quality issues, just Identifies.
   

   View Slide

5. @rainforestqa
   rainforest
   #1.1 how I think about QA
   QA is not a solvable problem.
   Two main components: Exploratory + Regressions.
   

   View Slide

6. @rainforestqa
   rainforest
   In QA, you have limited moves.
   Spend them wisely.
   

   View Slide

7. @rainforestqa
   rainforest
   #2: current landscape
   Let’s look at the current options.
   

   View Slide

8. @rainforestqa
   rainforest
   #2.1 manual
   Comes free.
   Half of QA is finding another set of eyes.
   

   View Slide

9. @rainforestqa
   rainforest
   Problem: manual testing is tedious.
   

   View Slide

10. @rainforestqa
    rainforest
    You can outsource,
    or hire a service.
    Save time by hiring more people.
    Scaling is hard.
    Try outside services.
    

    View Slide

11. @rainforestqa
    rainforest
    Problem: You won’t sleep easy while manually testing.
    Business Owners want scalability + reliability.
    

    View Slide

12. @rainforestqa
    rainforest
    Manual testing is inconsistent.
    Exploratory testing makes manual a necessity.
    Regression testing is what you automate.
    

    View Slide

13. @rainforestqa
    rainforest
    #2.2 automation
    Customer service + PMs can help do manual.
    Not true w/ automation. Requires engineers.
    

    View Slide

14. @rainforestqa
    rainforest
    What makes a good
    automated test?
    Fast.
    Accurate.
    Only automate if it saves you time.
    

    View Slide

15. @rainforestqa
    rainforest
    The problem with automation
    Observer effect.
    Greater distance from manual = Greater chance false pos/neg.
    

    View Slide

16. @rainforestqa
    rainforest
    The problem with automation
    Good thing
    ^
    Saves Time.
    Only worth automating a test if it saves time.
    

    View Slide

17. @rainforestqa
    rainforest
    #2.2.1 selenium
    The most popular option.
    You’re probably familiar with it.
    

    View Slide

18. @rainforestqa
    rainforest
    Easy to write.
    Accurate.
    

    View Slide

19. @rainforestqa
    rainforest
    Test in Selenium are Flakey.
    

    View Slide

20. @rainforestqa
    rainforest
    Infrastructure
    + Solid Test Environments
    + Wrappers
    How to fix? Better test environments.
    Easy to say, hard to do in reality.
    

    View Slide

21. @rainforestqa
    rainforest
    It’s synchronous, and slow.
    Slowness complicates CI.
    

    View Slide

22. @rainforestqa
    rainforest
    Selenium Grid + $$$
    Fix slowness with Selenium Grid.
    It’s a hack, not a real fix.
    

    View Slide

23. @rainforestqa
    rainforest
    #2.2.2 headless
    

    View Slide

24. @rainforestqa
    rainforest
    PhantomJS, fast but ugly.
    Often written in JS, but there are wrappers in ruby, etc.
    

    View Slide

25. @rainforestqa
    rainforest
    Headless browsers are fast.
    Means devs will actually run it.
    

    View Slide

26. @rainforestqa
    rainforest
    Headless is Inaccurate.
    When accuracy is needed, use Selenium or Manual.
    

    View Slide

27. @rainforestqa
    rainforest
    PhantomJS
    ZombieJS
    SimpleBrowser
    Popular browsers are PhantomJS and ZombieJS.
    Selenium can be hacked to run Headless.
    

    View Slide

28. @rainforestqa
    rainforest
    #3: rainforest
    Human powered QA.
    

    View Slide

29. @rainforestqa
    rainforest
    Inbetween
    manual & automation
    Great for flakey tests, or for prototyping new tests.
    Not so great at checking your Robots.txt file.
    

    View Slide

30. @rainforestqa
    rainforest
    #4 which option is best?
    Which one should you use?
    

    View Slide

31. @rainforestqa
    rainforest
    ALL OF THEM.
    Silly to lock yourself into one solution.
    

    View Slide

32. @rainforestqa
    rainforest
    You will always need
    Exploratory testing
    Need more exploratory help?
    Use a service or hire.
    

    View Slide

33. @rainforestqa
    rainforest
    You will always need
    Regression testing
    Need more regressions?
    Use the right tool for the right job.
    

    View Slide

34. @rainforestqa
    rainforest
    There will always be fires to fight.
    On both sides.
    

    View Slide

35. rainforest @rainforestqa
    Questions?
    @rainforestqa
    @thatmightbepaul
    

    View Slide