$30 off During Our Annual Pro Sale. View Details »

Use the Right Tool

Use the Right Tool

A lot of companies use one framework and only one for their testing. Why do that, when headless browsers are such a beautiful compliment to the traditional approach?

Paul Burt

April 03, 2015
Tweet

More Decks by Paul Burt

Other Decks in Technology

Transcript

  1. rainforest @rainforestqa
    Your QA options
    (Wait… there’s more than one?)
    Selenium = only way most people test apps.
    Bad for a lot of reasons. The most obvious is mobile.

    View Slide

  2. @rainforestqa
    rainforest
    Paul Burt
    QA Evangelist
    How did you get started in QA?
    What do you do at Rainforest?

    View Slide

  3. @rainforestqa
    rainforest
    Paul Burt
    Dev Evangelist, but won’t discuss Jesus.

    View Slide

  4. @rainforestqa
    rainforest
    #1: so, what is QA?
    The purpose of QA = discover unknowns.
    QA won’t fix quality issues, just Identifies.

    View Slide

  5. @rainforestqa
    rainforest
    #1.1 how I think about QA
    QA is not a solvable problem.
    Two main components: Exploratory + Regressions.

    View Slide

  6. @rainforestqa
    rainforest
    In QA, you have limited moves.
    Spend them wisely.

    View Slide

  7. @rainforestqa
    rainforest
    #2: current landscape
    Let’s look at the current options.

    View Slide

  8. @rainforestqa
    rainforest
    #2.1 manual
    Comes free.
    Half of QA is finding another set of eyes.

    View Slide

  9. @rainforestqa
    rainforest
    Problem: manual testing is tedious.

    View Slide

  10. @rainforestqa
    rainforest
    You can outsource,
    or hire a service.
    Save time by hiring more people.
    Scaling is hard.
    Try outside services.

    View Slide

  11. @rainforestqa
    rainforest
    Problem: You won’t sleep easy while manually testing.
    Business Owners want scalability + reliability.

    View Slide

  12. @rainforestqa
    rainforest
    Manual testing is inconsistent.
    Exploratory testing makes manual a necessity.
    Regression testing is what you automate.

    View Slide

  13. @rainforestqa
    rainforest
    #2.2 automation
    Customer service + PMs can help do manual.
    Not true w/ automation. Requires engineers.

    View Slide

  14. @rainforestqa
    rainforest
    What makes a good
    automated test?
    Fast.
    Accurate.
    Only automate if it saves you time.

    View Slide

  15. @rainforestqa
    rainforest
    The problem with automation
    Observer effect.
    Greater distance from manual = Greater chance false pos/neg.

    View Slide

  16. @rainforestqa
    rainforest
    The problem with automation
    Good thing
    ^
    Saves Time.
    Only worth automating a test if it saves time.

    View Slide

  17. @rainforestqa
    rainforest
    #2.2.1 selenium
    The most popular option.
    You’re probably familiar with it.

    View Slide

  18. @rainforestqa
    rainforest
    Easy to write.
    Accurate.

    View Slide

  19. @rainforestqa
    rainforest
    Test in Selenium are Flakey.

    View Slide

  20. @rainforestqa
    rainforest
    Infrastructure
    + Solid Test Environments
    + Wrappers
    How to fix? Better test environments.
    Easy to say, hard to do in reality.

    View Slide

  21. @rainforestqa
    rainforest
    It’s synchronous, and slow.
    Slowness complicates CI.

    View Slide

  22. @rainforestqa
    rainforest
    Selenium Grid + $$$
    Fix slowness with Selenium Grid.
    It’s a hack, not a real fix.

    View Slide

  23. @rainforestqa
    rainforest
    #2.2.2 headless

    View Slide

  24. @rainforestqa
    rainforest
    PhantomJS, fast but ugly.
    Often written in JS, but there are wrappers in ruby, etc.

    View Slide

  25. @rainforestqa
    rainforest
    Headless browsers are fast.
    Means devs will actually run it.

    View Slide

  26. @rainforestqa
    rainforest
    Headless is Inaccurate.
    When accuracy is needed, use Selenium or Manual.

    View Slide

  27. @rainforestqa
    rainforest
    PhantomJS
    ZombieJS
    SimpleBrowser
    Popular browsers are PhantomJS and ZombieJS.
    Selenium can be hacked to run Headless.

    View Slide

  28. @rainforestqa
    rainforest
    #3: rainforest
    Human powered QA.

    View Slide

  29. @rainforestqa
    rainforest
    Inbetween
    manual & automation
    Great for flakey tests, or for prototyping new tests.
    Not so great at checking your Robots.txt file.

    View Slide

  30. @rainforestqa
    rainforest
    #4 which option is best?
    Which one should you use?

    View Slide

  31. @rainforestqa
    rainforest
    ALL OF THEM.
    Silly to lock yourself into one solution.

    View Slide

  32. @rainforestqa
    rainforest
    You will always need
    Exploratory testing
    Need more exploratory help?
    Use a service or hire.

    View Slide

  33. @rainforestqa
    rainforest
    You will always need
    Regression testing
    Need more regressions?
    Use the right tool for the right job.

    View Slide

  34. @rainforestqa
    rainforest
    There will always be fires to fight.
    On both sides.

    View Slide

  35. rainforest @rainforestqa
    Questions?
    @rainforestqa
    @thatmightbepaul

    View Slide