Beyond Testing for DutchPHP

Beyond Testing for DutchPHP

Tools and approaches you need to know to build a good test suites for integration / functional / acceptance tests


Michael Bodnarchuk

June 24, 2016


  1. Dutch PHP 2016 Michael Bodnarchuk BEYOND TESTING

  2. Who am I • Michael Bodnarchuk @davert • Web-developer from

    Kyiv, Ukraine • Lead developer of Codeception testing framework and others: CodeceptJS, Robo, AspectMock, etc
  3. What do I do: Codeception • Full stack testing

    framework • Based on top of PHPUnit • Have modules that provide common steps for testing • Designed for Growth: PageObjects, Helpers, Extensions • BDD-ready with Gherkin
  4. Testing required for everyone

  5. Unit Test All The Things!

  6. • Is “unit testing” enough for us? • Does our

    architecture allow us to write unit tests? • Are we following TDD or not? • Does it really stands for quality? • Does it makes a perfect User Experience? Questions to be Asked

  8. We have a new fancy project! We’ve got legacy app

    We do great design No, we actually screwed it at some point Our unit tests proves everything done right “Register” button on site can’t be clicked TDD!!! DDD!!! BDD!!! Client is not going to pay for this Expectations vs Reality
  9. Testing Levels • Unit • Integration • Functional • UI

  10. Know Pros and Cons Stability to Changes Wide Coverage Invest

    into Infrastructure Speed of Development Stability of Execution Detailed Coverage Invest into Architecture Speed of Execution
  11. Remember, every app can be tested! • Don’t wait for

    refactoring to implement tests • Choose the testing level you are comfortable with • Don’t take too much time into testing. Create business value • Do slow but constant refactoring
  12. Unit Testing! is not what I’m going to talk about

  13. Beyond Testing what makes a test infrastructure

  14. Data Management for integration / functional / acceptance

  15. DATA ISOLATION STRATEGIES • Create/delete data needed only for a

    single test • Recreate the database between tests ◦ HINT: Use a container to restart database after each test • Create non-intersecting data
  16. ACCESSING DATABASE • Using internal database connection (ORM) ◦ HINT:

    Rollback transaction in the end of a test • External database connection • External API ◦ HINT: Use REST API for data in acceptance tests
  17. DEFINING TEST DATA WITH • Database Dumps • Fixtures •

    Factories (thephpleague/factory-muffin)

  19. Tools For Testing isolate everything you don’t own


  21. PHP VCR • Records API interaction at first run •

    Replays API responses with recorded data • Allows API tests to be run locally

  23. MAILCATCHER • Ruby application with web interface • SMTP-mock •

    Provides REST API for sent emails • API can be used in tests • Alternative in Go - MailHog
  24. WEB UI Testing why it is important for developers

  25. Choose The Right Tool • Selenium + Browsers ◦ HINT:

    use Docker for headless browsers • PhantomJS (headless browser) • Cloud Testing Service (SauceLabs, BrowserStack) ◦ HINT: make sure you use them in your region • Browser emulation via HTTP client
  26. How Developers can Improve Acceptance Testing • Communicate with QA

    team • Optimize usage of application (REST API for data) • Suggest better locators ◦ HINT: add locator classes or data-attributes to HTML elements • Suggest better tools • Use the same language (PHP)
  27. Atomic Acceptance Tests with Data Management Feature: CRUD for Post

    Scenario: When I create a post And I open a post And I edit a post Then I see it has changed Then I delete a post Feature: CRUD for Post Scenario: create a post Scenario: view post Scenario: edit a post Scenario: delete a post POST is created via API for each test which requires it One Post for everything :(
  28. Parallel Testing one day tests started to be slow

  29. PARALLEL TESTING • Tests separation • Processes isolation

  30. SET IT UP • Manually ◦ Use multiple nodes ◦

    Run several concurrent processes ◦ Ensure data is not interfering • Using Docker ◦ Everything is isolated by design
  31. HOW TO PARALLEL TESTS WITH DOCKER • Pack application into

    container ◦ HINT: Use docker-compose • Create script for running the tests • Use Jenkins Matrix to run concurrent builds
  32. #!/bin/sh echo "Starting Services...." service elasticsearch start > /dev/null 2>&1

    service nginx start > /dev/null 2>&1 service php5-fpm start > /dev/null 2>&1 service mysql start > /dev/null 2>&1 phantomjs --webdriver=4444 > /dev/null 2>&1 & mailcatcher -f > /dev/null 2>&1 & echo "Running tests" cd /project/$1 # switch to application codecept run $2 # run tests from specific suite
  33. docker run -it -v $WORKSPACE:/project app ./ $SUITE

  34. Is that enough to be confident? No, get a feedback

    from real usage!
  35. Monitoring • Advanced Google Analytics ◦ Hint: set Goals and

    Funnels to see if users are following right direction • Use Exception Notification • Logging Services • Realtime Customer Support ◦ Learn how users are using your applications
  36. Detect Bugs from User Activity Metrics

  37. Why do we need developers here? • To submit data

    via Google Analytics API • To place better tracking goals ◦ HINT: Track all valuable active elements in scenario • To detect bugs in technically perfect code
  38. Conclusions

  39. • Testing is not just about unit tests • Understand

    Pros and Cons of testing levels • Quality is not only Testing but UX and Monitoring too • Build a proper test infrastructure for your CI • Use proper data management strategy • Receive user feedback, write regressions tests • Constantly improving by refactoring! With tests you are secure to do this.
  40. TIME FOR QUESTIONS! • My name is Michael Bodnarchuk •

    Twitter: @davert • GitHub: DavertMik • Projects: Codeception, AspectMock, Robo