Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Beyond Testing

Beyond Testing

PHPSerbia 2017
* better unit tests
* managing test levels
* test data management
* REST API testing
* web ui testing
* parallel execution


Michael Bodnarchuk

May 28, 2017


  1. PHPSerbia 2017 Michael Bodnarchuk BEYOND TESTING

  2. Hi, I am Michael @davert and I have stickers!

  3. Codeception http://codeception.com • Full stack testing framework • Based on

    top of PHPUnit • Have modules that provide common steps for testing • Designed for Growth: PageObjects, Helpers, Extensions • BDD-ready with Gherkin
  4. Testing Better way to do it

  5. Ideal Test ▪Condition (Given) ▪Action (When) ▪Assertion (Then) ▪Action ▪Assertion

  6. DON'T DO IT THIS WAY move to configuration use traits

  7. IMPLEMENT YOUR OWN ASSERTIONS custom assertions make code more readable


  9. How to Write Unit / Integration Test • Separate configuration

    from a test • Do not use hierarchy for testcases (use traits) • Separate test code from support code • Make test simple and verbose
  10. Unit vs Integration Tests • Unit Tests ⇒ Domain Logic

    • Integration Tests ⇒ Infrastructure (+ Domain)
  11. • Does our architecture allow us to write unit tests?

    • Is unit testing enough for us? • Does it check User Interface / UX? Questions to be Asked
  12. Testing Levels • Unit • Integration • Functional • UI

  13. Know Pros and Cons Stability to Changes Wide Coverage Invest

    into Infrastructure Speed of Development Stability of Execution Detailed Coverage Invest into Architecture Speed of Execution
  14. Specification Detail

  15. TDD Process 1. Write most outer API failing test 2.

    Implement a failing test for the domain 3. Implement domain logic 4. Implement infrastructure 5. Proceed until tests pass
  16. Every app can be tested! • Choose the testing level

    you are comfortable with • Don’t bother about speed of tests • Don’t wait for refactoring to write tests • Don’t take too much time into testing. Create business value • Do slow but constant refactoring
  17. What is Hard to Test • Asynchronous stuff • Remote

    Services • Real Data
  18. Beyond Testing what they never tell you about testing

  19. Data Management for integration / functional / acceptance

  20. Accessing Database • Using internal database connection (ORM) ◦ HINT:

    Rollback transaction in the end of a test • External database connection • External API ◦ HINT: Use REST API for data in acceptance tests
  21. Data Isolation Strategies • Create/delete data needed only for a

    single test • Recreate the database between tests ◦ HINT: Use a container to restart database after each test • Create non-intersecting data
  22. Defining Test Data With • Database Dumps • Fixtures (nelmio/alice)

    • Factories (thephpleague/factory-muffin)
  23. FACTORY_MUFFIN IN REAL LIFE $fm->define(User::class)->setDefinitions([ 'name' => Faker::name(), 'email' =>

    Faker::email(), 'body' => Faker::text(), // generate a profile and return its Id 'profile_id' => 'factory|Profile' );
  24. Testing APIs REST APIs

  25. How To Test JSON Responses • by string comparison of

    response body • by data inclusion • by structure inclusion • by schema (Swagger, JSON-Schema)
  26. GET /tickets/3 { "ticket": { "id": 3, "from": "web", "description":

    "Lorem ipsum...", "priority": "important", "priority_value": 1, "report": { "user_agent": "Mozilla...", "url": "/tasks", "window": "1280x525", "resolution": "1600x1200" }, "reporter_info": { "name": "davert", "email": "davert@codeception.com", } "created_at": "2016-08-21T20:16:37Z", "updated_at": "2016-09-11T15:13:47Z" } }
  27. $I->wantTo('get a ticket by its id'); $I->sendGET('/api/tickets/3'); $I->seeResponseCodeIs(HttpCode::OK); // 200

    $I->seeResponseIsJson(); // check data in response $I->seeResponseContainsJson([ 'ticket' => 'id' => 3, 'from' => 'web' 'report' => [ 'url' => '/tasks' ]]); How we test in Codeception: Data Inclusion
  28. // check the structure of response $I->seeResponseMatchesJsonType([ 'ticket' => [

    'id' => 'integer', 'description' => 'string|null', 'priority' => 'string', 'created_at' => 'string:date', 'reporter_info' => [ 'email' => 'string:email' ]]]); How we test in Codeception: Structure Inclusion
  29. $I->sendGET('/api/tickets/3'); // use custom helper $I->seeResponseMatchesSwaggerSchema('ticket'); How we test in

    Codeception: Schema Check • extend Codeception in Helper\Api • implement seeResponseMatchesSwaggerSchema
  30. Mocking APIs • Record and reuse API responses (php-vcr) ◦

    HINT: Use PHP-VCR library github.com/php-vcr/php-vcr • Mock and Stub APIs using HTTP server ◦ HINT: Use Phiremock library github.com/mcustiel/phiremock ◦ or Mountebank http://www.mbtest.org
  31. WEB UI Testing why it is important for developers

  32. How Developers can Improve Acceptance Testing • Communicate with QA

    team • Suggest better locators HINT: add locator classes or data-attributes to HTML elements • Use the same language (PHP) • Solve data management issues (via REST API) • Suggest better tools
  33. Choose The Right Tool • Selenium + Browsers ◦ HINT:

    use Docker for headless browsers • PhantomJS (headless browser, unmaintained) • Cloud Testing Service (SauceLabs, BrowserStack) ◦ HINT: make sure you use them in your region • Browser emulation via HTTP client
  34. Atomic Acceptance Tests with Data Management Feature: CRUD for Post

    Scenario: When I create a post And I open a post And I edit a post Then I see it has changed Then I delete a post Feature: CRUD for Post Scenario: create a post Scenario: view post Scenario: edit a post Scenario: delete a post POST is created via API for each test which requires it One Post for everything :(
  35. Parallel Testing one day tests started to be slow

  36. PARALLEL TESTING • Tests separation • Processes isolation

  37. SET IT UP • Manually ◦ Use multiple nodes ◦

    Run several concurrent processes ◦ Ensure data is not interfering • Using Docker ◦ Everything is isolated by design

    into container • Create script for running the tests • Use Jenkins Matrix to setup concurrent builds
  39. What’s inside the container? • Prepared databases • ./runtests.sh starts

    all required services (nginx, mysql, selenium, redis, etc) • Container stops when tests are finished • No supervisors: we can execute one process per run
  40. #!/bin/sh echo "Starting Services...." service elasticsearch start > /dev/null 2>&1

    service nginx start > /dev/null 2>&1 service php5-fpm start > /dev/null 2>&1 service mysql start > /dev/null 2>&1 phantomjs --webdriver=4444 > /dev/null 2>&1 & mailcatcher -f > /dev/null 2>&1 & echo "Running tests" cd /project/$1 # switch to application codecept run $2 # run tests from specific suite
  41. docker run -it -v $WORKSPACE:/project app ./runtests.sh $SUITE

  42. Conclusions

  43. • Testing is not just about unit tests • Understand

    Pros and Cons of testing levels • Use proper data management strategy • Build a proper test infrastructure for your CI Constantly improve code by refactoring! It is safe to do this with tests.
  44. TIME FOR QUESTIONS! • My name is Michael Bodnarchuk •

    Twitter: @davert • GitHub: DavertMik • Projects: Codeception, CodeceptJS, Robo Task Runner