Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Beyond Testing

Beyond Testing

PHPSerbia 2017
* better unit tests
* managing test levels
* test data management
* REST API testing
* web ui testing
* parallel execution

Michael Bodnarchuk

May 28, 2017

More Decks by Michael Bodnarchuk

Other Decks in Programming


  1. Codeception http://codeception.com • Full stack testing framework • Based on

    top of PHPUnit • Have modules that provide common steps for testing • Designed for Growth: PageObjects, Helpers, Extensions • BDD-ready with Gherkin
  2. How to Write Unit / Integration Test • Separate configuration

    from a test • Do not use hierarchy for testcases (use traits) • Separate test code from support code • Make test simple and verbose
  3. Unit vs Integration Tests • Unit Tests ⇒ Domain Logic

    • Integration Tests ⇒ Infrastructure (+ Domain)
  4. • Does our architecture allow us to write unit tests?

    • Is unit testing enough for us? • Does it check User Interface / UX? Questions to be Asked
  5. Know Pros and Cons Stability to Changes Wide Coverage Invest

    into Infrastructure Speed of Development Stability of Execution Detailed Coverage Invest into Architecture Speed of Execution
  6. TDD Process 1. Write most outer API failing test 2.

    Implement a failing test for the domain 3. Implement domain logic 4. Implement infrastructure 5. Proceed until tests pass
  7. Every app can be tested! • Choose the testing level

    you are comfortable with • Don’t bother about speed of tests • Don’t wait for refactoring to write tests • Don’t take too much time into testing. Create business value • Do slow but constant refactoring
  8. Accessing Database • Using internal database connection (ORM) ◦ HINT:

    Rollback transaction in the end of a test • External database connection • External API ◦ HINT: Use REST API for data in acceptance tests
  9. Data Isolation Strategies • Create/delete data needed only for a

    single test • Recreate the database between tests ◦ HINT: Use a container to restart database after each test • Create non-intersecting data
  10. FACTORY_MUFFIN IN REAL LIFE $fm->define(User::class)->setDefinitions([ 'name' => Faker::name(), 'email' =>

    Faker::email(), 'body' => Faker::text(), // generate a profile and return its Id 'profile_id' => 'factory|Profile' );
  11. How To Test JSON Responses • by string comparison of

    response body • by data inclusion • by structure inclusion • by schema (Swagger, JSON-Schema)
  12. GET /tickets/3 { "ticket": { "id": 3, "from": "web", "description":

    "Lorem ipsum...", "priority": "important", "priority_value": 1, "report": { "user_agent": "Mozilla...", "url": "/tasks", "window": "1280x525", "resolution": "1600x1200" }, "reporter_info": { "name": "davert", "email": "[email protected]", } "created_at": "2016-08-21T20:16:37Z", "updated_at": "2016-09-11T15:13:47Z" } }
  13. $I->wantTo('get a ticket by its id'); $I->sendGET('/api/tickets/3'); $I->seeResponseCodeIs(HttpCode::OK); // 200

    $I->seeResponseIsJson(); // check data in response $I->seeResponseContainsJson([ 'ticket' => 'id' => 3, 'from' => 'web' 'report' => [ 'url' => '/tasks' ]]); How we test in Codeception: Data Inclusion
  14. // check the structure of response $I->seeResponseMatchesJsonType([ 'ticket' => [

    'id' => 'integer', 'description' => 'string|null', 'priority' => 'string', 'created_at' => 'string:date', 'reporter_info' => [ 'email' => 'string:email' ]]]); How we test in Codeception: Structure Inclusion
  15. $I->sendGET('/api/tickets/3'); // use custom helper $I->seeResponseMatchesSwaggerSchema('ticket'); How we test in

    Codeception: Schema Check • extend Codeception in Helper\Api • implement seeResponseMatchesSwaggerSchema
  16. Mocking APIs • Record and reuse API responses (php-vcr) ◦

    HINT: Use PHP-VCR library github.com/php-vcr/php-vcr • Mock and Stub APIs using HTTP server ◦ HINT: Use Phiremock library github.com/mcustiel/phiremock ◦ or Mountebank http://www.mbtest.org
  17. How Developers can Improve Acceptance Testing • Communicate with QA

    team • Suggest better locators HINT: add locator classes or data-attributes to HTML elements • Use the same language (PHP) • Solve data management issues (via REST API) • Suggest better tools
  18. Choose The Right Tool • Selenium + Browsers ◦ HINT:

    use Docker for headless browsers • PhantomJS (headless browser, unmaintained) • Cloud Testing Service (SauceLabs, BrowserStack) ◦ HINT: make sure you use them in your region • Browser emulation via HTTP client
  19. Atomic Acceptance Tests with Data Management Feature: CRUD for Post

    Scenario: When I create a post And I open a post And I edit a post Then I see it has changed Then I delete a post Feature: CRUD for Post Scenario: create a post Scenario: view post Scenario: edit a post Scenario: delete a post POST is created via API for each test which requires it One Post for everything :(
  20. SET IT UP • Manually ◦ Use multiple nodes ◦

    Run several concurrent processes ◦ Ensure data is not interfering • Using Docker ◦ Everything is isolated by design

    into container • Create script for running the tests • Use Jenkins Matrix to setup concurrent builds
  22. What’s inside the container? • Prepared databases • ./runtests.sh starts

    all required services (nginx, mysql, selenium, redis, etc) • Container stops when tests are finished • No supervisors: we can execute one process per run
  23. #!/bin/sh echo "Starting Services...." service elasticsearch start > /dev/null 2>&1

    service nginx start > /dev/null 2>&1 service php5-fpm start > /dev/null 2>&1 service mysql start > /dev/null 2>&1 phantomjs --webdriver=4444 > /dev/null 2>&1 & mailcatcher -f > /dev/null 2>&1 & echo "Running tests" cd /project/$1 # switch to application codecept run $2 # run tests from specific suite
  24. • Testing is not just about unit tests • Understand

    Pros and Cons of testing levels • Use proper data management strategy • Build a proper test infrastructure for your CI Constantly improve code by refactoring! It is safe to do this with tests.
  25. TIME FOR QUESTIONS! • My name is Michael Bodnarchuk •

    Twitter: @davert • GitHub: DavertMik • Projects: Codeception, CodeceptJS, Robo Task Runner