Beyond Testing - Speaker Deck

Slide 1

Slide 1 text

PHPSerbia 2017 Michael Bodnarchuk BEYOND TESTING

Slide 2

Slide 2 text

Hi, I am Michael @davert and I have stickers!

Slide 3

Slide 3 text

Codeception http://codeception.com ● Full stack testing framework ● Based on top of PHPUnit ● Have modules that provide common steps for testing ● Designed for Growth: PageObjects, Helpers, Extensions ● BDD-ready with Gherkin

Slide 4

Slide 4 text

Testing Better way to do it

Slide 5

Slide 5 text

Ideal Test ▪Condition (Given) ▪Action (When) ▪Assertion (Then) ▪Action ▪Assertion

Slide 6

Slide 6 text

DON'T DO IT THIS WAY move to configuration use traits

Slide 7

Slide 7 text

IMPLEMENT YOUR OWN ASSERTIONS custom assertions make code more readable

Slide 8

Slide 8 text

SIMPLIFY IT!

Slide 9

Slide 9 text

How to Write Unit / Integration Test ● Separate configuration from a test ● Do not use hierarchy for testcases (use traits) ● Separate test code from support code ● Make test simple and verbose

Slide 10

Slide 10 text

Unit vs Integration Tests ● Unit Tests ⇒ Domain Logic ● Integration Tests ⇒ Infrastructure (+ Domain)

Slide 11

Slide 11 text

● Does our architecture allow us to write unit tests? ● Is unit testing enough for us? ● Does it check User Interface / UX? Questions to be Asked

Slide 12

Slide 12 text

Testing Levels ● Unit ● Integration ● Functional ● UI (Acceptance)

Slide 13

Slide 13 text

Know Pros and Cons Stability to Changes Wide Coverage Invest into Infrastructure Speed of Development Stability of Execution Detailed Coverage Invest into Architecture Speed of Execution

Slide 14

Slide 14 text

Specification Detail

Slide 15

Slide 15 text

TDD Process 1. Write most outer API failing test 2. Implement a failing test for the domain 3. Implement domain logic 4. Implement infrastructure 5. Proceed until tests pass

Slide 16

Slide 16 text

Every app can be tested! ● Choose the testing level you are comfortable with ● Don’t bother about speed of tests ● Don’t wait for refactoring to write tests ● Don’t take too much time into testing. Create business value ● Do slow but constant refactoring

Slide 17

Slide 17 text

What is Hard to Test ● Asynchronous stuff ● Remote Services ● Real Data

Slide 18

Slide 18 text

Beyond Testing what they never tell you about testing

Slide 19

Slide 19 text

Data Management for integration / functional / acceptance

Slide 20

Slide 20 text

Accessing Database ● Using internal database connection (ORM) ○ HINT: Rollback transaction in the end of a test ● External database connection ● External API ○ HINT: Use REST API for data in acceptance tests

Slide 21

Slide 21 text

Data Isolation Strategies ● Create/delete data needed only for a single test ● Recreate the database between tests ○ HINT: Use a container to restart database after each test ● Create non-intersecting data

Slide 22

Slide 22 text

Defining Test Data With ● Database Dumps ● Fixtures (nelmio/alice) ● Factories (thephpleague/factory-muffin)

Slide 23

Slide 23 text

FACTORY_MUFFIN IN REAL LIFE $fm->define(User::class)->setDefinitions([ 'name' => Faker::name(), 'email' => Faker::email(), 'body' => Faker::text(), // generate a profile and return its Id 'profile_id' => 'factory|Profile' );

Slide 24

Slide 24 text

Testing APIs REST APIs

Slide 25

Slide 25 text

How To Test JSON Responses ● by string comparison of response body ● by data inclusion ● by structure inclusion ● by schema (Swagger, JSON-Schema)

Slide 26

Slide 26 text

GET /tickets/3 { "ticket": { "id": 3, "from": "web", "description": "Lorem ipsum...", "priority": "important", "priority_value": 1, "report": { "user_agent": "Mozilla...", "url": "/tasks", "window": "1280x525", "resolution": "1600x1200" }, "reporter_info": { "name": "davert", "email": "[email protected]", } "created_at": "2016-08-21T20:16:37Z", "updated_at": "2016-09-11T15:13:47Z" } }

Slide 27

Slide 27 text

$I->wantTo('get a ticket by its id'); $I->sendGET('/api/tickets/3'); $I->seeResponseCodeIs(HttpCode::OK); // 200 $I->seeResponseIsJson(); // check data in response $I->seeResponseContainsJson([ 'ticket' => 'id' => 3, 'from' => 'web' 'report' => [ 'url' => '/tasks' ]]); How we test in Codeception: Data Inclusion

Slide 28

Slide 28 text

// check the structure of response $I->seeResponseMatchesJsonType([ 'ticket' => [ 'id' => 'integer', 'description' => 'string|null', 'priority' => 'string', 'created_at' => 'string:date', 'reporter_info' => [ 'email' => 'string:email' ]]]); How we test in Codeception: Structure Inclusion

Slide 29

Slide 29 text

$I->sendGET('/api/tickets/3'); // use custom helper $I->seeResponseMatchesSwaggerSchema('ticket'); How we test in Codeception: Schema Check ● extend Codeception in Helper\Api ● implement seeResponseMatchesSwaggerSchema

Slide 30

Slide 30 text

Mocking APIs ● Record and reuse API responses (php-vcr) ○ HINT: Use PHP-VCR library github.com/php-vcr/php-vcr ● Mock and Stub APIs using HTTP server ○ HINT: Use Phiremock library github.com/mcustiel/phiremock ○ or Mountebank http://www.mbtest.org

Slide 31

Slide 31 text

WEB UI Testing why it is important for developers

Slide 32

Slide 32 text

How Developers can Improve Acceptance Testing ● Communicate with QA team ● Suggest better locators HINT: add locator classes or data-attributes to HTML elements ● Use the same language (PHP) ● Solve data management issues (via REST API) ● Suggest better tools

Slide 33

Slide 33 text

Choose The Right Tool ● Selenium + Browsers ○ HINT: use Docker for headless browsers ● PhantomJS (headless browser, unmaintained) ● Cloud Testing Service (SauceLabs, BrowserStack) ○ HINT: make sure you use them in your region ● Browser emulation via HTTP client

Slide 34

Slide 34 text

Atomic Acceptance Tests with Data Management Feature: CRUD for Post Scenario: When I create a post And I open a post And I edit a post Then I see it has changed Then I delete a post Feature: CRUD for Post Scenario: create a post Scenario: view post Scenario: edit a post Scenario: delete a post POST is created via API for each test which requires it One Post for everything :(

Slide 35

Slide 35 text

Parallel Testing one day tests started to be slow

Slide 36

Slide 36 text

PARALLEL TESTING ● Tests separation ● Processes isolation

Slide 37

Slide 37 text

SET IT UP ● Manually ○ Use multiple nodes ○ Run several concurrent processes ○ Ensure data is not interfering ● Using Docker ○ Everything is isolated by design

Slide 38

Slide 38 text

HOW TO RUN PARALLEL TESTS WITH DOCKER ● Pack application into container ● Create script for running the tests ● Use Jenkins Matrix to setup concurrent builds

Slide 39

Slide 39 text

What’s inside the container? ● Prepared databases ● ./runtests.sh starts all required services (nginx, mysql, selenium, redis, etc) ● Container stops when tests are finished ● No supervisors: we can execute one process per run

Slide 40

Slide 40 text

#!/bin/sh echo "Starting Services...." service elasticsearch start > /dev/null 2>&1 service nginx start > /dev/null 2>&1 service php5-fpm start > /dev/null 2>&1 service mysql start > /dev/null 2>&1 phantomjs --webdriver=4444 > /dev/null 2>&1 & mailcatcher -f > /dev/null 2>&1 & echo "Running tests" cd /project/$1 # switch to application codecept run $2 # run tests from specific suite

Slide 41

Slide 41 text

docker run -it -v $WORKSPACE:/project app ./runtests.sh $SUITE

Slide 42

Slide 42 text

Conclusions

Slide 43

Slide 43 text

● Testing is not just about unit tests ● Understand Pros and Cons of testing levels ● Use proper data management strategy ● Build a proper test infrastructure for your CI Constantly improve code by refactoring! It is safe to do this with tests.

Slide 44

Slide 44 text

TIME FOR QUESTIONS! ● My name is Michael Bodnarchuk ● Twitter: @davert ● GitHub: DavertMik ● Projects: Codeception, CodeceptJS, Robo Task Runner