Beyond Testing

PHPSerbia 2017
* better unit tests
* managing test levels
* test data management
* REST API testing
* web ui testing
* parallel execution

Transcript

1. PHPSerbia 2017 Michael Bodnarchuk BEYOND TESTING
2. Hi, I am Michael @davert and I have stickers!
3. Codeception http://codeception.com • Full stack testing framework • Based on top of PHPUnit • Have modules that provide common steps for testing • Designed for Growth: PageObjects, Helpers, Extensions • BDD-ready with Gherkin
4. Testing Better way to do it
5. Ideal Test ▪Condition (Given) ▪Action (When) ▪Assertion (Then) ▪Action ▪Assertion
6. DON'T DO IT THIS WAY move to configuration use traits
7. IMPLEMENT YOUR OWN ASSERTIONS custom assertions make code more readable
8. SIMPLIFY IT! 
9. How to Write Unit / Integration Test • Separate configuration from a test • Do not use hierarchy for testcases (use traits) • Separate test code from support code • Make test simple and verbose
10. Unit vs Integration Tests • Unit Tests ⇒ Domain Logic • Integration Tests ⇒ Infrastructure (+ Domain)
11. • Does our architecture allow us to write unit tests? • Is unit testing enough for us? • Does it check User Interface / UX? Questions to be Asked
12. Testing Levels • Unit • Integration • Functional • UI (Acceptance)
13. Know Pros and Cons Stability to Changes Wide Coverage Invest into Infrastructure Speed of Development Stability of Execution Detailed Coverage Invest into Architecture Speed of Execution
14. Specification Detail
15. TDD Process 1. Write most outer API failing test 2. Implement a failing test for the domain 3. Implement domain logic 4. Implement infrastructure 5. Proceed until tests pass
16. Every app can be tested! • Choose the testing level you are comfortable with • Don’t bother about speed of tests • Don’t wait for refactoring to write tests • Don’t take too much time into testing. Create business value • Do slow but constant refactoring
17. What is Hard to Test • Asynchronous stuff • Remote Services • Real Data
18. Beyond Testing what they never tell you about testing
19. Data Management for integration / functional / acceptance
20. Accessing Database • Using internal database connection (ORM) ◦ HINT: Rollback transaction in the end of a test • External database connection • External API ◦ HINT: Use REST API for data in acceptance tests
21. Data Isolation Strategies • Create/delete data needed only for a single test • Recreate the database between tests ◦ HINT: Use a container to restart database after each test • Create non-intersecting data
22. Defining Test Data With • Database Dumps • Fixtures (nelmio/alice) • Factories (thephpleague/factory-muffin)
23. FACTORY_MUFFIN IN REAL LIFE $fm->define(User::class)->setDefinitions([ 'name' => Faker::name(), 'email' => Faker::email(), 'body' => Faker::text(), // generate a profile and return its Id 'profile_id' => 'factory|Profile' );
24. Testing APIs REST APIs
25. How To Test JSON Responses • by string comparison of response body • by data inclusion • by structure inclusion • by schema (Swagger, JSON-Schema)
26. GET /tickets/3 { "ticket": { "id": 3, "from": "web", "description": "Lorem ipsum...", "priority": "important", "priority_value": 1, "report": { "user_agent": "Mozilla...", "url": "/tasks", "window": "1280x525", "resolution": "1600x1200" }, "reporter_info": { "name": "davert", "email": "davert@codeception.com", } "created_at": "2016-08-21T20:16:37Z", "updated_at": "2016-09-11T15:13:47Z" } }
27. $I->wantTo('get a ticket by its id'); $I->sendGET('/api/tickets/3'); $I->seeResponseCodeIs(HttpCode::OK); // 200 $I->seeResponseIsJson(); // check data in response $I->seeResponseContainsJson([ 'ticket' => 'id' => 3, 'from' => 'web' 'report' => [ 'url' => '/tasks' ]]); How we test in Codeception: Data Inclusion
28. // check the structure of response $I->seeResponseMatchesJsonType([ 'ticket' => [ 'id' => 'integer', 'description' => 'string|null', 'priority' => 'string', 'created_at' => 'string:date', 'reporter_info' => [ 'email' => 'string:email' ]]]); How we test in Codeception: Structure Inclusion
29. $I->sendGET('/api/tickets/3'); // use custom helper $I->seeResponseMatchesSwaggerSchema('ticket'); How we test in Codeception: Schema Check • extend Codeception in Helper\Api • implement seeResponseMatchesSwaggerSchema
30. Mocking APIs • Record and reuse API responses (php-vcr) ◦ HINT: Use PHP-VCR library github.com/php-vcr/php-vcr • Mock and Stub APIs using HTTP server ◦ HINT: Use Phiremock library github.com/mcustiel/phiremock ◦ or Mountebank http://www.mbtest.org
31. WEB UI Testing why it is important for developers
32. How Developers can Improve Acceptance Testing • Communicate with QA team • Suggest better locators HINT: add locator classes or data-attributes to HTML elements • Use the same language (PHP) • Solve data management issues (via REST API) • Suggest better tools
33. Choose The Right Tool • Selenium + Browsers ◦ HINT: use Docker for headless browsers • PhantomJS (headless browser, unmaintained) • Cloud Testing Service (SauceLabs, BrowserStack) ◦ HINT: make sure you use them in your region • Browser emulation via HTTP client
34. Atomic Acceptance Tests with Data Management Feature: CRUD for Post Scenario: When I create a post And I open a post And I edit a post Then I see it has changed Then I delete a post Feature: CRUD for Post Scenario: create a post Scenario: view post Scenario: edit a post Scenario: delete a post POST is created via API for each test which requires it One Post for everything :(
35. Parallel Testing one day tests started to be slow
36. PARALLEL TESTING • Tests separation • Processes isolation
37. SET IT UP • Manually ◦ Use multiple nodes ◦ Run several concurrent processes ◦ Ensure data is not interfering • Using Docker ◦ Everything is isolated by design
38. HOW TO RUN PARALLEL TESTS WITH DOCKER • Pack application into container • Create script for running the tests • Use Jenkins Matrix to setup concurrent builds
39. What’s inside the container? • Prepared databases • ./runtests.sh starts all required services (nginx, mysql, selenium, redis, etc) • Container stops when tests are finished • No supervisors: we can execute one process per run
40. #!/bin/sh echo "Starting Services...." service elasticsearch start > /dev/null 2>&1 service nginx start > /dev/null 2>&1 service php5-fpm start > /dev/null 2>&1 service mysql start > /dev/null 2>&1 phantomjs --webdriver=4444 > /dev/null 2>&1 & mailcatcher -f > /dev/null 2>&1 & echo "Running tests" cd /project/$1 # switch to application codecept run $2 # run tests from specific suite
41. docker run -it -v $WORKSPACE:/project app ./runtests.sh $SUITE
42. Conclusions
43. • Testing is not just about unit tests • Understand Pros and Cons of testing levels • Use proper data management strategy • Build a proper test infrastructure for your CI Constantly improve code by refactoring! It is safe to do this with tests.
44. TIME FOR QUESTIONS! • My name is Michael Bodnarchuk • Twitter: @davert • GitHub: DavertMik • Projects: Codeception, CodeceptJS, Robo Task Runner