Designing Tests Architecture That Does Not Suck

Slide 1

Slide 1 text

DESIGNING TEST ARCHITECTURE THAT DOES NOT SUCK MICHAEL BODNARCHUK @DAVERT

Slide 2

Slide 2 text

ABOUT ME Michael Bodnarchuk @davert Web developer from Kyiv, Ukraine Lead developer of Codeception testing framework Also author of CodeceptJS, Robo and others Tech Consultant, CTO at SDCLabs

Slide 3

Slide 3 text

I'M FROM

Slide 4

Slide 4 text

WE STRUGGLE REAL RUSSIAN INTERVENTION

Slide 5

Slide 5 text

WHAT OUR IT IS LIKE Outsource companies Magento o ce We ❤ Symfony, Yii, Phalcon, Laravel IT engineers are rich!

Slide 6

Slide 6 text

TESTING BUSINESS EXPECTATIONS

Slide 7

Slide 7 text

WHY DO WE TEST To ensure software works as expected To discover bugs in software (before users) To measure performance To seek for security issues

Slide 8

Slide 8 text

WHAT TESTS WE CAN AUTOMATE To ensure software works as expected To discover bugs in software (before users) To measure performance To seek for security issues

Slide 9

Slide 9 text

We automate tests to execute them at any time

Slide 10

Slide 10 text

AUTOMATED TESTING To establish trust For constant changes To stabilize current codebase

Slide 11

Slide 11 text

TESTING IS TOLD US TO BE LIKE THIS:

Slide 12

Slide 12 text

We talk about how to test but we don't say WHAT TO TEST

Slide 13

Slide 13 text

PRIORITY FIRST Crucial business scenarios Security cases Algorithms, functions with complex logic Everything that is hard to test manually

Slide 14

Slide 14 text

TESTS SHOULD BE Independent - not a ect each other Atomic - concentrated on one feature

Slide 15

Slide 15 text

START WITH GENERAL Feature: customer registration Background: Given I am unregistered customer Scenario: registering successfully When I register Then I should be registered

Slide 16

Slide 16 text

Slide 17

Slide 17 text

QUALITIES OF A TEST 1. Readability 2. Stability 3. Speed

Slide 18

Slide 18 text

READABILITY

Slide 19

Slide 19 text

TEST SHOULD BE EASY TO FOLLOW TEST SHOULD BE SIMPLE TO UPDATE CODE CAN BE REUSED TO TEST SIMILAR CASES

Slide 20

Slide 20 text

$request = $this->getRequest() ->setRequestUri('/user/profile/1') ->setParams(array('user_id'=>1)); $controller = $this->getMock( 'UserController', array('render'), array($request, $response, $request->getParams()) ); $controller->expects($this->once()) ->method('render') ->will($this->returnValue(true)); $this->assertTrue($controller->profileAction()); $this->assertTrue($controller->view->user_id == 1);

Slide 21

Slide 21 text

STABILITY

Slide 22

Slide 22 text

TEST SHOULD BE STABLE BY EXECUTION TEST SHOULD BE STABLE TO CHANGES

Slide 23

Slide 23 text

$mock = $this->getMock('Client', array('getInputFilter')); $mock->expects($this->once()) // is it important? ->method('getInputFilter') // hardcoded method name ->will($this->returnValue($preparedFilterObject)); $formFactory = $this->getMock('Symfony\Component\Form\FormFactoryInterfa $formFactory ->expects($this->once()) ->method('create') ->will($this->returnValue($form))

Slide 24

Slide 24 text

Codeception + WebDriver // what if HTML changes? $I->click('//body/div[3]/p[1]/div[2]/div/span'); // what if browser will render it longer? $I->wait(1);

Slide 25

Slide 25 text

HOW TO WRITE STABLE TESTS Don't mix speci cation with implementation Focus on result, not on the path Use interfaces for tests

Slide 26

Slide 26 text

WE NEED A TABLE

Slide 27

Slide 27 text

JUST BUY IT Blogpost: Expectation vs Implementation

Slide 28

Slide 28 text

HOW TO WRITE STABLE TESTS Don't mix speci cation with implementation Focus on result, not on the path Use interfaces for tests

Slide 29

Slide 29 text

FOCUS ON RESULT Will the test have to duplicate exactly the application code? Will assertions in the test duplicate any behavior covered by library code? Is this detail important, or is it only an internal concern? Blogpost: The Right Way To Test React Components

Slide 30

Slide 30 text

HOW TO WRITE STABLE TESTS Don't mix speci cation with implementation Focus on result, not on the path Use interfaces for tests

Slide 31

Slide 31 text

INTERFACES???

Slide 32

Slide 32 text

WHAT ARE INTERFACES Interface de ne rules to get things done Interfaces considered stable Interface is not just a keyword

Slide 33

Slide 33 text

5 STAGES OF INTERFACE CHANGE

Slide 34

Slide 34 text

ANCHOR TESTS TO STABLE PARTS: Web Interface Public API (REST, GraphQL, SOAP) PHP Interfaces Public Methods in Domain

Slide 35

Slide 35 text

CONSIDER WHAT IS STABLE FOR YOU

Slide 36

Slide 36 text

CAN WE TEST PRIVATE METHODS? Technically: yes Ideally: no Practically: yes, if you consider them stable

Slide 37

Slide 37 text

SPEED

Slide 38

Slide 38 text

FOR ONE TEST CASE: fast enough for instant feedback < 20 s FOR ALL TESTS should be run on CI easy to split into parallel processes < 20 min

Slide 39

Slide 39 text

QUESTIONS TO BE ASKED Should we sacri ce readability for speed? If so, why do you develop in PHP and not in C?

Slide 40

Slide 40 text

Think how you can test a feature with minimal e ort

Slide 41

Slide 41 text

TEST INFRASTRUCTURE Let's talk about implementation

Slide 42

Slide 42 text

OUTER AND INNER TESTING Outer: test from the public interface Inner: test from the source code

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

TEST TYPES Outer Acceptance: Browser-based UI tests Characterization: CURL-based request/response Inner Functional: Request/response emulation Integration: Service with its dependencies Unit: Service in pure isolation

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

HOW TO BUILD TEST ARCHITECTURE?

Slide 49

Slide 49 text

WHAT TO TEST Write down speci cations Choose speci cations which should be tested Write examples for speci cation Choose the testing layer

Slide 50

Slide 50 text

The more speci c example we need to test the more detailed layer we choose.

Slide 51

Slide 51 text

ACCEPTANCE VS FUNCTIONAL VS UNIT 1. Choose a testing layer where test would be Readable Stable Fast enough 2. Write a test 3. Repeat 4. Refactor!

Slide 52

Slide 52 text

UNIT VS INTEGRATION TESTS Unit Tests for pure functions algorithms complex data dozen execution paths Integration tests for everything else

Slide 53

Slide 53 text

MOCKS Are dangerous: A ect readability A ect stability Should be used for Async services 3rd-party services Remote services

Slide 54

Slide 54 text

Even you can write a unit test with mocks it doesn't mean you should

Slide 55

Slide 55 text

TDD || !TDD Hard to start (nothing is stable) Build on top of interfaces Use TDD to discover speci cations

Slide 56

Slide 56 text

BDD || !BDD Writing tests in English is not about BDD at all BDD transforms speci cation to tests BDD has its cost (additional abstraction layer) Use BDD when non-technical mates involved (when management is actually going to read your tests)

Slide 57

Slide 57 text

TEST ARCHITECTURE TEMPLATES

Slide 58

Slide 58 text

NEW PROJECT. HOW TO TEST? Domain Layer should have unit / integration tests Application layer should have integration / functional tests UI should have acceptance tests with positive scenarios

Slide 59

Slide 59 text

EARLY STAGES STARTUP. HOW TO TEST? Uncertainty Problem: We don't have strict requirements We can do a pivot any day We are unsure of EVERYTHING Solution: Test only when you stabilize the code Start with Public API, Domain Logic

Slide 60

Slide 60 text

LEGACY PROJECT. HOW TO TEST? Detect the critical parts of a system Write acceptance tests for them Refactor old code Cover the new code with unit tests

Slide 61

Slide 61 text

CONCLUSIONS 1. Discover what to test 2. Find a suitable level of testing 3. Write readable+stable+fast tests!

Slide 62

Slide 62 text

QUESTIONS! Michael Bodnarchuk @davert Author of Testing Framework Consultant & Trainer at Codeception SDCLabs