Beyond Testing

Things you should know maintaining stable testing suite

Transcript

1. Michael Bodnarchuk BEYOND TESTING
2. WHO AM I • Michael Bodnarchuk @davert • Web-developer from Kyiv, Ukraine • Lead developer of Codeception testing framework and other OS tools like: Robo, AspectMock, etc
3. WHAT IS IT ALL ABOUT • What to do when PHPUnit is installed and first tests are done • What you should know writing a test • Choosing the best tool for testing • Creating test environment
4. KINDS OF TESTS • Acceptance • Functional • Integration • Unit
5. BEST PRACTICES SOME KIND OF
6. IDEAL TEST ▪Condition (Given) ▪Action (When) ▪Assertion (Then)
7. HOW TO WRITE TEST • Separate configuration from test • Do not use hierarchy for testcases (use traits when needed) • Separate test code from support code • Make test simple and verbose
8. DON'T DO IT THIS WAY move to configuration use traits
9. IMPLEMENT YOUR OWN ASSERTIONS custom assertions make code more readable
10. MAKE IT SIMPLER!
11. WHAT IS HARD TO TEST • Asynchronous stuff • Remote Services • Real Data
12. TESTS CRITERIA • Execution Stability ↓ • Stability to Changes ↑ • Speed ↓ • Coverage ↑ • Preciseness ↓ • Readability ↕ Unit => Integration => Functional => Acceptance
13. PRICE OF A FAULT REGRESSION TESTS
14. DATA MANAGEMENT HOW TO TEST DB INTERACTION
15. MANAGING TEST DATA WITH • Fixtures (nelmio/alice) • Dumps • Factories (thephpleague/factory-muffin)
16. FACTORY_MUFFIN IN REAL LIFE
17. DATA CLEANUP STRATEGIES • Manually create/delete data between tests • Recreate the database between tests • Using transactions – reverting changes with rollback
18. TOOLS FOR TESTING YOU SHOULD PROBABLY KNOW
19. PHPUNIT • Standard de-facto • Monolithic framework • Two mocking engines included (why not 3?) • JUnit, HTML, reports... and codecoverage • And other 100500 sometimes used features
20. PHPSPEC • TDD framework • Classes are generated from tests • Dependencies are described through mocking • Does not replace PHPUnit • For development, not for testing
21. BEHAT • BDD-framework • Ubiquitous language (Gherkin) • Acceptance testing via Mink (Selenium, Goutte, etc)
22. CODECEPTION • BDD-style testing framework • Scenario DSL for describing tests • Over 20 modules to cover the most of possible cases • Testing via Selenium, PhpBrowser, frameworks...
23. TESTING EMAILS: MAILCATCHER
24. MAILCATCHER • Ruby application with web interface • SMTP-mock • Provides REST API for sent emails • API can be used in tests • Alternative in Go - MailHog
25. TESTING WEB APPLICATIONS • Selenium • PhantomJS • Browser emulation via HTTP client
26. USING SELENIUM ON CI SERVER • Selenium Server • Xvfb – Virtual Framebuffer • Firefox or Chromium
27. INSTEAD OF CONFIGURING IT MANUALLY docker run -d -p 4444:4444 -e APP_HOST=myapp davert/selenium-env
28. WHAT IS DOCKER? • Docker is an open platform distributed apps • Runs and manages isolated Linux containers • Provides GitHub-like infrastructure
29. PARALLEL TESTING • Tests separation • Jobs isolation
30. HOW TO PARALLEL TESTS WITH DOCKER • Build container(s) for test environment • Run container(s) with Docker Compose • Use Jenkins Matrix to run concurrent builds
31. docker run -i -t -v $WORKSPACE:/project test_container ./runtests.sh $SUITE
32. CONCLUSIONS • Testing is not just about unit tests • Tests are code: make them easy to read and maintain • Testing is about tools, environment and infrastructure as well
33. QUESTIONS? • My name is Michael Bodnarchuk • Twitter: @davert • GitHub: DavertMik • ….and don't forget to try http://codeception.com