Beyond Testing for DutchPHP

Tools and approaches you need to know to build a good test suites for integration / functional / acceptance tests

Transcript

1. Dutch PHP 2016 Michael Bodnarchuk BEYOND TESTING
2. Who am I • Michael Bodnarchuk @davert • Web-developer from Kyiv, Ukraine • Lead developer of Codeception testing framework and others: CodeceptJS, Robo, AspectMock, etc
3. What do I do: Codeception http://codeception.com • Full stack testing framework • Based on top of PHPUnit • Have modules that provide common steps for testing • Designed for Growth: PageObjects, Helpers, Extensions • BDD-ready with Gherkin
4. Testing required for everyone
5. Unit Test All The Things!
6. • Is “unit testing” enough for us? • Does our architecture allow us to write unit tests? • Are we following TDD or not? • Does it really stands for quality? • Does it makes a perfect User Experience? Questions to be Asked
7. AND WHAT ABOUT REGRESSION TESTS?
8. We have a new fancy project! We’ve got legacy app We do great design No, we actually screwed it at some point Our unit tests proves everything done right “Register” button on site can’t be clicked TDD!!! DDD!!! BDD!!! Client is not going to pay for this Expectations vs Reality
9. Testing Levels • Unit • Integration • Functional • UI (Acceptance)
10. Know Pros and Cons Stability to Changes Wide Coverage Invest into Infrastructure Speed of Development Stability of Execution Detailed Coverage Invest into Architecture Speed of Execution
11. Remember, every app can be tested! • Don’t wait for refactoring to implement tests • Choose the testing level you are comfortable with • Don’t take too much time into testing. Create business value • Do slow but constant refactoring
12. Unit Testing! is not what I’m going to talk about
13. Beyond Testing what makes a test infrastructure
14. Data Management for integration / functional / acceptance
15. DATA ISOLATION STRATEGIES • Create/delete data needed only for a single test • Recreate the database between tests ◦ HINT: Use a container to restart database after each test • Create non-intersecting data
16. ACCESSING DATABASE • Using internal database connection (ORM) ◦ HINT: Rollback transaction in the end of a test • External database connection • External API ◦ HINT: Use REST API for data in acceptance tests
17. DEFINING TEST DATA WITH • Database Dumps • Fixtures • Factories (thephpleague/factory-muffin)
18. FACTORY_MUFFIN IN REAL LIFE
19. Tools For Testing isolate everything you don’t own
20. USING APIS: PHP VCR
21. PHP VCR • Records API interaction at first run • Replays API responses with recorded data • Allows API tests to be run locally
22. TESTING EMAILS: MAILCATCHER
23. MAILCATCHER • Ruby application with web interface • SMTP-mock • Provides REST API for sent emails • API can be used in tests • Alternative in Go - MailHog
24. WEB UI Testing why it is important for developers
25. Choose The Right Tool • Selenium + Browsers ◦ HINT: use Docker for headless browsers • PhantomJS (headless browser) • Cloud Testing Service (SauceLabs, BrowserStack) ◦ HINT: make sure you use them in your region • Browser emulation via HTTP client
26. How Developers can Improve Acceptance Testing • Communicate with QA team • Optimize usage of application (REST API for data) • Suggest better locators ◦ HINT: add locator classes or data-attributes to HTML elements • Suggest better tools • Use the same language (PHP)
27. Atomic Acceptance Tests with Data Management Feature: CRUD for Post Scenario: When I create a post And I open a post And I edit a post Then I see it has changed Then I delete a post Feature: CRUD for Post Scenario: create a post Scenario: view post Scenario: edit a post Scenario: delete a post POST is created via API for each test which requires it One Post for everything :(
28. Parallel Testing one day tests started to be slow
29. PARALLEL TESTING • Tests separation • Processes isolation
30. SET IT UP • Manually ◦ Use multiple nodes ◦ Run several concurrent processes ◦ Ensure data is not interfering • Using Docker ◦ Everything is isolated by design
31. HOW TO PARALLEL TESTS WITH DOCKER • Pack application into container ◦ HINT: Use docker-compose • Create script for running the tests • Use Jenkins Matrix to run concurrent builds
32. #!/bin/sh echo "Starting Services...." service elasticsearch start > /dev/null 2>&1 service nginx start > /dev/null 2>&1 service php5-fpm start > /dev/null 2>&1 service mysql start > /dev/null 2>&1 phantomjs --webdriver=4444 > /dev/null 2>&1 & mailcatcher -f > /dev/null 2>&1 & echo "Running tests" cd /project/$1 # switch to application codecept run $2 # run tests from specific suite
33. docker run -it -v $WORKSPACE:/project app ./runtests.sh $SUITE
34. Is that enough to be confident? No, get a feedback from real usage!
35. Monitoring • Advanced Google Analytics ◦ Hint: set Goals and Funnels to see if users are following right direction • Use Exception Notification • Logging Services • Realtime Customer Support ◦ Learn how users are using your applications
36. Detect Bugs from User Activity Metrics
37. Why do we need developers here? • To submit data via Google Analytics API • To place better tracking goals ◦ HINT: Track all valuable active elements in scenario • To detect bugs in technically perfect code
38. Conclusions
39. • Testing is not just about unit tests • Understand Pros and Cons of testing levels • Quality is not only Testing but UX and Monitoring too • Build a proper test infrastructure for your CI • Use proper data management strategy • Receive user feedback, write regressions tests • Constantly improving by refactoring! With tests you are secure to do this.
40. TIME FOR QUESTIONS! • My name is Michael Bodnarchuk • Twitter: @davert • GitHub: DavertMik • Projects: Codeception, AspectMock, Robo