The practical test pyramid and how we build it

Testing a web application is not an easy task. Among all kind of tests (Unit, Integration, Screenshot, E2E), you have to choose wisely where to put your time and money. We are going to present you different strategies for building a practical test pyramid for your app. We will also share you some best practices for CI/CD and testing tools such as Protractor, Cypress and Saucelabs, etc

Transcript

1. Zhenyi ZHANG Software Engineer @ Criteo The practical test pyramid and how we build it
2. 2 • Full Stack Developer @ WebApps https://github.com/zhenyi2697 @zhenyi2697 Zhenyi ZHANG
3. 3 • • High level testing principles • Practical details about how we test • Focused on UI testing for Web applications • Opinionated What this talk is about
4. Why do we write tests ?
5. 5 • • Documentation • Safe refactoring • Reduce maintenance costs • Responsibility • Confidence ! Why do we write tests ?
6. 6 • Why do we write tests ? Ref: http://kcd.im/write-tests-slides
7. But… Testing is hard
8. 8 • • Hard to set up • Time consuming • Hard to maintain • Don’t know where to start Testing is hard
9. 9 • • Stateful • Change too often • Hard to split • Hard to debug Testing UI is extremely hard
10. How to find the good balance ?
11. 11 • Testing ROI (Return On Investment) Time spent writing tests Coverage / Confidence Sweet Spot Ref: Establishing testing patterns with software design principles by Aaron Abramov
12. 12 • The Testing Pyramid Unit Test Integration Test E2E Test
13. 13 • • Test functions & components separately • Easy to set up • Jasmine, Mocha, Jest, Qunit • Fast to run • Coverage support Unit Tests
14. 14 • Unit Test Too trivial Implementation details Too verbose Difficult to maintain
15. 15 • Unit Test
16. 16 • • Test the whole system ✅ • Difficult to set up ⛔ • Flaky ⛔ • Slow to run ⛔ • Late feedback ⛔ E2E Tests
17. 17 • • Good isolation ✅ • Verbose to write ⛔ • Not easy to debug ⛔ Components Integration Tests Screenshots Tests Backend-less Integration Tests • Test rendered CSS ✅ • Difficult to maintain ⛔ • Flaky ⛔ • Not self documentary ⛔ • Test against real DOM in a real browser ✅ • Easy to understand and visualize ✅ • Not very easy to split ⛔ Integration tests
18. 18 • The (UI) Testing Diamond Unit Test Integration Backend-less Test E2E Test Test function and business rules with unit tests Test components with backend-less integration tests Test the most critical paths with E2E tests Cheap Expensive Fast Slow
19. How to do this in practice ?
20. 20 • • No Selenium • Easy to set up • Developer friendly • Screenshot & video support • It’s open sourced and free ! We use Cypress
21. 21 • Cypress Architecture Cypress Browser App UI App Server XHR Request Interceptor Ref: Testing Angular with Cypress.io by Joe Eames C L
22. Cypress Demo
23. 23 • Best practices – Page Objects BasePage FeaturePage EEFeaturePage BLFeaturePage Mocks Mocks Mocks Mocks Mocks Widgets Mocks Mocks Requests • Defines shared functions for all pages • Defines element getters and actions • Include and use widgets • Use mocks or real backend requests to set up tests • Specs only interact with pages
24. 24 • • Alternatives • Use id • Use ‘js-’ prefixed classes • Use custom attribute such as ‘data-test’ • Reason for using custom attributes • CSS classes should only be used for UI rendering • Less confusion for future developers • Possible to be stripped off by building tool for release Best practices – Element Selectors
25. 25 • Best practices – Continuous Integration Development Pre-submit Post-submit • Manual check • Lint • Prettier check • Unit Test • Backend-less integration tests • E2E test
26. 26 • Best practices – E2E test settings Set up in DB • Run setup SQL before tests • Clean environment for each run • Hard to setup and difficult to maintain • Hard to debug locally Set up via UI • Go through UI or call endpoints • Possible to provide mocks for irreversible changes • Easy to setup • Requires cancellable operations
27. 27 • • Think about the testing ROI for your application • Configure Cypress & start writing E2E tests • Write backend-less integration tests for new features • Integrate them into CI Legacy project - Where to start ?
28. 28 • • Kent C. Dodds – Write tests. Not too many. Mostly integration • Brian Mann – I see your point, but… (Part 1) • Gleb Bahmutov – I see your point, but… (Part 2) • Joe Eames - Testing Angular with Cypress.io • Aaron Abramov – Establishing testing patterns with software design principles • Assert(js) Conf 2018 • Cypress Talks References & Inspirations
29. Thank you ! Questions ?