Next-Generation Web Application End-to-End Testing with Cypress

This talk was originally presented at Devoxx Belgium 2018.
-
We all want to consistently ship quality software. One of the best ways to do that is by end-to-end (E2E) testing our code as it simulates the real actions of our beloved users. Let’s be frank though, E2E testing is notoriously time-consuming, expensive, slow, and often flaky. Unfortunately, these issues lead to developers doing very little to no E2E testing, and taking solace in a plethora of unit-tests that cannot fully test the intended experience of users.

We can do better. We should do better. We will do better.

We’ll explore modern tools and libraries, such as Cypress, that enable us to easily and beautifully write flake-free, reliable, deterministic E2E tests for our web applications. E2E tests will give us confidence in the software we ship down the wire.

Transcript

1. Next-Generation Web Application End-to-End Testing Amir Rustamzadeh @amirrustam Devoxx Belgium - 2018
2. Sr. Software Engineer @amirrustam Amir Rustamzadeh amir@cypress.io
3. @amirrustam ! E2E testing? I spend hours chasing flaky tests! " # Too expensive for project budget. In our last project, E2E tests slowed down development.
4. E2E Integration Unit Visit The Testing Pyramid @amirrustam
5. E2E Integration Unit Visit The Testing Pyramid Easy, developers really like to write these. @amirrustam
6. E2E Integration Unit Visit The Testing Pyramid Easy, developers really like to write these. More Difficult, costly, slower… Often completely ignored @amirrustam
7. E2E Integration Unit Visit The Testing Pyramid Best for users & business @amirrustam
8. Don’t let your users be your first E2E testers. @amirrustam
9. Fast, easy and reliable testing for anything that runs in a browser. Free. Open Source. MIT License @amirrustam
10. $ npm install -D cypress ☝ All-in-one tool You get everything you need ❤ Familiar tools underneath Mocha, Chia, Sinon @amirrustam
11. @amirrustam
12. @amirrustam
13. @amirrustam Let’s see it in action…. https://github.com/cypress-io/cypress-example-todomvc
14. ) Fast Test Runs @amirrustam
15. Cypress Architecture Cypress Test Runner Your App @amirrustam Browser
16. ⚡ Native-access to the DOM and your app. @amirrustam
17. @amirrustam Username Password Login You can mutate state programmatically. There is no need to always manipulate the UI to create the desired state.
18. ⚙ Component Testing cypress-react-unit-testing vue angular angularjs hyper-app svelte cycle https://github.com/bahmutov/cypress-vue-unit-test#test-adapters-for-other-frameworks @amirrustam
19. Component Mounting Cypress Test Runner My Component My Component @amirrustam Browser
20. @amirrustam import mount from 'cypress-vue-unit-test' import MyComponent from './app/components/MyComponent.vue' describe('My Component', () => { beforeEach(mount(MyComponent)) it('loads', () => { cy.get('button').should('be.visible') }) })
21. ⏳ Automatic Waiting Demo @amirrustam
22. 0 Easy stubbing, mocking, and spying. @amirrustam
23. @amirrustam it('starts with zero items', () => { // start Cypress network server // spy on route `GET /todos` // THEN visit the page cy.server() cy.route('GET', '/todos').as('todos') cy.visit('/') cy .wait('@todos') // wait for `GET /todos` response // inspect the server's response .its('response.body') .should('have.length', 0) // then check the DOM cy.get('li.todo').should('have.length', 0) })
24. @amirrustam it('posts new item to the server', () => { cy.server() cy.route('POST', '/todos').as('new-item') cy.visit('/') cy.get(‘.new-todo').type('buy cookies{enter}') cy.wait('@new-item').its('request.body').should('have.contain', { title: 'buy cookies', completed: false }) })
25. @amirrustam it('starts with zero items (stubbed response)', () => { // start Cypress network server // spy on route `GET /todos` // THEN visit the page cy.server() cy.route('GET', '/todos', []).as('todos') cy.visit('/') cy .wait('@todos') // wait for `GET /todos` response // inspect the server's response .its('response.body') .should('have.length', 0) // then check the DOM cy.get('li.todo').should('have.length', 0) })
26. @amirrustam it('starts with zero items (fixture)', () => { // start Cypress network server // stub route `GET /todos`, return data from fixture file // THEN visit the page cy.server() cy.route('GET', '/todos', ‘fixture:sample-todos').as('todos') cy.visit('/') cy .wait('@todos') // wait for `GET /todos` response // inspect the server's response .its('response.body') .should('have.length', 0) // then check the DOM cy.get('li.todo').should('have.length', 0) })
27. Flake-free, consistent, and reliable. @amirrustam
28. @amirrustam How are commands executed? cy.get('.todo-list li’).should(‘not.exist') cy.get(‘.main’).should('not.exist') cy.get('.footer').should('not.exist') 1 2 3 4 5 6
29. Debuggability @amirrustam
30. @amirrustam
31. Cypress Architecture Cypress Test Runner Your App @amirrustam ⚙ Cypress Node Backend Browser
32. @amirrustam Browser Node Backend cy.exec(…) Execute system commands outside of the scope of Cypress. ✅ Seed a database ✅ Running build scripts ✅ Start and kill processes
33. @amirrustam Browser Node Backend cy.exec(…) cy.exec('rake db:seed').its('code').should('eq', 0) cy.exec('npm run my-script’) .its(‘stdout') .should('contain', 'Done running the script')
34. @amirrustam Browser Node Backend cy.task(…) // in test describe('e2e', () => { beforeEach(() => { cy.task('defaults:db') cy.visit('/') }) it('displays article values', () => { cy.get('.article-list') .should('have.length', 10) }) })
35. @amirrustam Browser Node Backend cy.task(…) // in plugins/index.js file // we require some code in our app that // is responsible for seeding our database const db = require('../../server/src/db') module.exports = (on, config) => { on('task', { 'defaults:db': () => { return db.seed('defaults') } }) }
36. 4 cypress open Great for everyday development workflow Cypress + Editor Side-by-Side TDD Nirvana cypress run Testing in CI/CD Pipeline Headless More efficient for running all your tests @amirrustam
37. 4 cypress open Great for everyday development workflow Cypress + Editor Side-by-Side TDD Nirvana w/ Realtime Reloading cypress run --record Record to Cypress Dashboard @amirrustam
38. Dashboard @amirrustam
39. cypress run --record --parallel Parallelization Now available in Cypress 3.1 @amirrustam
40. cypress run --parallel cypress run --parallel cypress run --parallel Your CI Machines 3x parallelism Ready for work signup.spec.js login.spec.js widget.spec.js @amirrustam
41. Previous Spec Test Runs Spec Test Duration History Test Duration (seconds) The Future Automatic Load Balancing of Test Specs Duration Forecast @amirrustam
42. Automatic Load Balancing of Test Specs login.spec.js signup.spec.js widget.spec.js @amirrustam
43. @amirrustam cypress-example-kitchensink
44. @amirrustam cypress-dashboard 83%
45. @amirrustam Parallelization Tips Do not write all tests in one file. Break them up so they can be load balanced on CI machines. Do not write tests that are dependent on each other. Tests should pass in isolation.
46. Cypress Parallelization Easily optimize your CI usage You save a lot of ⏳ time and money @amirrustam
47. cypress run --group frontend Test Run Grouping Now available in Cypress 3.1
48. @amirrustam cypress-example-kitchensink
49. @amirrustam “OK Amir, but what about Selenium”
50. Renaissance of the Web 2005 2006 2014 2013 @amirrustam
51. Renaissance of the Web 2005 2006 2014 2013 More complexity moved from backend to frontend. @amirrustam
52. Stateless Stateful @amirrustam
53. 2005 2006 2014 2013 Renaissance of the Web + Testing Automation 2004 @amirrustam
54. Selenium Architecture Selenium Server Drivers Tests Python Javascript Java @amirrustam
55. Selenium Architecture Selenium Server Drivers Tests Python Javascript Java ⚠ Classical Syncing Dilemma @amirrustam
56. Selenium Architecture Flakiness Non-deterministic @amirrustam
57. ✋ JSDOM or DOM Emulation JSDOM is not a real browser. Your users don’t run your app with JSDOM. One of the primary reasons for E2E is to simulate real user actions. @amirrustam
58. Documentation docs.cypress.io
59. Best Practices docs.cypress.io @amirrustam
60. Roadmap docs.cypress.io @amirrustam
61. @amirrustam Mobile Desktop Cross Browser Support
62. @amirrustam Test Retries ✅ 999 Tests Pass ❌ 1 Test Fails Might just be a CI mishap
63. docs.cypress.io/examples/examples/tutorials.html Tutorial Videos
64. Testing Workshop docs.cypress.io/examples/examples/workshop.html
65. cypress run --group frontend Introducing Test Run Grouping Now available in Cypress 3.1
66. We are trying to drastically improve the status quo of testing on the web. @amirrustam
67. Thank You npm install -D cypress Free. Open Source. MIT License Follow @amirrustam for link to slides.