Scalable Testing Practices for Modern Apps

A brief glance at some testing practices that can be leveraged to give us confidence in the software we ship to the web.

Transcript

1. Nuxt Nation 2021 Scalable Testing Practices for Modern Apps Amir Rustamzadeh Director of Developer Experience @amirrustam
2. What types of tests should I write?
3. UI Layer Client Logic Layer Service Layer Application Layers @amirrustam Testable App Layers
4. UI Layer Client Logic Layer Service Layer Application Layers Mocked Service Layer End-to-End (E2E) Visual Regression Integration Component Types of tests Frontend Unit Tests Backend Unit Tests API Tests @amirrustam Testable App Layers
5. UI Layer Client Logic Layer Service Layer Application Layers Mocked Service Layer End-to-End (E2E) Visual Regression Integration Component Types of tests Frontend Unit Tests Backend Unit Tests API Tests @amirrustam Testable App Layers
6. Every testing type has its use case.
7. Component tests support component-driven development. E2E tests validate user journeys.
8. A quick glance at component testing with Cypress. It’s currently in beta
9. What is component testing? Virtual DOM Mount component to an emulated DOM Ability to test components in isolation
10. 01 Confidence With virtual or emulated DOMs, such as JSDOM Debugging Experience 02 No introspection into the visual nature of components. Development Experience 03 Can’t see the full extend of changes as you are iterating. Issues with Status Quo of Component Testing
11. Cypress has a new approach to component testing. It’s new and in the beta phase. But you can start using it today. Let’s take a look.
12. Cypress approach to Component Testing Browser DOM Mount component to browser DOM
13. How to run component tests ➜ ~ npx cypress open ➜ ~ npx cypress open-ct
14. Component Testing in Cypress
17. How to start a component test DatePicker.cy.spec.js Javascript import { mount } from '@cypress/vue' import DatePicker from ‘../DatePicker.vue' describe(‘Date Picker', () => { it('select a month from the picker', () => { mount(DatePicker ) // … } ) } ) login.spec.js Javascript describe(‘My App', () => { it(‘login to app', () => { cy.visit(“localhost:8080/login” ) // … } ) } ) End-to-End (E2E) Component Test (CT)
18. Check out the Component Testing Docs https://on.cypress.io/component-testing
19. Am I testing enough? Am I over or under testing? When am I done with testing?
20. One solution to this question is code coverage.
21. But remember that code coverage is a helping hand and not a definitive indicator of feature or user-journey coverage.
22. But remember that code coverage is a helping hand and not a definitive indicator of feature or user-journey coverage.
23. To get started with adding code coverage to your project….. Read our code coverage guide http://on.cypress.io/code-coverage Use the Cypress Code Coverage Plugin https://github.com/cypress-io/code-coverage
24. Responsive Testing Also, an example of why code-coverage is not a full proof indicator of coverage.
25. To get additional confidence in our test suites, our testing strategy should also test responsive nature of our apps.
26. todo-app.spec.js Javascript it('creates 2 items', () => { // Set viewport to 550px x 750p x cy.viewport(550, 750) //… or
 
 cy.viewport('iphone-6') } ) https://on.cypress.io/viewport
27. Visual Testing ….and here is another example of why code-coverage is not a full proof indicator of coverage.
28. Functional and logical tests give us a tremendous amount of confidence but we also need to assert the visual nature of our app to gain full confidence.
29. A comprehensive testing strategy should include visual tests.
30. To get started with visual regression testing…. Read our visual testing guide http://on.cypress.io/visual-testing
31. To get started with visual regression testing…. Read our visual testing guide http://on.cypress.io/visual-testing todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') // command cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion
 // take a snapshot once in a desired state
 cy.mySnapshotCommand( ) } )
32. Managing Flaky Tests
33. What is test flake? Time Test A
34. What is test flake? Time Test A Test A No code changed
35. What is test flake? Time Test A Test A No code changed Test A is considered to be flaky, since it yielded different results with the same inputs.
36. Flaky tests do not provide deterministic results and significantly reduce confidence in our tests.
37. Cypress provides a flake resistant API out of the box.
38. todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') // command cy.focused() // command .should('have.class', 'new-todo') // assertion cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion } ) Elements are automatically awaited for existence, visibility, and actionability. Assertion are retried
39. todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') // command cy.focused() // command .should('have.class', 'new-todo') // assertion cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion } ) All commands run in the exact order they are declared. The Cypress API is more declarative than imperative.
40. Test Retries is one way to manage flake in your testing pipeline.
41. Time Test A Test A No code changed Test A is still considered to be flaky, but its failure did not halt our CI pipeline. Attempt 1 Attempt 2 Attempt 3 Running tests with Test Retries enabled
42. Configuring Test Retries at the Project Level cypress.json JSON cypress.json JSON { "retries": { // Configure retry attempts for `cypress run ` // Default is 0 "runMode": 2 , // Configure retry attempts for `cypress open ` // Default is 0 "openMode": 0 } } { “retries”: 0 } Configuration for each mode Configuration for all modes
43. Configuring Test Retries at the Test Level login.spec.js login.spec.js Javascript describe('User sign-up and login', () => { it ( 'allows user to login' , { retries: { runMode: 2 , openMode: 1 , } , } , () => { // .. . } ) } ) describe('User bank accounts', { retries: { runMode: 2 , openMode: 1 , } }, () => { it('allows a user to view their transactions', () => { // ... } it('allows a user to edit their transactions', () => { // ... } } ) Configuration for a test Configuration for a suite
44. Test retries helps to keep your CI pipeline running smoothly, but also enables detection of flake.
45. Flake Detection within Cypress Dashboard
46. Flake Test Analytics within Cypress Dashboard
47. Thank you Cypress ❤ Nuxt