Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scalable Testing Practices for Modern Apps

Scalable Testing Practices for Modern Apps

A brief glance at some testing practices that can be leveraged to give us confidence in the software we ship to the web.

C78c829ff61f9cfd6fe0b7ac45437672?s=128

Amir Rustamzadeh

September 15, 2021
Tweet

Transcript

  1. Nuxt Nation 2021 Scalable Testing Practices for Modern Apps Amir

    Rustamzadeh Director of Developer Experience @amirrustam
  2. What types of tests should I write?

  3. UI Layer Client Logic Layer Service Layer Application Layers @amirrustam

    Testable App Layers
  4. UI Layer Client Logic Layer Service Layer Application Layers Mocked

    Service Layer End-to-End (E2E) Visual Regression Integration Component Types of tests Frontend Unit Tests Backend Unit Tests API Tests @amirrustam Testable App Layers
  5. UI Layer Client Logic Layer Service Layer Application Layers Mocked

    Service Layer End-to-End (E2E) Visual Regression Integration Component Types of tests Frontend Unit Tests Backend Unit Tests API Tests @amirrustam Testable App Layers
  6. Every testing type has its use case.

  7. Component tests support component-driven development. E2E tests validate user journeys.

  8. A quick glance at component testing with Cypress. It’s currently

    in beta
  9. What is component testing? Virtual DOM Mount component to an

    emulated DOM Ability to test components in isolation
  10. 01 Confidence With virtual or emulated DOMs, such as JSDOM

    Debugging Experience 02 No introspection into the visual nature of components. Development Experience 03 Can’t see the full extend of changes as you are iterating. Issues with Status Quo of Component Testing
  11. Cypress has a new approach to component testing. It’s new

    and in the beta phase. But you can start using it today. Let’s take a look.
  12. Cypress approach to Component Testing Browser DOM Mount component to

    browser DOM
  13. How to run component tests ➜ ~ npx cypress open

    ➜ ~ npx cypress open-ct
  14. Component Testing in Cypress

  15. None
  16. None
  17. How to start a component test DatePicker.cy.spec.js Javascript import {

    mount } from '@cypress/vue' import DatePicker from ‘../DatePicker.vue' describe(‘Date Picker', () => { it('select a month from the picker', () => { mount(DatePicker ) // … } ) } ) login.spec.js Javascript describe(‘My App', () => { it(‘login to app', () => { cy.visit(“localhost:8080/login” ) // … } ) } ) End-to-End (E2E) Component Test (CT)
  18. Check out the Component Testing Docs https://on.cypress.io/component-testing

  19. Am I testing enough? Am I over or under testing?

    When am I done with testing?
  20. One solution to this question is code coverage.

  21. But remember that code coverage is a helping hand and

    not a definitive indicator of feature or user-journey coverage.
  22. But remember that code coverage is a helping hand and

    not a definitive indicator of feature or user-journey coverage.
  23. To get started with adding code coverage to your project…..

    Read our code coverage guide http://on.cypress.io/code-coverage Use the Cypress Code Coverage Plugin https://github.com/cypress-io/code-coverage
  24. Responsive Testing Also, an example of why code-coverage is not

    a full proof indicator of coverage.
  25. To get additional confidence in our test suites, our testing

    strategy should also test responsive nature of our apps.
  26. todo-app.spec.js Javascript it('creates 2 items', () => { // Set

    viewport to 550px x 750p x cy.viewport(550, 750) //… or
 
 cy.viewport('iphone-6') } ) https://on.cypress.io/viewport
  27. Visual Testing ….and here is another example of why code-coverage

    is not a full proof indicator of coverage.
  28. Functional and logical tests give us a tremendous amount of

    confidence but we also need to assert the visual nature of our app to gain full confidence.
  29. A comprehensive testing strategy should include visual tests.

  30. To get started with visual regression testing…. Read our visual

    testing guide http://on.cypress.io/visual-testing
  31. To get started with visual regression testing…. Read our visual

    testing guide http://on.cypress.io/visual-testing todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') // command cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion
 // take a snapshot once in a desired state
 cy.mySnapshotCommand( ) } )
  32. Managing Flaky Tests

  33. What is test flake? Time Test A

  34. What is test flake? Time Test A Test A No

    code changed
  35. What is test flake? Time Test A Test A No

    code changed Test A is considered to be flaky, since it yielded different results with the same inputs.
  36. Flaky tests do not provide deterministic results and significantly reduce

    confidence in our tests.
  37. Cypress provides a flake resistant API out of the box.

  38. todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') //

    command cy.focused() // command .should('have.class', 'new-todo') // assertion cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion } ) Elements are automatically awaited for existence, visibility, and actionability. Assertion are retried
  39. todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') //

    command cy.focused() // command .should('have.class', 'new-todo') // assertion cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion } ) All commands run in the exact order they are declared. The Cypress API is more declarative than imperative.
  40. Test Retries is one way to manage flake in your

    testing pipeline.
  41. Time Test A Test A No code changed Test A

    is still considered to be flaky, but its failure did not halt our CI pipeline. Attempt 1 Attempt 2 Attempt 3 Running tests with Test Retries enabled
  42. Configuring Test Retries at the Project Level cypress.json JSON cypress.json

    JSON { "retries": { // Configure retry attempts for `cypress run ` // Default is 0 "runMode": 2 , // Configure retry attempts for `cypress open ` // Default is 0 "openMode": 0 } } { “retries”: 0 } Configuration for each mode Configuration for all modes
  43. Configuring Test Retries at the Test Level login.spec.js login.spec.js Javascript

    describe('User sign-up and login', () => { it ( 'allows user to login' , { retries: { runMode: 2 , openMode: 1 , } , } , () => { // .. . } ) } ) describe('User bank accounts', { retries: { runMode: 2 , openMode: 1 , } }, () => { it('allows a user to view their transactions', () => { // ... } it('allows a user to edit their transactions', () => { // ... } } ) Configuration for a test Configuration for a suite
  44. Test retries helps to keep your CI pipeline running smoothly,

    but also enables detection of flake.
  45. Flake Detection within Cypress Dashboard

  46. Flake Test Analytics within Cypress Dashboard

  47. Thank you Cypress ❤ Nuxt