Slide 1

Slide 1 text

Nuxt Nation 2021 Scalable Testing Practices for Modern Apps Amir Rustamzadeh Director of Developer Experience @amirrustam

Slide 2

Slide 2 text

What types of tests should I write?

Slide 3

Slide 3 text

UI Layer Client Logic Layer Service Layer Application Layers @amirrustam Testable App Layers

Slide 4

Slide 4 text

UI Layer Client Logic Layer Service Layer Application Layers Mocked Service Layer End-to-End (E2E) Visual Regression Integration Component Types of tests Frontend Unit Tests Backend Unit Tests API Tests @amirrustam Testable App Layers

Slide 5

Slide 5 text

UI Layer Client Logic Layer Service Layer Application Layers Mocked Service Layer End-to-End (E2E) Visual Regression Integration Component Types of tests Frontend Unit Tests Backend Unit Tests API Tests @amirrustam Testable App Layers

Slide 6

Slide 6 text

Every testing type has its use case.

Slide 7

Slide 7 text

Component tests support component-driven development. E2E tests validate user journeys.

Slide 8

Slide 8 text

A quick glance at component testing with Cypress. It’s currently in beta

Slide 9

Slide 9 text

What is component testing? Virtual DOM Mount component to an emulated DOM Ability to test components in isolation

Slide 10

Slide 10 text

01 Confidence With virtual or emulated DOMs, such as JSDOM Debugging Experience 02 No introspection into the visual nature of components. Development Experience 03 Can’t see the full extend of changes as you are iterating. Issues with Status Quo of Component Testing

Slide 11

Slide 11 text

Cypress has a new approach to component testing. It’s new and in the beta phase. But you can start using it today. Let’s take a look.

Slide 12

Slide 12 text

Cypress approach to Component Testing Browser DOM Mount component to browser DOM

Slide 13

Slide 13 text

How to run component tests ➜ ~ npx cypress open ➜ ~ npx cypress open-ct

Slide 14

Slide 14 text

Component Testing in Cypress

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

How to start a component test DatePicker.cy.spec.js Javascript import { mount } from '@cypress/vue' import DatePicker from ‘../DatePicker.vue' describe(‘Date Picker', () => { it('select a month from the picker', () => { mount(DatePicker ) // … } ) } ) login.spec.js Javascript describe(‘My App', () => { it(‘login to app', () => { cy.visit(“localhost:8080/login” ) // … } ) } ) End-to-End (E2E) Component Test (CT)

Slide 18

Slide 18 text

Check out the Component Testing Docs https://on.cypress.io/component-testing

Slide 19

Slide 19 text

Am I testing enough? Am I over or under testing? When am I done with testing?

Slide 20

Slide 20 text

One solution to this question is code coverage.

Slide 21

Slide 21 text

But remember that code coverage is a helping hand and not a definitive indicator of feature or user-journey coverage.

Slide 22

Slide 22 text

But remember that code coverage is a helping hand and not a definitive indicator of feature or user-journey coverage.

Slide 23

Slide 23 text

To get started with adding code coverage to your project….. Read our code coverage guide http://on.cypress.io/code-coverage Use the Cypress Code Coverage Plugin https://github.com/cypress-io/code-coverage

Slide 24

Slide 24 text

Responsive Testing Also, an example of why code-coverage is not a full proof indicator of coverage.

Slide 25

Slide 25 text

To get additional confidence in our test suites, our testing strategy should also test responsive nature of our apps.

Slide 26

Slide 26 text

todo-app.spec.js Javascript it('creates 2 items', () => { // Set viewport to 550px x 750p x cy.viewport(550, 750) //… or
 
 cy.viewport('iphone-6') } ) https://on.cypress.io/viewport

Slide 27

Slide 27 text

Visual Testing ….and here is another example of why code-coverage is not a full proof indicator of coverage.

Slide 28

Slide 28 text

Functional and logical tests give us a tremendous amount of confidence but we also need to assert the visual nature of our app to gain full confidence.

Slide 29

Slide 29 text

A comprehensive testing strategy should include visual tests.

Slide 30

Slide 30 text

To get started with visual regression testing…. Read our visual testing guide http://on.cypress.io/visual-testing

Slide 31

Slide 31 text

To get started with visual regression testing…. Read our visual testing guide http://on.cypress.io/visual-testing todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') // command cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion
 // take a snapshot once in a desired state
 cy.mySnapshotCommand( ) } )

Slide 32

Slide 32 text

Managing Flaky Tests

Slide 33

Slide 33 text

What is test flake? Time Test A

Slide 34

Slide 34 text

What is test flake? Time Test A Test A No code changed

Slide 35

Slide 35 text

What is test flake? Time Test A Test A No code changed Test A is considered to be flaky, since it yielded different results with the same inputs.

Slide 36

Slide 36 text

Flaky tests do not provide deterministic results and significantly reduce confidence in our tests.

Slide 37

Slide 37 text

Cypress provides a flake resistant API out of the box.

Slide 38

Slide 38 text

todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') // command cy.focused() // command .should('have.class', 'new-todo') // assertion cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion } ) Elements are automatically awaited for existence, visibility, and actionability. Assertion are retried

Slide 39

Slide 39 text

todo-app.spec.js Javascript it('creates 2 items', () => { cy.visit('/') // command cy.focused() // command .should('have.class', 'new-todo') // assertion cy.get('.new-todo') // command .type('todo A{enter}') // command .type('todo B{enter}') // command cy.get('.todo-list li') // command .should('have.length', 2) // assertion } ) All commands run in the exact order they are declared. The Cypress API is more declarative than imperative.

Slide 40

Slide 40 text

Test Retries is one way to manage flake in your testing pipeline.

Slide 41

Slide 41 text

Time Test A Test A No code changed Test A is still considered to be flaky, but its failure did not halt our CI pipeline. Attempt 1 Attempt 2 Attempt 3 Running tests with Test Retries enabled

Slide 42

Slide 42 text

Configuring Test Retries at the Project Level cypress.json JSON cypress.json JSON { "retries": { // Configure retry attempts for `cypress run ` // Default is 0 "runMode": 2 , // Configure retry attempts for `cypress open ` // Default is 0 "openMode": 0 } } { “retries”: 0 } Configuration for each mode Configuration for all modes

Slide 43

Slide 43 text

Configuring Test Retries at the Test Level login.spec.js login.spec.js Javascript describe('User sign-up and login', () => { it ( 'allows user to login' , { retries: { runMode: 2 , openMode: 1 , } , } , () => { // .. . } ) } ) describe('User bank accounts', { retries: { runMode: 2 , openMode: 1 , } }, () => { it('allows a user to view their transactions', () => { // ... } it('allows a user to edit their transactions', () => { // ... } } ) Configuration for a test Configuration for a suite

Slide 44

Slide 44 text

Test retries helps to keep your CI pipeline running smoothly, but also enables detection of flake.

Slide 45

Slide 45 text

Flake Detection within Cypress Dashboard

Slide 46

Slide 46 text

Flake Test Analytics within Cypress Dashboard

Slide 47

Slide 47 text

Thank you Cypress ❤ Nuxt