Slide 1

Slide 1 text

DESIGN DECISIONS DESIGN DECISIONS FOR PERFECT JS E2E TESTING FRAMEWORK FOR PERFECT JS E2E TESTING FRAMEWORK by Michael Bodnarchuk 2019

Slide 2

Slide 2 text

ABOUT ME ABOUT ME Michael Bodnarchuk @davert Web developer from Kyiv, Ukraine Lead developer of CodeceptJS Also author of Codeception, Robo and others Tech Consultant, CTO at SDCLabs

Slide 3

Slide 3 text

CONFERENCE SPEAKING CONFERENCE SPEAKING

Slide 4

Slide 4 text

MY VISION MY VISION Tests should be simple to write and understand Tests have their priority. Don't write tests for everything Tests should follow business values Testing should be joyful

Slide 5

Slide 5 text

SO YOU WANT SOME TEST SO YOU WANT SOME TEST AUTOMATION? AUTOMATION?

Slide 6

Slide 6 text

QUESTIONS TO BE ASKED QUESTIONS TO BE ASKED what language to choose how to execute tests how to write tests how to control browser

Slide 7

Slide 7 text

WHAT LANGUAGE TO CHOOSE? WHAT LANGUAGE TO CHOOSE? Java Ruby Python JavaScript

Slide 8

Slide 8 text

JAVASCRIPT! JAVASCRIPT! Rich ecosystem Not limited to Selenium In browser execution [WARNING] Deal with promises

Slide 9

Slide 9 text

LET'S MAKE A SNOWMAN LET'S MAKE A SNOWMAN 1. Testing Framework 2. Assertion library 3. Browser Driver 4. Runner

Slide 10

Slide 10 text

TESTING FRAMEWORK TESTING FRAMEWORK How tests are written mocha jasmine cucumber jest ava

Slide 11

Slide 11 text

Testing frameworks are pretty much the same for e2e

Slide 12

Slide 12 text

ASSERTION LIBRARIES ASSERTION LIBRARIES How to make assertions chai framework-relevant native assert

Slide 13

Slide 13 text

In NodeJS assertion libraries are decoupled from a testing framework.

Slide 14

Slide 14 text

BROWSER DRIVER BROWSER DRIVER Protractor webdriverio Cypress.io Puppeteer TestCafe NightmareJS

Slide 15

Slide 15 text

RUNNER RUNNER CLI tool to execute tests mocha protractor wdio cypress testcafe

Slide 16

Slide 16 text

IN DETAILS IN DETAILS

Slide 17

Slide 17 text

PROTRACTOR PROTRACTOR Selenium (built on top of o cial library) Good Documentation Jasmine Testing Framework Runner Ooooooutdated Protractor 6 will break everything

Slide 18

Slide 18 text

EXAMPLE EXAMPLE beforeEach(function() { browser.get('http://www.angularjs.org'); todoList = element.all(by.repeater('todo in todoList.todos')); }); it('should add a todo', function() { var addTodo = element(by.model('todoList.todoText')); var addButton = element(by.css('[value="add"]')); addTodo.sendKeys('write a protractor test'); addButton.click(); expect(todoList.count()).toEqual(3); expect(todoList.get(2).getText()).toEqual('write a protractor test'); });

Slide 19

Slide 19 text

WEBDRIVERIO WEBDRIVERIO Alternative Selenium implementation Mobile testing (Native Apps) with Appium Awesome documentation v4 to v5 upgrade... W3C spec + JSONWire spec Standalone / Jasmine / Mocha / Cucumber integration

Slide 20

Slide 20 text

EXAMPLE EXAMPLE // page object class FormPage extends Page { get username () { return $('#username') } get password () { return $('#password') } } // test browser.url('/form'); FormPage.username.setValue('foo') FormPage.password.setValue('bar') FormPage.submit()

Slide 21

Slide 21 text

CYPRESS.IO CYPRESS.IO Chrome-based, runs inside a browser Mocha testing framework + chai assertions UI Debugger Good documentation Auto retry failed steps No XPath No le uploads No multiple browsers, multiple tabs No iframes LIMITATIONS!!!

Slide 22

Slide 22 text

EXAMPLE EXAMPLE it('adds 2 todos', function () { cy.visit('/'); cy.get('.new-todo') .type('learn testing{enter}') .type('be cool{enter}') cy.get('.todo-list li').should('have.length', 2) })

Slide 23

Slide 23 text

HOW MUCH DOES HOW MUCH DOES YOUR TEST YOUR TEST AUTOMATION AUTOMATION FRAMEWORK COST? FRAMEWORK COST?

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

RISE OF CYPRESS.IO RISE OF CYPRESS.IO

Slide 26

Slide 26 text

Cypress.io is over-estimated as end 2 end testing framework

Slide 27

Slide 27 text

PUPPETEER PUPPETEER O cial Google Chrome DevTools library Standalone library (no testing framework) Good API Documentation Provides full browser control

Slide 28

Slide 28 text

EXAMPLE EXAMPLE beforeEach(async () => { const page = await browser.newPage() await page.setViewport({ width: 1280, height: 800 }) await page.goto('https://www.walmart.com/ip/Super-Mario-Odyssey-Nint await page.click('button.prod-ProductCTA--primary') await page.waitForSelector('.Cart-PACModal-ItemInfoContainer') await page.screenshot({path: screenshot}) await browser.close() });

Slide 29

Slide 29 text

RISE OF PUPPETEER RISE OF PUPPETEER

Slide 30

Slide 30 text

TESTCAFE TESTCAFE Cross-browser client-side testing Proxy server for mocking all requests Doesn't control browser Custom test framework, assertions, runner Parallel execution built-in Multi-browser setup

Slide 31

Slide 31 text

EXAMPLE EXAMPLE test('Dealing with text using keyboard', async t => { await t .typeText(page.nameInput, 'Peter Parker') // Type name .click(page.nameInput, { caretPos: 5 }) // Move caret position .pressKey('backspace') // Erase a character .expect(page.nameInput.value).eql('Pete Parker') // Check result .pressKey('home right . delete delete delete') // Pick even shorter .expect(page.nameInput.value).eql('P. Parker'); // Check result });

Slide 32

Slide 32 text

HOW TO CHOOSE DRIVER HOW TO CHOOSE DRIVER Learn how the tool works Consider its limitations Check documentation Upgrade strategy Look into source code

Slide 33

Slide 33 text

ASYNCHRONITY ASYNCHRONITY In JavaScript all browser commands are promises Driver Strategy Protractor 5 control ow Protractor 6 async/await webdriverio sync bers, async/await cypress control ow Puppeteer async/await TestCafe async/await

Slide 34

Slide 34 text

HOW TO CHOOSE TESTING FRAMEWORK HOW TO CHOOSE TESTING FRAMEWORK Use the one provided by driver Except...

Slide 35

Slide 35 text

...CUCUMBER ...CUCUMBER Has its own runner & testing framework To hide JS complexity To work as BDD tool Supported by standalone libraries Protractor WebdriverIO Puppeteer

Slide 36

Slide 36 text

EXAMPLE EXAMPLE Feature: Visit the app dashboard As a citizen I should be able to log in to the app with DigiD In order to access my personal information Scenario: Log in with DigiD Given I am logged in with DigiD as 123456789 And there are the following toggles: personal When I visit the dashboard Then I should be greeted with H.A. Janssen

Slide 37

Slide 37 text

CONCLUSIONS CONCLUSIONS Get your requirements Learn ecosystem Choose the testing stack

Slide 38

Slide 38 text

LET'S BUILD PERFECT TESTING FRAMEWORK! LET'S BUILD PERFECT TESTING FRAMEWORK!

Slide 39

Slide 39 text

WE DID IT! WE DID IT! ...but not that snowman!

Slide 40

Slide 40 text

WHATEVER YOU CHOOSE, YOU LOSE! WHATEVER YOU CHOOSE, YOU LOSE! New testing frameworks emerge Cool fancy library will be legacy tomorow You hit issues with edge cases Di erent API You can't migrate your code

Slide 41

Slide 41 text

SURVIVE THE CHANGE SURVIVE THE CHANGE Write high level test code Separate scenario from browser control Use Cucumber or CodeceptJS

Slide 42

Slide 42 text

CODECEPTJS CODECEPTJS Multi-driver testing framework webdriverio Puppeteer Protractor Custom runner, mocha-based test framework High level API (with Cucumber support) Interactive debug mode Auto retry failed steps

Slide 43

Slide 43 text

ARCHITECTURE ARCHITECTURE WebDriverIO Protractor Nightmare Puppeteer Electron WebDriver API CODECEPTJS Selenium Server Firefox Browser Chrome Browser Edge Browser DevTools Protocol Cloud Browsers HELPERS

Slide 44

Slide 44 text

EXAMPLE EXAMPLE Scenario('todomvc', (I, loginPage) => { const user = await I.createUser('davert'); loginPage.login(davert); I.see('davert', 'nav'); I.click('Create Todo'); I.see('1 item left', '.todo-count'); I.fillField('What needs to be done?', 'Write a test'); I.pressKey('Enter'); I.see('Write a test', '.todo-list'); I.see('2 items left', '.todo-count'); I.fillField('What needs to be done?', 'Write a code'); I.pressKey('Enter'); I.see('Write a code', '.todo-list'); I.see('3 items left', '.todo-count'); });

Slide 45

Slide 45 text

LIVE DEVELOPMENT LIVE DEVELOPMENT I.amOnPage('/'); pause(); Call pause() to interrupt the test Use interactive shell to try di erent commands Copy successful commands into a test

Slide 46

Slide 46 text

CONCLUSIONS CONCLUSIONS For advanced e2e testing use webdriverio For full browser control use puppeteer For high-level automated e2e tests use codeceptjs For component testing use cypress.io For simple multi-browser testing use testcafe For BDD use CucumberJS

Slide 47

Slide 47 text

THANK YOU! THANK YOU! Michael Bodnarchuk Web developer from Kyiv, Ukraine Open-source developer, author of Codeception, CodeceptJS testing frameworks Consultant @ SDCLabs