Save 37% off PRO during our Black Friday Sale! »

DESIGN DECISIONS FOR PERFECT JS E2E TESTING FRAMEWORK

DESIGN DECISIONS FOR PERFECT JS E2E TESTING FRAMEWORK

Starting test automation in JS in 2019 is like...

Aa0518da9d7119444cb02a8f27017d8a?s=128

Michael Bodnarchuk

February 09, 2019
Tweet

Transcript

  1. DESIGN DECISIONS DESIGN DECISIONS FOR PERFECT JS E2E TESTING FRAMEWORK

    FOR PERFECT JS E2E TESTING FRAMEWORK by Michael Bodnarchuk 2019
  2. ABOUT ME ABOUT ME Michael Bodnarchuk @davert Web developer from

    Kyiv, Ukraine Lead developer of CodeceptJS Also author of Codeception, Robo and others Tech Consultant, CTO at SDCLabs
  3. CONFERENCE SPEAKING CONFERENCE SPEAKING

  4. MY VISION MY VISION Tests should be simple to write

    and understand Tests have their priority. Don't write tests for everything Tests should follow business values Testing should be joyful
  5. SO YOU WANT SOME TEST SO YOU WANT SOME TEST

    AUTOMATION? AUTOMATION?
  6. QUESTIONS TO BE ASKED QUESTIONS TO BE ASKED what language

    to choose how to execute tests how to write tests how to control browser
  7. WHAT LANGUAGE TO CHOOSE? WHAT LANGUAGE TO CHOOSE? Java Ruby

    Python JavaScript
  8. JAVASCRIPT! JAVASCRIPT! Rich ecosystem Not limited to Selenium In browser

    execution [WARNING] Deal with promises
  9. LET'S MAKE A SNOWMAN LET'S MAKE A SNOWMAN 1. Testing

    Framework 2. Assertion library 3. Browser Driver 4. Runner
  10. TESTING FRAMEWORK TESTING FRAMEWORK How tests are written mocha jasmine

    cucumber jest ava
  11. Testing frameworks are pretty much the same for e2e

  12. ASSERTION LIBRARIES ASSERTION LIBRARIES How to make assertions chai framework-relevant

    native assert
  13. In NodeJS assertion libraries are decoupled from a testing framework.

  14. BROWSER DRIVER BROWSER DRIVER Protractor webdriverio Cypress.io Puppeteer TestCafe NightmareJS

  15. RUNNER RUNNER CLI tool to execute tests mocha protractor wdio

    cypress testcafe
  16. IN DETAILS IN DETAILS

  17. PROTRACTOR PROTRACTOR Selenium (built on top of o cial library)

    Good Documentation Jasmine Testing Framework Runner Ooooooutdated Protractor 6 will break everything
  18. EXAMPLE EXAMPLE beforeEach(function() { browser.get('http://www.angularjs.org'); todoList = element.all(by.repeater('todo in todoList.todos'));

    }); it('should add a todo', function() { var addTodo = element(by.model('todoList.todoText')); var addButton = element(by.css('[value="add"]')); addTodo.sendKeys('write a protractor test'); addButton.click(); expect(todoList.count()).toEqual(3); expect(todoList.get(2).getText()).toEqual('write a protractor test'); });
  19. WEBDRIVERIO WEBDRIVERIO Alternative Selenium implementation Mobile testing (Native Apps) with

    Appium Awesome documentation v4 to v5 upgrade... W3C spec + JSONWire spec Standalone / Jasmine / Mocha / Cucumber integration
  20. EXAMPLE EXAMPLE // page object class FormPage extends Page {

    get username () { return $('#username') } get password () { return $('#password') } } // test browser.url('/form'); FormPage.username.setValue('foo') FormPage.password.setValue('bar') FormPage.submit()
  21. CYPRESS.IO CYPRESS.IO Chrome-based, runs inside a browser Mocha testing framework

    + chai assertions UI Debugger Good documentation Auto retry failed steps No XPath No le uploads No multiple browsers, multiple tabs No iframes LIMITATIONS!!!
  22. EXAMPLE EXAMPLE it('adds 2 todos', function () { cy.visit('/'); cy.get('.new-todo')

    .type('learn testing{enter}') .type('be cool{enter}') cy.get('.todo-list li').should('have.length', 2) })
  23. HOW MUCH DOES HOW MUCH DOES YOUR TEST YOUR TEST

    AUTOMATION AUTOMATION FRAMEWORK COST? FRAMEWORK COST?
  24. None
  25. RISE OF CYPRESS.IO RISE OF CYPRESS.IO

  26. Cypress.io is over-estimated as end 2 end testing framework

  27. PUPPETEER PUPPETEER O cial Google Chrome DevTools library Standalone library

    (no testing framework) Good API Documentation Provides full browser control
  28. EXAMPLE EXAMPLE beforeEach(async () => { const page = await

    browser.newPage() await page.setViewport({ width: 1280, height: 800 }) await page.goto('https://www.walmart.com/ip/Super-Mario-Odyssey-Nint await page.click('button.prod-ProductCTA--primary') await page.waitForSelector('.Cart-PACModal-ItemInfoContainer') await page.screenshot({path: screenshot}) await browser.close() });
  29. RISE OF PUPPETEER RISE OF PUPPETEER

  30. TESTCAFE TESTCAFE Cross-browser client-side testing Proxy server for mocking all

    requests Doesn't control browser Custom test framework, assertions, runner Parallel execution built-in Multi-browser setup
  31. EXAMPLE EXAMPLE test('Dealing with text using keyboard', async t =>

    { await t .typeText(page.nameInput, 'Peter Parker') // Type name .click(page.nameInput, { caretPos: 5 }) // Move caret position .pressKey('backspace') // Erase a character .expect(page.nameInput.value).eql('Pete Parker') // Check result .pressKey('home right . delete delete delete') // Pick even shorter .expect(page.nameInput.value).eql('P. Parker'); // Check result });
  32. HOW TO CHOOSE DRIVER HOW TO CHOOSE DRIVER Learn how

    the tool works Consider its limitations Check documentation Upgrade strategy Look into source code
  33. ASYNCHRONITY ASYNCHRONITY In JavaScript all browser commands are promises Driver

    Strategy Protractor 5 control ow Protractor 6 async/await webdriverio sync bers, async/await cypress control ow Puppeteer async/await TestCafe async/await
  34. HOW TO CHOOSE TESTING FRAMEWORK HOW TO CHOOSE TESTING FRAMEWORK

    Use the one provided by driver Except...
  35. ...CUCUMBER ...CUCUMBER Has its own runner & testing framework To

    hide JS complexity To work as BDD tool Supported by standalone libraries Protractor WebdriverIO Puppeteer
  36. EXAMPLE EXAMPLE Feature: Visit the app dashboard As a citizen

    I should be able to log in to the app with DigiD In order to access my personal information Scenario: Log in with DigiD Given I am logged in with DigiD as 123456789 And there are the following toggles: personal When I visit the dashboard Then I should be greeted with H.A. Janssen
  37. CONCLUSIONS CONCLUSIONS Get your requirements Learn ecosystem Choose the testing

    stack
  38. LET'S BUILD PERFECT TESTING FRAMEWORK! LET'S BUILD PERFECT TESTING FRAMEWORK!

  39. WE DID IT! WE DID IT! ...but not that snowman!

  40. WHATEVER YOU CHOOSE, YOU LOSE! WHATEVER YOU CHOOSE, YOU LOSE!

    New testing frameworks emerge Cool fancy library will be legacy tomorow You hit issues with edge cases Di erent API You can't migrate your code
  41. SURVIVE THE CHANGE SURVIVE THE CHANGE Write high level test

    code Separate scenario from browser control Use Cucumber or CodeceptJS
  42. CODECEPTJS CODECEPTJS Multi-driver testing framework webdriverio Puppeteer Protractor Custom runner,

    mocha-based test framework High level API (with Cucumber support) Interactive debug mode Auto retry failed steps
  43. ARCHITECTURE ARCHITECTURE WebDriverIO Protractor Nightmare Puppeteer Electron WebDriver API CODECEPTJS

    Selenium Server Firefox Browser Chrome Browser Edge Browser DevTools Protocol Cloud Browsers HELPERS
  44. EXAMPLE EXAMPLE Scenario('todomvc', (I, loginPage) => { const user =

    await I.createUser('davert'); loginPage.login(davert); I.see('davert', 'nav'); I.click('Create Todo'); I.see('1 item left', '.todo-count'); I.fillField('What needs to be done?', 'Write a test'); I.pressKey('Enter'); I.see('Write a test', '.todo-list'); I.see('2 items left', '.todo-count'); I.fillField('What needs to be done?', 'Write a code'); I.pressKey('Enter'); I.see('Write a code', '.todo-list'); I.see('3 items left', '.todo-count'); });
  45. LIVE DEVELOPMENT LIVE DEVELOPMENT I.amOnPage('/'); pause(); Call pause() to interrupt

    the test Use interactive shell to try di erent commands Copy successful commands into a test
  46. CONCLUSIONS CONCLUSIONS For advanced e2e testing use webdriverio For full

    browser control use puppeteer For high-level automated e2e tests use codeceptjs For component testing use cypress.io For simple multi-browser testing use testcafe For BDD use CucumberJS
  47. THANK YOU! THANK YOU! Michael Bodnarchuk Web developer from Kyiv,

    Ukraine Open-source developer, author of Codeception, CodeceptJS testing frameworks Consultant @ SDCLabs