DESIGN DECISIONS FOR PERFECT JS E2E TESTING FRAMEWORK

Starting test automation in JS in 2019 is like...

Transcript

1. DESIGN DECISIONS DESIGN DECISIONS FOR PERFECT JS E2E TESTING FRAMEWORK FOR PERFECT JS E2E TESTING FRAMEWORK by Michael Bodnarchuk 2019
2. ABOUT ME ABOUT ME Michael Bodnarchuk @davert Web developer from Kyiv, Ukraine Lead developer of CodeceptJS Also author of Codeception, Robo and others Tech Consultant, CTO at SDCLabs
3. CONFERENCE SPEAKING CONFERENCE SPEAKING
4. MY VISION MY VISION Tests should be simple to write and understand Tests have their priority. Don't write tests for everything Tests should follow business values Testing should be joyful
5. SO YOU WANT SOME TEST SO YOU WANT SOME TEST AUTOMATION? AUTOMATION?
6. QUESTIONS TO BE ASKED QUESTIONS TO BE ASKED what language to choose how to execute tests how to write tests how to control browser
7. WHAT LANGUAGE TO CHOOSE? WHAT LANGUAGE TO CHOOSE? Java Ruby Python JavaScript
8. JAVASCRIPT! JAVASCRIPT! Rich ecosystem Not limited to Selenium In browser execution [WARNING] Deal with promises
9. LET'S MAKE A SNOWMAN LET'S MAKE A SNOWMAN 1. Testing Framework 2. Assertion library 3. Browser Driver 4. Runner
10. TESTING FRAMEWORK TESTING FRAMEWORK How tests are written mocha jasmine cucumber jest ava
11. Testing frameworks are pretty much the same for e2e
12. ASSERTION LIBRARIES ASSERTION LIBRARIES How to make assertions chai framework-relevant native assert
13. In NodeJS assertion libraries are decoupled from a testing framework.
14. BROWSER DRIVER BROWSER DRIVER Protractor webdriverio Cypress.io Puppeteer TestCafe NightmareJS
15. RUNNER RUNNER CLI tool to execute tests mocha protractor wdio cypress testcafe
16. IN DETAILS IN DETAILS
17. PROTRACTOR PROTRACTOR Selenium (built on top of o cial library) Good Documentation Jasmine Testing Framework Runner Ooooooutdated Protractor 6 will break everything
18. EXAMPLE EXAMPLE beforeEach(function() { browser.get('http://www.angularjs.org'); todoList = element.all(by.repeater('todo in todoList.todos')); }); it('should add a todo', function() { var addTodo = element(by.model('todoList.todoText')); var addButton = element(by.css('[value="add"]')); addTodo.sendKeys('write a protractor test'); addButton.click(); expect(todoList.count()).toEqual(3); expect(todoList.get(2).getText()).toEqual('write a protractor test'); });
19. WEBDRIVERIO WEBDRIVERIO Alternative Selenium implementation Mobile testing (Native Apps) with Appium Awesome documentation v4 to v5 upgrade... W3C spec + JSONWire spec Standalone / Jasmine / Mocha / Cucumber integration
20. EXAMPLE EXAMPLE // page object class FormPage extends Page { get username () { return $('#username') } get password () { return $('#password') } } // test browser.url('/form'); FormPage.username.setValue('foo') FormPage.password.setValue('bar') FormPage.submit()
21. CYPRESS.IO CYPRESS.IO Chrome-based, runs inside a browser Mocha testing framework + chai assertions UI Debugger Good documentation Auto retry failed steps No XPath No le uploads No multiple browsers, multiple tabs No iframes LIMITATIONS!!!
22. EXAMPLE EXAMPLE it('adds 2 todos', function () { cy.visit('/'); cy.get('.new-todo') .type('learn testing{enter}') .type('be cool{enter}') cy.get('.todo-list li').should('have.length', 2) })
23. HOW MUCH DOES HOW MUCH DOES YOUR TEST YOUR TEST AUTOMATION AUTOMATION FRAMEWORK COST? FRAMEWORK COST?
25. RISE OF CYPRESS.IO RISE OF CYPRESS.IO
26. Cypress.io is over-estimated as end 2 end testing framework
27. PUPPETEER PUPPETEER O cial Google Chrome DevTools library Standalone library (no testing framework) Good API Documentation Provides full browser control
28. EXAMPLE EXAMPLE beforeEach(async () => { const page = await browser.newPage() await page.setViewport({ width: 1280, height: 800 }) await page.goto('https://www.walmart.com/ip/Super-Mario-Odyssey-Nint await page.click('button.prod-ProductCTA--primary') await page.waitForSelector('.Cart-PACModal-ItemInfoContainer') await page.screenshot({path: screenshot}) await browser.close() });
29. RISE OF PUPPETEER RISE OF PUPPETEER
30. TESTCAFE TESTCAFE Cross-browser client-side testing Proxy server for mocking all requests Doesn't control browser Custom test framework, assertions, runner Parallel execution built-in Multi-browser setup
31. EXAMPLE EXAMPLE test('Dealing with text using keyboard', async t => { await t .typeText(page.nameInput, 'Peter Parker') // Type name .click(page.nameInput, { caretPos: 5 }) // Move caret position .pressKey('backspace') // Erase a character .expect(page.nameInput.value).eql('Pete Parker') // Check result .pressKey('home right . delete delete delete') // Pick even shorter .expect(page.nameInput.value).eql('P. Parker'); // Check result });
32. HOW TO CHOOSE DRIVER HOW TO CHOOSE DRIVER Learn how the tool works Consider its limitations Check documentation Upgrade strategy Look into source code
33. ASYNCHRONITY ASYNCHRONITY In JavaScript all browser commands are promises Driver Strategy Protractor 5 control ow Protractor 6 async/await webdriverio sync bers, async/await cypress control ow Puppeteer async/await TestCafe async/await
34. HOW TO CHOOSE TESTING FRAMEWORK HOW TO CHOOSE TESTING FRAMEWORK Use the one provided by driver Except...
35. ...CUCUMBER ...CUCUMBER Has its own runner & testing framework To hide JS complexity To work as BDD tool Supported by standalone libraries Protractor WebdriverIO Puppeteer
36. EXAMPLE EXAMPLE Feature: Visit the app dashboard As a citizen I should be able to log in to the app with DigiD In order to access my personal information Scenario: Log in with DigiD Given I am logged in with DigiD as 123456789 And there are the following toggles: personal When I visit the dashboard Then I should be greeted with H.A. Janssen
37. CONCLUSIONS CONCLUSIONS Get your requirements Learn ecosystem Choose the testing stack
38. LET'S BUILD PERFECT TESTING FRAMEWORK! LET'S BUILD PERFECT TESTING FRAMEWORK!
39. WE DID IT! WE DID IT! ...but not that snowman!
40. WHATEVER YOU CHOOSE, YOU LOSE! WHATEVER YOU CHOOSE, YOU LOSE! New testing frameworks emerge Cool fancy library will be legacy tomorow You hit issues with edge cases Di erent API You can't migrate your code
41. SURVIVE THE CHANGE SURVIVE THE CHANGE Write high level test code Separate scenario from browser control Use Cucumber or CodeceptJS
42. CODECEPTJS CODECEPTJS Multi-driver testing framework webdriverio Puppeteer Protractor Custom runner, mocha-based test framework High level API (with Cucumber support) Interactive debug mode Auto retry failed steps
43. ARCHITECTURE ARCHITECTURE WebDriverIO Protractor Nightmare Puppeteer Electron WebDriver API CODECEPTJS Selenium Server Firefox Browser Chrome Browser Edge Browser DevTools Protocol Cloud Browsers HELPERS
44. EXAMPLE EXAMPLE Scenario('todomvc', (I, loginPage) => { const user = await I.createUser('davert'); loginPage.login(davert); I.see('davert', 'nav'); I.click('Create Todo'); I.see('1 item left', '.todo-count'); I.fillField('What needs to be done?', 'Write a test'); I.pressKey('Enter'); I.see('Write a test', '.todo-list'); I.see('2 items left', '.todo-count'); I.fillField('What needs to be done?', 'Write a code'); I.pressKey('Enter'); I.see('Write a code', '.todo-list'); I.see('3 items left', '.todo-count'); });
45. LIVE DEVELOPMENT LIVE DEVELOPMENT I.amOnPage('/'); pause(); Call pause() to interrupt the test Use interactive shell to try di erent commands Copy successful commands into a test
46. CONCLUSIONS CONCLUSIONS For advanced e2e testing use webdriverio For full browser control use puppeteer For high-level automated e2e tests use codeceptjs For component testing use cypress.io For simple multi-browser testing use testcafe For BDD use CucumberJS
47. THANK YOU! THANK YOU! Michael Bodnarchuk Web developer from Kyiv, Ukraine Open-source developer, author of Codeception, CodeceptJS testing frameworks Consultant @ SDCLabs