Automation in Mobile Testing

Transcript

1. Messekongress ITfurVersicherungen - November 2018 Automation in Mobile Testing @N26 With Milan Kuveljic Head of Quality Engineering at N26
2. ⬒ 300+ engineers - Berlin/Barcelona/New York ⬒ Deploying to production 200+ times per week ⬒ Building crash-free mobile applications ⬒ Achieving regularly 5 stars in the App/Play Stores ⬒ Having Web app built for accessibility N26 is a tech company building modern banking @milan_kuveljic
3. Milan Kuveljic Head of Quality Engineering @N26 milan_kuveljic testingin.space
4. Our journey Automation principles Scope definition Architecture Tools and frameworks Continuous testing
5. Mobile Engineers build the app Test environment Test Engineers check the functionality
6. Mobile Engineers build the app Test environment Tests Passed
7. CHALLENGE
8. Testing becomes more time consuming and slows down the release process
9. Number of mobile devices and operating systems is increasing over time
11. AUTOMATION
12. INITIAL ASSUMPTIONS Automation as a silver bullet Automate all the things Automation is testing The best tool will solve all problems Automation is a tester’s job
13. Principles
14. Principles A test executed once it’s not a test anymore - it’s a check
15. Principles If you need to repeat a check, automate it
16. Principles Effective team collaboration Testing approach and scope Evaluate the right tools Continuous improvements
17. Test Engineers Mobile Engineers
18. Mobile Engineering
19. Scope UNIT TESTS UI INTEGRATION/COMPONENT TESTS E2E TESTS EXPLORATORY TESTING
20. Scope UI INTEGRATION/COMPONENT TESTS - Easy to implement - No external dependencies - API, 3rd party - Less prone to flakiness - Does not require application file - Mocking - Execution on real devices and simulators
21. Scope E2E TESTS - User journey - HTTP connections - Prone to flakiness - Requires application file - Execution on real devices and simulators
22. ARCHITECTURE
23. Architecture Domain Test Data Infrastructure Devices Reporting Integration
24. Architecture Domain Test structure Test suites
25. Architecture Test Data Configurable static test data Dynamic test data - API helpers IT’S ALSO AUTOMATION
26. Architecture Infrastructure Test Environments Mocks Limited Production Environment IT’S ALSO AUTOMATION
27. Architecture Devices In house mobile devices Devices on Cloud
28. Architecture Integration Manual build execution Continuous integration servers IT’S ALSO AUTOMATION
29. Architecture Reporting HTML reporting XML reporting Data Visualisation & Monitoring IT’S ALSO AUTOMATION
30. Capabilities of a specific tool Tool meets specific needs. Tool doesn’t seem to provide full test coverage Tool costs exceed the team budget Tools
31. Tools and frameworks https://dzone.com/articles/mobile-test-automation-framework-trends
32. Tools UI INTEGRATION/COMPONENT TESTS AUTOMATED E2E TESTS
33. SauceLabs devices Jenkins + Docker Chat channel Appium + Java/Kt + TestNG + PageObject Model E2E Framework Reporting dashboard
34. SauceLabs devices Jenkins + Docker Chat channel Espresso + XCUITest UI Framework Reporting dashboard
35. CONTINUOUS TESTING
36. Continuous testing Pull request
38. Continuous testing takeaways • Layered test architecture • Regression rate reduced • Mobile automation in the development scope • Extensive access to cloud devices and OS versions • Team ownership • Leverage tools • Constant and fast build feedbacks UNIT TESTS UI INTEGRATION/COMPONENT TESTS AUTOMATED E2E TESTS EXPLORATORY TESTING
39. @milan_kuveljic Thank you!