Automation in Mobile Testing

Transcript

1. Messekongress ITfurVersicherungen - November 2018 Automation in Mobile Testing @N26

   With Milan Kuveljic Head of Quality Engineering at N26

2. ⬒ 300+ engineers - Berlin/Barcelona/New York ⬒ Deploying to production

   200+ times per week ⬒ Building crash-free mobile applications ⬒ Achieving regularly 5 stars in the App/Play Stores ⬒ Having Web app built for accessibility N26 is a tech company building modern banking @milan_kuveljic

3. Milan Kuveljic Head of Quality Engineering @N26 milan_kuveljic testingin.space

4. Our journey Automation principles Scope definition Architecture Tools and frameworks

   Continuous testing

5. Mobile Engineers build the app Test environment Test Engineers check

   the functionality

6. Mobile Engineers build the app Test environment Tests Passed

7. CHALLENGE

8. Testing becomes more time consuming and slows down the release

   process

9. Number of mobile devices and operating systems is increasing over

   time
10. None

11. AUTOMATION

12. INITIAL ASSUMPTIONS Automation as a silver bullet Automate all the

    things Automation is testing The best tool will solve all problems Automation is a tester’s job

13. Principles

14. Principles A test executed once it’s not a test anymore

    - it’s a check

15. Principles If you need to repeat a check, automate it

16. Principles Effective team collaboration Testing approach and scope Evaluate the

    right tools Continuous improvements

17. Test Engineers Mobile Engineers

18. Mobile Engineering

19. Scope UNIT TESTS UI INTEGRATION/COMPONENT TESTS E2E TESTS EXPLORATORY TESTING

20. Scope UI INTEGRATION/COMPONENT TESTS - Easy to implement - No

    external dependencies - API, 3rd party - Less prone to flakiness - Does not require application file - Mocking - Execution on real devices and simulators

21. Scope E2E TESTS - User journey - HTTP connections -

    Prone to flakiness - Requires application file - Execution on real devices and simulators

22. ARCHITECTURE

23. Architecture Domain Test Data Infrastructure Devices Reporting Integration

24. Architecture Domain Test structure Test suites

25. Architecture Test Data Configurable static test data Dynamic test data

    - API helpers IT’S ALSO AUTOMATION

26. Architecture Infrastructure Test Environments Mocks Limited Production Environment IT’S ALSO

    AUTOMATION

27. Architecture Devices In house mobile devices Devices on Cloud

28. Architecture Integration Manual build execution Continuous integration servers IT’S ALSO

    AUTOMATION

29. Architecture Reporting HTML reporting XML reporting Data Visualisation & Monitoring

    IT’S ALSO AUTOMATION

30. Capabilities of a specific tool Tool meets specific needs. Tool

    doesn’t seem to provide full test coverage Tool costs exceed the team budget Tools

31. Tools and frameworks https://dzone.com/articles/mobile-test-automation-framework-trends

32. Tools UI INTEGRATION/COMPONENT TESTS AUTOMATED E2E TESTS

33. SauceLabs devices Jenkins + Docker Chat channel Appium + Java/Kt

    + TestNG + PageObject Model E2E Framework Reporting dashboard

34. SauceLabs devices Jenkins + Docker Chat channel Espresso + XCUITest

    UI Framework Reporting dashboard

35. CONTINUOUS TESTING

36. Continuous testing Pull request

37. None

38. Continuous testing takeaways • Layered test architecture • Regression rate

    reduced • Mobile automation in the development scope • Extensive access to cloud devices and OS versions • Team ownership • Leverage tools • Constant and fast build feedbacks UNIT TESTS UI INTEGRATION/COMPONENT TESTS AUTOMATED E2E TESTS EXPLORATORY TESTING

39. @milan_kuveljic Thank you!