Mobile Testing and Automation (OpenLondon 2017)

At Ocado Technology, we create all of the technology and software to run the Ocado retail business.

Automation is very important to us at Ocado, from automating our delivery warehouses with robots, right through to building and testing our mobile applications.

I will be discussing the build pipeline and testing approaches we use for the Android and iOS shopping apps.

There will be a particular focus on the ways we are reducing the number of manual steps and trying to automate as much as possible.

Transcript

1. MOBILE TESTING AND AUTOMATION Andrew Lord
2. Ocado is the world’s largest dedicated online grocery retailer We have 500,000 active shoppers 47,000 SKUs Over 50 miles of conveyors across 2 large distribution centres 200,000 orders a week ‘picked’ 3 million routing calculations – per second
3. THE SHOPPING APPS Android and iOS
4. THE JOURNEY
5. RELEASE TESTING • Manual testing • Before a production release • Various devices • Various OS versions
6. CONTINUOUS INTEGRATION • Build of all variants • Unit testing suite • Code quality checks • Static code analysis
7. ANALYTICS • App crashes • User behaviour • User sessions • Minimal effort
8. UNIT TESTING • Tests each component in isolation • Class, function, module etc. • Cover as much of project as possible • Quick to run • No dependencies on external services
9. CODE QUALITY CHECKS • Code style • Static code analysis • Complexity • Dependencies
10. QA TESTING • Dedicated tester • Manual tests • Ran for each change • Catch issues missed by developer • Catch issues early
11. UI TESTING • Test the app as a whole • Act like a real user • Test user-interface and interaction • Replace manual tests • Fake data vs real data
12. A/B TESTING • Choose between different options • Customer groups receive one option • Usually 50/50 • Monitor usage • Decision based on data
13. DIFFICULTIES The Journey to Automation
14. REWARDS The Journey to Automation
15. THE FUTURE The Journey to Automation
16. USER EXPERIENCE Android and iOS
17. CONSISTENCY across platforms
18. WORK IN PROGRESS
19. PERFORMANCE • Important to ensure using app is a smooth experience. • Memory usage, page loading and lag. • Don’t have automated performance testing - yet! • Monitoring, such as New Relic.
20. ocadotechnology @OcadoTechnology MOBILE TESTING AND AUTOMATION @andrewlord1990 andrewlord1990