CocoaHeads x PAUG Meetup - Quality assurance and release process

Les enjeux d’une mise en prod poussée à des millions d’utilisateurs, ou comment s’assurer de la qualité d’une app de la PR jusqu’au store.

Transcript

1. CocoaHeads x PAUG Meetup 14 November 2019 Quality Assurance and Release Process
2. Meet the Speakers Olivier iOS Developer Anthony Android Developer
3. Context Some numbers
4. Android App figures Created on July 2013 40+ contributors 40% Kotlin 60% Java
5. iOS App figures Created on August 2013 20+ contributors 70% Swift 30% Objective-C
6. 65% 35% 1.2M user/day 100 - 200k booking/day 650k user/day 80 - 150k booking/day Traffic on each platform Mobile Web
7. 14 users/second Booking every second No instant updates, no rollbacks Huge traffic Mobile apps are not SPA Why so careful ? Even a small bug can reach thousands of users The main goal is to maintain apps stability Direct impact Ensure quality & stability
8. Challenges we're facing Traffic & update management Architecture / Modularization UI Migration
9. Android Studio Bitbucket Jenkins Fastlane SVN Xcode GitLab Bitrise Fastlane Development cycle
10. Strategy shifted and expanded
12. Does this feature behave as expected? PRODUCT TESTING Does this code give the expected result? CODE TESTING
13. Unit tests 1842 3067 War rooms Pre- Release PR and code review
14. Quiz
16. How many E2E test cases for this feature? A. 3 B. 15 C. 20 D. 30
17. Easy! As a logged in user, I can fill and certify my phone number from my private profile As a logged in user, I can a fill and certify my phone number during a publication I get an error message if my phone format is not valid
18. I get an error if the phone number is already used I get an error if the phone number is blacklisted I get an error message if the SMS code is wrong I can fill and certify my phone number during the account creation I can fill and certify my phone number during a booking I can certify my phone number after an account creation during a booking Really, guys?
20. Automation at full throttle
21. Automated tests stack Sauce Labs Qualcium Appium
22. From 3 hours to less than 1 hour Now executing even more test scenarios Testing both mobile apps continuously
24. Alpha channel / Dogfooding Beta channel Progressive Rollout Release
25. Crash / Monitoring Feature flags Soft / Hard updates Post- Release
26. What’s next Integrate native UI tests Continuous improvement QA Crash-free users: 99.9%