Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mobile release trains

Mobile release trains

- What is a release train
- What problems does it solve
- How to implement it for your mobile team

Avatar for Luis G. Valle

Luis G. Valle

March 04, 2021
Tweet

More Decks by Luis G. Valle

Other Decks in Programming

Transcript

  1. What is Travelperk? ⭐ Make business travelers happy ⭐ Help

    companies save money World’s largest inventory Flights, hotels, trains, accommodation All in one place, no more chasing receipts Search, book, pay, invoice, share and report — all on the same platform. Save money Cheaper options, low commissions, more data to optimise, Expensify integration
  2. Mobile Release trains at Travelperk Days 1-7 Development phase Day

    8 - 15.00 Code freeze QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint
  3. Work process for mobile engineering • One codebase per platform

    • Branches ◦ /release/x.x.x latest released code ◦ /master should always be releasable ◦ /feature/<feature-id> work branches • Small frequent PRs • Code reviews, pair programming • Feature flags
  4. Old release process Feature-based release ✅ Works great for one

    squad team ❌ Hold a release until desired feature is completed (no bug fixing) ❌ Multi-squad problems: ◦ Separate features block each other ◦ Delays and uncertainty ◦ Who owns QA for the single binary? is the feature done? Work on a feature Ship new app version Yes No
  5. “A release is the process of making a version of

    your app available to users” - Abraham Lincoln
  6. Stages of release processes Main branch where developers merge changes

    Merged changes are automatically tested There’s a CI server running those tests Continuous Integration Changes merged into the main branch are automatically deployed to a production-like environment Release to production is manual Continuous Delivery Changes merged into the main branch are automatically deployed to production No human intervention Continuous Deployment Source: https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
  7. Stages of release processes (for mobile) Main branch where developers

    merge changes Merged changes are automatically tested There’s a CI server running those tests Continuous Integration Changes merged into the main branch are automatically deployed to a production-like environment Release to production is manual Continuous Delivery Changes merged into the main branch are automatically deployed to production No human intervention Continuous Deployment Source: https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
  8. Why mobile releases are different from web/backend Distribution environment Updates

    are not instant: review process can take days and even reject the update Updates are not mandatory: users may never update to a new release Release as a single binary Apps are distributed as one single binary: we can’t update only parts of it To remove feature’s code you need to ship a new version of the whole project Rollback App stores do not allow rollbacks Any new bug means • New release for the whole app needs to be done (single binary) • It won’t be fixed today (we don’t control distribution environments)
  9. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train
  10. Notion page with things to do before starting a release

    train Mobile release train: Checklist [ ] Assign release captains for Android and iOS [ ] Check which feature flags will need enabling [ ] Force upgrade after the release? [ ] Test cases for new features are added to Test Checklist [ ] Check all translations are in Phrase [ ] Release notes created and translated [ ] Changelog updated [ ] Release train announced in Slack [ ] Clone Amplitude release dashboard
  11. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Train “departs” at 15.00 o’clock strictly Release branch cut from master New changes to master not allowed into the train from this point QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train: Departure
  12. desc "Start a new Release train" lane :start_release do #

    Code freeze release_version = prompt(text: "Please provide a version number for the new release:") create_branch(from: "master", release_version: release_version) # Generate builds create_staging_build(branch_name: git_branch) create_production_build(branch_name: git_branch) UI.success "🍻 STAGING build and PRODUCTION build generated for current branch" end Mobile release train: Departure
  13. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Release Captain drives this phase Manual* testing of production build. Core & feature tests shared in spreadsheet Feature flags checked here Only fixes accepted at this stage QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint * Working towards automation in Q1 2021 Mobile release train: QA
  14. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Allowed into the release branch if critical bugs are found during QA Non-critical bug fixes will board the next release train Needs agreement between Release Captain and PM Bug fix Mobile release train: bug fixing
  15. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Semi-automated process Release notes handled by PM Validation process in stores can take up to 2-3 days QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train: store submission
  16. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    1% > 2% > 5% > 20% > … > 100% to closely monitor adoption issues Staged rollout can be halted but never rolled back Hotfixes allowed for critical bugs -> 100% QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Hotfix Mobile release train: staged rollout
  17. Days 1-7 Development phase Day 8 - 15.00 Code freeze

    Rollout reaches 100% Release tagged on Github Release branch merged back to master QA phase Day 10 - 17.00 Store submission Staged rollout phase Day 15 100% release New sprint Mobile release train: Release
  18. desc "Finish release train: Tag version, create PR from release

    branch to master" lane :finish_release do # Get the release branch release_branch = sh("git branch -r | grep 'release' | tail -1 | tr -d ' ' | cut -d'/' -f2-").chomp # Check out release_branch and pull changes sh("git checkout #{release_branch}") # Get current version current_version = get_version_name # Tag release and push sh("git tag -a v#{current_version} -m '' && git push origin v#{current_version}") # Create Pull request open_pr_release(version: current_version) UI.success "🍻 Success!" end end Mobile release train: Finish a release
  19. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling
  20. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  21. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  22. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Start and finish release trains at fixed times ➔ Automatically create new release builds when hotfixes are merged into release branches and submit them ➔ Automatically run SAST/DAST tool nightly ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  23. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Start and finish release trains at fixed times ➔ Automatically create new release builds when hotfixes are merged into release branches and submit them ➔ Automatically run SAST/DAST tool nightly ➔ Release dashboard is created automatically ➔ Publish release rollout status reports automatically on Slack ➔ Halt rollout if error rate goes over a threshold ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  24. Start & finish release trains Build release & submit to

    stores Monitor release rollout & track errors Release train checklist & QA Communicate train status Tooling: working towards automation ➔ Triggered automatically from Bitrise at specific times ➔ Start and finish release trains at fixed times ➔ Automatically create new release builds when hotfixes are merged into release branches and submit them ➔ Automatically run SAST/DAST tool nightly ➔ Release dashboard is created automatically ➔ Publish release rollout status reports automatically on Slack ➔ Halt rollout if error rate goes over a threshold ➔ All train status changes are announced automatically ➔ Capacity to query: what’s the rollout status? Error rate? ➔ Capacity to action: halt release, generate stage build,... ➔ Automate the different checks starting with scripts. Ex, active feature flags, release changelog, etc. ➔ Currently evaluating platforms for E2E testing automation. Goal: nightly E2E tests running on cloud devices
  25. 3 Execute the process manually first 4 Find automation opportunities

    2 Define a process that works for you 1 Identify what are the pain points for your company