Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mobile release trains

Mobile release trains

- What is a release train
- What problems does it solve
- How to implement it for your mobile team

Luis G. Valle

March 04, 2021
Tweet

More Decks by Luis G. Valle

Other Decks in Programming

Transcript

  1. Mobile release
    trains
    Luis González Valle
    @lgvalle

    View Slide

  2. What is Travelperk?
    ⭐ Make business travelers happy
    ⭐ Help companies save money
    World’s largest inventory
    Flights, hotels, trains, accommodation
    All in one place, no more chasing receipts
    Search, book, pay, invoice, share and report — all on the
    same platform.
    Save money
    Cheaper options, low commissions, more data to optimise,
    Expensify integration

    View Slide

  3. Mobile Release trains at
    Travelperk
    Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint

    View Slide

  4. Why?
    Squads at Travelperk
    💛 With mobile capabilities in 2020

    View Slide

  5. Why?
    Squads at Travelperk
    💛 With mobile capabilities in 2021

    View Slide

  6. Why?
    Squads at Travelperk
    💛 With mobile capabilities in 2022?

    View Slide

  7. Work process for mobile engineering
    ● One codebase per platform
    ● Branches
    ○ /release/x.x.x latest released code
    ○ /master should always be releasable
    ○ /feature/ work branches
    ● Small frequent PRs
    ● Code reviews, pair programming
    ● Feature flags

    View Slide

  8. Old release process
    Feature-based release
    ✅ Works great for one squad team
    ❌ Hold a release until desired feature is completed (no bug fixing)
    ❌ Multi-squad problems:
    ○ Separate features block each other
    ○ Delays and uncertainty
    ○ Who owns QA for the single binary?
    is the
    feature
    done?
    Work on a
    feature
    Ship new
    app version
    Yes
    No

    View Slide

  9. What is a
    release?

    View Slide

  10. “A release is the
    process of making a
    version of your app
    available to users”
    - Abraham Lincoln

    View Slide

  11. Stages of release processes
    Main branch where developers merge changes
    Merged changes are automatically tested
    There’s a CI server running those tests

    Continuous Integration
    Changes merged into the main branch are automatically deployed to a production-like
    environment
    Release to production is manual

    Continuous Delivery
    Changes merged into the main branch are automatically deployed to production
    No human intervention

    Continuous Deployment
    Source: https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment

    View Slide

  12. Stages of release processes (for mobile)
    Main branch where developers merge changes
    Merged changes are automatically tested
    There’s a CI server running those tests

    Continuous Integration
    Changes merged into the main branch are automatically deployed to a production-like
    environment
    Release to production is manual

    Continuous Delivery
    Changes merged into the main branch are automatically deployed to production
    No human intervention

    Continuous Deployment
    Source: https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment

    View Slide

  13. Why mobile releases are different from web/backend
    Distribution environment
    Updates are not instant: review
    process can take days and even
    reject the update
    Updates are not mandatory:
    users may never update to a new
    release
    Release as a single binary
    Apps are distributed as one single
    binary: we can’t update only parts
    of it
    To remove feature’s code you
    need to ship a new version of the
    whole project
    Rollback
    App stores do not allow rollbacks
    Any new bug means
    ● New release for the whole
    app needs to be done
    (single binary)
    ● It won’t be fixed today (we
    don’t control distribution
    environments)

    View Slide

  14. Release trains

    View Slide

  15. Fixed schedule
    Metrics
    Release train main features
    Release captain Feature flags QA

    View Slide

  16. Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint
    Mobile release train

    View Slide

  17. Notion page with things to do before starting a release train
    Mobile release train: Checklist
    [ ] Assign release captains for Android and iOS
    [ ] Check which feature flags will need enabling
    [ ] Force upgrade after the release?
    [ ] Test cases for new features are added to Test Checklist
    [ ] Check all translations are in Phrase
    [ ] Release notes created and translated
    [ ] Changelog updated
    [ ] Release train announced in Slack
    [ ] Clone Amplitude release dashboard

    View Slide

  18. Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    Train “departs” at 15.00 o’clock strictly
    Release branch cut from master
    New changes to master not allowed into
    the train from this point
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint
    Mobile release train: Departure

    View Slide

  19. desc "Start a new Release train"
    lane :start_release do
    # Code freeze
    release_version = prompt(text: "Please provide a version number for the new release:")
    create_branch(from: "master", release_version: release_version)
    # Generate builds
    create_staging_build(branch_name: git_branch)
    create_production_build(branch_name: git_branch)
    UI.success "🍻 STAGING build and PRODUCTION build generated for current branch"
    end
    Mobile release train: Departure

    View Slide

  20. Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    Release Captain drives this phase
    Manual* testing of production build.
    Core & feature tests shared in spreadsheet
    Feature flags checked here
    Only fixes accepted at this stage
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint
    * Working towards automation in Q1 2021
    Mobile release train: QA

    View Slide

  21. Mobile release train: QA checklist

    View Slide

  22. Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint
    Allowed into the release branch if critical
    bugs are found during QA
    Non-critical bug fixes will board the next
    release train
    Needs agreement between Release
    Captain and PM
    Bug fix
    Mobile release train: bug fixing

    View Slide

  23. Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    Semi-automated process
    Release notes handled by PM
    Validation process in stores can take up to
    2-3 days
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint
    Mobile release train: store submission

    View Slide

  24. Mobile release train: store submission

    View Slide

  25. Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    1% > 2% > 5% > 20% > … > 100%
    to closely monitor adoption issues
    Staged rollout can be halted but never
    rolled back
    Hotfixes allowed for critical bugs -> 100%
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint
    Hotfix
    Mobile release train: staged rollout

    View Slide

  26. Mobile release train: staged rollout

    View Slide

  27. Days 1-7
    Development phase
    Day 8 - 15.00
    Code freeze
    Rollout reaches 100%
    Release tagged on Github
    Release branch merged back to master
    QA phase
    Day 10 - 17.00
    Store submission
    Staged rollout phase
    Day 15
    100% release
    New sprint
    Mobile release train: Release

    View Slide

  28. desc "Finish release train: Tag version, create PR from release branch to master"
    lane :finish_release do
    # Get the release branch
    release_branch = sh("git branch -r | grep 'release' | tail -1 | tr -d ' ' | cut -d'/' -f2-").chomp
    # Check out release_branch and pull changes
    sh("git checkout #{release_branch}")
    # Get current version
    current_version = get_version_name
    # Tag release and push
    sh("git tag -a v#{current_version} -m '' && git push origin v#{current_version}")
    # Create Pull request
    open_pr_release(version: current_version)
    UI.success "🍻 Success!"
    end
    end
    Mobile release train: Finish a release

    View Slide

  29. Tools &
    automation

    View Slide

  30. Start & finish release trains
    Build release & submit to stores
    Monitor release rollout & track errors
    Release train checklist & QA
    Communicate train status
    Tooling

    View Slide

  31. Start & finish release trains
    Build release & submit to stores
    Monitor release rollout & track errors
    Release train checklist & QA
    Communicate train status
    Tooling: working towards automation
    ➔ Automate the different checks starting with scripts. Ex, active
    feature flags, release changelog, etc.
    ➔ Currently evaluating platforms for E2E testing automation.
    Goal: nightly E2E tests running on cloud devices

    View Slide

  32. Start & finish release trains
    Build release & submit to stores
    Monitor release rollout & track errors
    Release train checklist & QA
    Communicate train status
    Tooling: working towards automation
    ➔ Triggered automatically from Bitrise at specific times
    ➔ Automate the different checks starting with scripts. Ex, active
    feature flags, release changelog, etc.
    ➔ Currently evaluating platforms for E2E testing automation.
    Goal: nightly E2E tests running on cloud devices

    View Slide

  33. Start & finish release trains
    Build release & submit to stores
    Monitor release rollout & track errors
    Release train checklist & QA
    Communicate train status
    Tooling: working towards automation
    ➔ Triggered automatically from Bitrise at specific times
    ➔ Start and finish release trains at fixed times
    ➔ Automatically create new release builds when hotfixes are
    merged into release branches and submit them
    ➔ Automatically run SAST/DAST tool nightly
    ➔ Automate the different checks starting with scripts. Ex, active
    feature flags, release changelog, etc.
    ➔ Currently evaluating platforms for E2E testing automation.
    Goal: nightly E2E tests running on cloud devices

    View Slide

  34. Start & finish release trains
    Build release & submit to stores
    Monitor release rollout & track errors
    Release train checklist & QA
    Communicate train status
    Tooling: working towards automation
    ➔ Triggered automatically from Bitrise at specific times
    ➔ Start and finish release trains at fixed times
    ➔ Automatically create new release builds when hotfixes are
    merged into release branches and submit them
    ➔ Automatically run SAST/DAST tool nightly
    ➔ Release dashboard is created automatically
    ➔ Publish release rollout status reports automatically on Slack
    ➔ Halt rollout if error rate goes over a threshold
    ➔ Automate the different checks starting with scripts. Ex, active
    feature flags, release changelog, etc.
    ➔ Currently evaluating platforms for E2E testing automation.
    Goal: nightly E2E tests running on cloud devices

    View Slide

  35. Start & finish release trains
    Build release & submit to stores
    Monitor release rollout & track errors
    Release train checklist & QA
    Communicate train status
    Tooling: working towards automation
    ➔ Triggered automatically from Bitrise at specific times
    ➔ Start and finish release trains at fixed times
    ➔ Automatically create new release builds when hotfixes are
    merged into release branches and submit them
    ➔ Automatically run SAST/DAST tool nightly
    ➔ Release dashboard is created automatically
    ➔ Publish release rollout status reports automatically on Slack
    ➔ Halt rollout if error rate goes over a threshold
    ➔ All train status changes are announced automatically
    ➔ Capacity to query: what’s the rollout status? Error rate?
    ➔ Capacity to action: halt release, generate stage build,...
    ➔ Automate the different checks starting with scripts. Ex, active
    feature flags, release changelog, etc.
    ➔ Currently evaluating platforms for E2E testing automation.
    Goal: nightly E2E tests running on cloud devices

    View Slide

  36. 3
    Execute the process manually first
    4
    Find automation opportunities
    2
    Define a process that works for you
    1
    Identify what are the pain points
    for your company

    View Slide

  37. https://www.applause.com/blog/release-planning-mobile-release-train
    https://developers.soundcloud.com/blog/quality-mobile-trains
    https://www.scaledagileframework.com/agile-release-train/
    https://medium.com/@SkyscannerEng/the-present-and-future-of-app-release-at-skyscanner-b393c88972a8
    https://glovoapp.zoom.us/rec/play/UN2cJPb6wUS5ehpxbPid_FpFBZgbnD3MhCru8jfXtkaQnZPAZsAoYLSK1G
    O3-swBXkXLEenqNeNxiFjZ.fNXbB5-sP6nTyaRs
    https://medium.com/trinity-mirror-digital/why-we-started-riding-release-trains-for-our-mobile-applications-4
    1f344b3ac50
    Further reading

    View Slide

  38. We are hiring!
    https://www.travelperk.com/careers/

    View Slide

  39. Thanks!
    Luis González Valle
    @lgvalle

    View Slide