Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mobile Continuous Integration at SoundCloud

Mobile Continuous Integration at SoundCloud

In the past two years at SoundCloud, we've shipped a brand new iPhone and iPad app and completely transformed the way we work. The team scaled from 3 to 10 developers, we've moved away from pull-requests and rely heavily on automated testing. As SoundCloud grew, its backend infrastructure got more complex with the addition of many microservices. Ensuring our apps to work with these microservices in production is key to SoundCloud's continuing success. That's why we're building custom continuous integration solutions for our mobile apps.

https://www.youtube.com/watch?v=Rq721qtKKNk

Vincent Garrigues

September 16, 2015
Tweet

More Decks by Vincent Garrigues

Other Decks in Programming

Transcript

  1. Mobile Continuous Integration
    at SoundCloud
    NSSpain

    2015-09-16

    Vincent Garrigues
    @garriguv

    View full-size slide

  2. Why invest in
    a mobile CI
    pipeline?

    View full-size slide

  3. • move fast
    • keep codebase healthy
    • confidence
    • ship reliable apps

    View full-size slide

  4. • move fast
    • keep codebase healthy
    • confidence
    • ship reliable apps

    View full-size slide

  5. • started from scratch
    • months in development
    • millions of users

    View full-size slide

  6. iOS Crash Complaints (avg per Week)
    0
    35
    70
    105
    140
    April May June July August
    2014 SoundCloud community team

    View full-size slide

  7. iOS Crash Complaints (avg per Week)
    0
    35
    70
    105
    140
    April 2014 August 2014 December 2014 April 2015 August 2015
    2015 SoundCloud community team

    View full-size slide

  8. How do we
    work?

    View full-size slide

  9. • pairing
    • short lived branches
    • feature flags

    View full-size slide

  10. - (id)soundcloudInternalProvider
    {
    if ([FlipTheSwitch isDevEventgatewayEnabled]) {
    return [self eventGatewayProvider];
    } else {
    return [self eventLoggerProvider];
    }
    }
    Feature flagging
    github.com/michaelengland/fliptheswitch

    View full-size slide

  11. Feature flagging
    github.com/michaelengland/fliptheswitch

    View full-size slide

  12. 2012 2013 2014 2015


















    View full-size slide

  13. started very
    simple

    View full-size slide

  14. • unit tests
    • α and β builds
    • app store build

    View full-size slide

  15. Overview of
    the current
    system

    View full-size slide

  16. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    View full-size slide

  17. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β
    ~ 10min ~ 10min ~ 3min
    ~ 20min

    View full-size slide

  18. Let's have a
    look at the
    details

    View full-size slide

  19. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    View full-size slide

  20. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  21. build
    local libraries

    unit tests
    i18n
    push
    unit
    tests
    acceptance
    tests build
    linter
    dependencies

    View full-size slide

  22. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~3000
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  23. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~3000
    ~7000
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  24. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~3000
    local libraries

    unit tests
    i18n
    push
    ~7000

    View full-size slide

  25. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~3000
    local libraries

    unit tests
    i18n
    push
    ~7000

    View full-size slide

  26. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    View full-size slide

  27. • frank
    • cucumber
    http://www.testingwithfrank.com

    https://cukes.info

    View full-size slide

  28. • flexible platform & OS
    • bandwidth and reachability
    settings
    • launch arguments

    View full-size slide

  29. mobile api
    proxy
    The proxy can change:
    • response status code
    • response body
    • record and undo actions
    (like, repost…)

    View full-size slide

  30. ~350
    acceptance
    tests

    View full-size slide

  31. ~350
    acceptance
    tests
    ~200 min

    View full-size slide

  32. distributed
    cucumber

    View full-size slide

  33. distributed cucumber

    View full-size slide

  34. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    ~ 10min ~ 10min ~ 3min
    ~ 20min

    View full-size slide

  35. acceptance
    tests
    • iOS version
    • iPhone (4S, 5, 6, 6+)
    • iPad (retina and non retina)
    • feature flag configurations

    View full-size slide

  36. What's the value
    of an acceptance
    test?

    View full-size slide

  37. avoid
    regressions on
    hard to test
    flows/screens

    View full-size slide

  38. keeping the
    build green

    View full-size slide

  39. https://www.youtube.com/watch?v=BhMSzC1crr0 SpaceX

    View full-size slide

  40. Flaky tests
    • test driver is flaky
    • backend is flaky
    • app is unpredictable

    View full-size slide

  41. Flaky tests
    • identify
    • isolate
    • fix

    View full-size slide

  42. flakyrazor
    1. Take failing test out of the
    test pool
    2. Run the test multiple times
    (flaky or failing?)
    3. Assign it to the author/
    committer

    View full-size slide

  43. flakyrazor
    4. Assess test value
    5. Act on test duration changes
    6. Show statistics on why the
    test failed

    View full-size slide

  44. Taking care of
    our build
    machines

    View full-size slide







  45. ❤ ❤

    View full-size slide

  46. • we have images for all the
    machines
    • CLIs to provision the
    machines remotely
    • OS X server stores the
    images and controls the
    imaging process

    View full-size slide

  47. Thank you!
    NSSpain

    2015-09-16

    Vincent Garrigues
    @garriguv

    View full-size slide