Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mobile Continuous Integration at SoundCloud

Mobile Continuous Integration at SoundCloud

In the past two years at SoundCloud, we've shipped a brand new iPhone and iPad app and completely transformed the way we work. The team scaled from 3 to 10 developers, we've moved away from pull-requests and rely heavily on automated testing. As SoundCloud grew, its mobile application got more complex and the number of contributors expanded as well. Maintaining consistency and quality across teams is key to SoundCloud's continuing success. That's why we're building custom continuous integration solutions for our mobile apps.

Vincent Garrigues

April 27, 2015
Tweet

More Decks by Vincent Garrigues

Other Decks in Programming

Transcript

  1. Continuous Integration at
    SoundCloud
    Istanbul Tech Talks 2015
    Vincent Garrigues
    @garriguv

    View full-size slide

  2. Why invest in
    a mobile CI
    pipeline?

    View full-size slide

  3. • move fast
    • keep codebase healthy
    • confidence
    • ship reliable apps

    View full-size slide

  4. • move fast
    • keep codebase healthy
    • confidence
    • ship reliable apps

    View full-size slide

  5. • started from scratch
    • months in development
    • millions of users

    View full-size slide

  6. iOS Crash Complaints (avg per Week)
    0
    35
    70
    105
    140
    April May June July August
    2014 SoundCloud community team

    View full-size slide

  7. iOS Crash Complaints (avg per Week)
    0
    35
    70
    105
    140
    April 2014 July 2014 October 2014 January 2015
    2015 SoundCloud community team

    View full-size slide

  8. How do we
    work?

    View full-size slide

  9. • work on master
    • little to no branches
    • pairing
    • feature flags

    View full-size slide

  10. - (id)soundcloudInternalProvider
    {
    if ([FlipTheSwitch isDevEventgatewayEnabled]) {
    return [self eventGatewayProvider];
    } else {
    return [self eventLoggerProvider];
    }
    }
    Feature flagging
    github.com/michaelengland/fliptheswitch

    View full-size slide

  11. Feature flagging

    View full-size slide

  12. 2012 2013 2014 2015


















    View full-size slide

  13. started very
    simple

    View full-size slide

  14. • unit tests
    • α and β builds
    • app store build

    View full-size slide

  15. Overview of
    the current
    system

    View full-size slide

  16. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    View full-size slide

  17. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β
    ~ 5min 30sec ~ 7min ~ 3min
    ~ 15min

    View full-size slide

  18. Let's have a
    look at the
    details

    View full-size slide

  19. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    View full-size slide

  20. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  21. build
    local libraries

    unit tests
    i18n
    push
    unit
    tests
    acceptance
    tests build
    linter
    dependencies

    View full-size slide

  22. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~2000
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  23. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~2000
    ~5000
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  24. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~2000
    ~5000
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  25. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~5000
    ~2000
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  26. build
    unit
    tests
    acceptance
    tests build
    linter
    dependencies
    ~5000
    ~2000
    local libraries

    unit tests
    i18n
    push

    View full-size slide

  27. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    View full-size slide

  28. • frank
    • cucumber
    http://www.testingwithfrank.com

    https://cukes.info

    View full-size slide

  29. • flexible platform & OS
    • bandwidth and reachability
    settings
    • launch arguments

    View full-size slide

  30. stubby4j


    datasets

    View full-size slide

  31. mobile api
    proxy
    The proxy can change:
    • response status code
    • response body
    • record and undo actions
    (like, repost…)

    View full-size slide

  32. ~360
    acceptance
    tests

    View full-size slide

  33. ~360
    acceptance
    tests
    ~250 min

    View full-size slide

  34. distributed
    cucumber

    View full-size slide

  35. distributed cucumber

    View full-size slide

  36. build

    analysis

    unit tests
    acceptance
    tests
    AppStore

    AdHoc

    α and β

    ~ 5min 30sec ~ 7min ~ 3min
    ~ 15min

    View full-size slide

  37. acceptance
    tests
    • iOS version
    • iPhone (4S, 5, 6, 6+)
    • iPad (retina and non retina)
    • feature flag configurations

    View full-size slide

  38. What's the value
    of an acceptance
    test?

    View full-size slide

  39. avoid
    regressions on
    hard to test
    flows/screens

    View full-size slide

  40. keeping the
    build green

    View full-size slide

  41. https://www.youtube.com/watch?v=BhMSzC1crr0 SpaceX

    View full-size slide

  42. Flaky tests
    • test driver is flaky
    • backend is flaky
    • app is unpredictable

    View full-size slide

  43. Flaky tests
    • identify
    • isolate
    • fix

    View full-size slide

  44. flakyrazor
    1. Take failing test out of the
    test pool
    2. Run the test multiple times
    (flaky or failing?)
    3. Assign it to the author/
    committer

    View full-size slide

  45. flakyrazor
    4. Assess test value
    5. Act on test duration changes
    6. Show statistics on why the
    test failed

    View full-size slide

  46. Taking care of
    our build
    machines

    View full-size slide







  47. ❤ ❤

    View full-size slide

  48. • we have images for all the
    machines
    • CLIs to provision the
    machines remotely
    • OS X server stores the
    images and controls the
    imaging process

    View full-size slide

  49. What about
    Android?

    View full-size slide

  50. Thank you!
    Istanbul Tech Talks 2015
    Vincent Garrigues
    @garriguv

    View full-size slide