Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
No content
Slide 2
Slide 2 text
Continuous Integration at SoundCloud Istanbul Tech Talks 2015 Vincent Garrigues @garriguv
Slide 3
Slide 3 text
Why invest in a mobile CI pipeline?
Slide 4
Slide 4 text
• move fast • keep codebase healthy • confidence • ship reliable apps
Slide 5
Slide 5 text
• move fast • keep codebase healthy • confidence • ship reliable apps
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
• started from scratch • months in development • millions of users
Slide 9
Slide 9 text
iOS Crash Complaints (avg per Week) 0 35 70 105 140 April May June July August 2014 SoundCloud community team
Slide 10
Slide 10 text
iOS Crash Complaints (avg per Week) 0 35 70 105 140 April 2014 July 2014 October 2014 January 2015 2015 SoundCloud community team
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
How do we work?
Slide 13
Slide 13 text
• work on master • little to no branches • pairing • feature flags
Slide 14
Slide 14 text
- (id)soundcloudInternalProvider { if ([FlipTheSwitch isDevEventgatewayEnabled]) { return [self eventGatewayProvider]; } else { return [self eventLoggerProvider]; } } Feature flagging github.com/michaelengland/fliptheswitch
Slide 15
Slide 15 text
Feature flagging
Slide 16
Slide 16 text
Timeline
Slide 17
Slide 17 text
2012 2013 2014 2015
Slide 18
Slide 18 text
started very simple
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
⛔
Slide 21
Slide 21 text
No content
Slide 22
Slide 22 text
• unit tests • α and β builds • app store build
Slide 23
Slide 23 text
Overview of the current system
Slide 24
Slide 24 text
build analysis unit tests acceptance tests AppStore AdHoc α and β
Slide 25
Slide 25 text
build analysis unit tests acceptance tests AppStore AdHoc α and β ~ 5min 30sec ~ 7min ~ 3min ~ 15min
Slide 26
Slide 26 text
Let's have a look at the details
Slide 27
Slide 27 text
build analysis unit tests acceptance tests AppStore AdHoc α and β
Slide 28
Slide 28 text
build unit tests acceptance tests build linter dependencies local libraries unit tests i18n push
Slide 29
Slide 29 text
build local libraries unit tests i18n push unit tests acceptance tests build linter dependencies
Slide 30
Slide 30 text
build unit tests acceptance tests build linter dependencies ~2000 local libraries unit tests i18n push
Slide 31
Slide 31 text
build unit tests acceptance tests build linter dependencies ~2000 ~5000 local libraries unit tests i18n push
Slide 32
Slide 32 text
build unit tests acceptance tests build linter dependencies ~2000 ~5000 local libraries unit tests i18n push
Slide 33
Slide 33 text
build unit tests acceptance tests build linter dependencies ~5000 ~2000 local libraries unit tests i18n push
Slide 34
Slide 34 text
build unit tests acceptance tests build linter dependencies ~5000 ~2000 local libraries unit tests i18n push
Slide 35
Slide 35 text
build analysis unit tests acceptance tests AppStore AdHoc α and β
Slide 36
Slide 36 text
• frank • cucumber http://www.testingwithfrank.com https://cukes.info
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
• flexible platform & OS • bandwidth and reachability settings • launch arguments
Slide 39
Slide 39 text
stubby4j ✅ ⛔ datasets
Slide 40
Slide 40 text
mobile api proxy The proxy can change: • response status code • response body • record and undo actions (like, repost…)
Slide 41
Slide 41 text
~360 acceptance tests
Slide 42
Slide 42 text
~360 acceptance tests ~250 min
Slide 43
Slide 43 text
Sharding
Slide 44
Slide 44 text
distributed cucumber
Slide 45
Slide 45 text
distributed cucumber
Slide 46
Slide 46 text
x20 VM
Slide 47
Slide 47 text
build analysis unit tests acceptance tests AppStore AdHoc α and β ~ 5min 30sec ~ 7min ~ 3min ~ 15min
Slide 48
Slide 48 text
acceptance tests • iOS version • iPhone (4S, 5, 6, 6+) • iPad (retina and non retina) • feature flag configurations
Slide 49
Slide 49 text
What's the value of an acceptance test?
Slide 50
Slide 50 text
avoid regressions on hard to test flows/screens
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
keeping the build green
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
No content
Slide 56
Slide 56 text
pre-ci
Slide 57
Slide 57 text
https://www.youtube.com/watch?v=BhMSzC1crr0 SpaceX
Slide 58
Slide 58 text
Flaky tests
Slide 59
Slide 59 text
Flaky tests • test driver is flaky • backend is flaky • app is unpredictable
Slide 60
Slide 60 text
Flaky tests • identify • isolate • fix
Slide 61
Slide 61 text
flakyrazor
Slide 62
Slide 62 text
flakyrazor 1. Take failing test out of the test pool 2. Run the test multiple times (flaky or failing?) 3. Assign it to the author/ committer
Slide 63
Slide 63 text
flakyrazor 4. Assess test value 5. Act on test duration changes 6. Show statistics on why the test failed
Slide 64
Slide 64 text
Taking care of our build machines
Slide 65
Slide 65 text
❤ ❤ ❤ ❤ ❤ ❤ ❤ ❤
Slide 66
Slide 66 text
No content
Slide 67
Slide 67 text
• we have images for all the machines • CLIs to provision the machines remotely • OS X server stores the images and controls the imaging process
Slide 68
Slide 68 text
What about Android?
Slide 69
Slide 69 text
Thank you! Istanbul Tech Talks 2015 Vincent Garrigues @garriguv