Etsy Journey to Continuous Integration for Mobile Apps

Slides presented during Velocity NY 2014.

While traditional websites can leverage their continuous delivery infrastructure to quickly identify and fix bugs, releasing mobile apps requires a 3rd party approval. Major bugs and security vulnerabilities call for emergency releases, which still require the app store approval and can mean days of poor user experience, developer frustration, and a negative impact to the app ratings and reviews.

A comprehensive suite of automated functional tests can greatly help detect defects early across the broad spectrum of devices and configurations. The challenge then becomes setting up a reliable, scalable, and sustainable infrastructure to automatically build and test your app on simulators and devices, as soon as a developer pushes new code.

Today, we want to share with the audience Etsy’s journey to setting up our Continuous Integration infrastructure for our mobile apps teams.

Transcript

1. Etsy’s Journey to Continuous Integration for Mobile Apps Tuesday, September 16, 14
2. @kepioo bit.ly/etsyautobots Tuesday, September 16, 14
3. 1990’s Tuesday, September 16, 14
4. 2010 2011 2012 2013 2014 50 web deploys per day 0 100 Tuesday, September 16, 14
5. Tuesday, September 16, 14
6. 2014 Tuesday, September 16, 14
7. iOS Average App Review Time 09 / 14 6 days bit.ly/ios-review-trend Tuesday, September 16, 14
8. Continuous Integration Tuesday, September 16, 14
9. “release” constantly (not necessarily to customers) Tuesday, September 16, 14
10. “If the code is not out there, it’s not teaching you anything” Tuesday, September 16, 14
11. Culture Tuesday, September 16, 14
12. Culture reviews try CI cluster Tests Apps Rotations Feature flags Bug Hunt automated build Tuesday, September 16, 14
13. Automated builds Tuesday, September 16, 14
14. command line or it does not count... Tuesday, September 16, 14
15. ipa build --scheme “Etsy” --configuration Release bit.ly/ios-ipa Tuesday, September 16, 14
16. build package for Test Flight package for App Store spot check with Test Flight submit Application Loader Tuesday, September 16, 14
17. Every commit should build the mainline bit.ly/mfowler-ci Tuesday, September 16, 14
18. We use Jenkins (not Xcode bots) Provisioning profiles in repo Build all targets Groovy post build plugin Tuesday, September 16, 14
19. bit.ly/nintendo-lamps Tuesday, September 16, 14
20. + = Tuesday, September 16, 14
21. magic to auto-resolve code signing Tuesday, September 16, 14
22. Tuesday, September 16, 14
23. ipa build --scheme “Etsy” --configuration Release --identity “iPhone: Etsy” --embed “/foo/bar.provision” Tuesday, September 16, 14
24. Every commit should build the mainline on an integration machine bit.ly/mfowler-ci Tuesday, September 16, 14
25. 25 mac mini Opscode Chef provisioning Homebrew + rbenv clicking required for initial Xcode install Tuesday, September 16, 14
26. iOS certificates signing identities Android SDK’s, build tools emulators Tuesday, September 16, 14
27. Everyone commits to the mainline everyday bit.ly/mfowler-ci Tuesday, September 16, 14
28. Branch in code Push to master Tuesday, September 16, 14
29. self.window.rootViewController = boringController Tuesday, September 16, 14
30. if ([ [EtsyConfig config] reskinEnabled ] ) { self.window.rootViewController = fancyController; } else { self.window.rootViewController = boringController; } Tuesday, September 16, 14
31. [userDefaults objectForKey:@”EtsyReskinEnabled”] Tuesday, September 16, 14
32. $server_config[‘ios_boe’][‘reskinEnabled”] = ‘on’; Tuesday, September 16, 14
33. $server_config[‘ios_boe’][‘reskinEnabled”] = ‘on’; $server_config[‘ios_boe’][‘reskinEnabled”] = ‘staff’; $server_config[‘ios_boe’][‘reskinEnabled”] = ‘5%’; Tuesday, September 16, 14
34. code “mess” feature flag clean up? Tuesday, September 16, 14
35. flag changes detection version ranges support Tuesday, September 16, 14
36. Code reviews Tuesday, September 16, 14
37. $> review -r nkammah -cc ios@etsy.com --title "Velocity demo" Tuesday, September 16, 14
38. Build Lint Unit Functional 1 - review script 2 - Github pull request builder 3 - Build Flow + Groovy console 1 2 3 Tuesday, September 16, 14
39. Tuesday, September 16, 14
40. bit.ly/try-poster Tuesday, September 16, 14
41. Build Lint Unit Functional 1 - bit.ly/trylib 2 - Build Flow + Groovy console 2 git diff origin/master 1 Tuesday, September 16, 14
42. bit.ly/trylib Test developer changes on CI cluster Tuesday, September 16, 14
43. unit tests Tuesday, September 16, 14
44. legacy code testing Tuesday, September 16, 14
45. feature unit test > Tuesday, September 16, 14
46. Testing dojos Tuesday, September 16, 14
47. 6 developers 1 computer / 1 projector 3 minutes to code then rotate simple testing objective Share / Learn / Have Fun Tuesday, September 16, 14
48. Functional tests Tuesday, September 16, 14
49. Tuesday, September 16, 14
50. Page Object Model bit.ly/page-object-model Tuesday, September 16, 14
51. In-House simulator Outsource on-device Tuesday, September 16, 14
52. 300 devices in the pool 4 dedicated iOS 4 dedicated android bit.ly/app-thwack Tuesday, September 16, 14
53. rolling window pass rate Tuesday, September 16, 14
54. Dog Food Tuesday, September 16, 14
55. etsy.com/appbuilds App Store Tuesday, September 16, 14
56. Tuesday, September 16, 14
57. App Rotations Tuesday, September 16, 14
58. 8 volunteers in a room QA facilitator Mix of devices find bugs + + Tuesday, September 16, 14
59. Bug Hunt (gamified focused testing) Tuesday, September 16, 14
60. Mission Tuesday, September 16, 14
61. Screenshot(s) Tuesday, September 16, 14
62. Details Tuesday, September 16, 14
63. Score ! Tuesday, September 16, 14
64. bit.ly/bug-hunt Tuesday, September 16, 14
65. Imagine Tuesday, September 16, 14
66. `git push` => < 15 min Tuesday, September 16, 14
67. Start your journey Tuesday, September 16, 14
68. Package your app without Xcode Organize a test dojo Write a functional test Build reviews automatically Tuesday, September 16, 14
69. Office hours wed. 11:20 am table 2 Slides : bit.ly/etsy-apps-ci Tuesday, September 16, 14
70. What about Android ? Tuesday, September 16, 14
71. We’re hiring! bit.ly/join-etsy Tuesday, September 16, 14
72. Questions? Tuesday, September 16, 14