Intro to Continuous Integration at SoundCloud

Intro to Continuous Integration at SoundCloud

In the past year at SoundCloud, we've improved our CI from a single machine running our limited unit test suite to several machines running a linter, unit tests, acceptance tests and building several versions of our app including the AppStore version for every commit. This enabled us to move away from pull-requests and code reviews and use trunk based development and pairing, as well as scale the team from 3 to 7 developers.
We've stopped relying on Xcode Schemes for builds, we're using instead command line tools and compile-time parameters to automatically build internal versions of our app and distribute it internally.
In order to make our release process more reliable, the AppStore version of the app is created by re-signing an AdHoc build. This enables us to test the exact same binary that will be submitted to the AppStore.
This talk will focus on 3 areas of our continuous integration: testing, building with user defined build settings and signing apps.

E76dc85e3486993280ae6b4d2810f670?s=128

Vincent Garrigues

September 04, 2014
Tweet

Transcript

  1. None
  2. Continuous Integration at SoundCloud iOSDevUK september 2014 ! Vincent Garrigues

    @garriguv
  3. What is CI?

  4. “Continuous integration (CI) is the practice, in software engineering, of

    merging all developer working copies with a shared mainline several times a day.” http://en.wikipedia.org/wiki/Continuous_integration
  5. • trunk based development • lots of testing • CI

    pipeline
  6. None
  7. None
  8. None
  9. None
  10. None
  11. None
  12. None
  13. None
  14. None
  15. Why CI?

  16. • move fast • keep codebase healthy • ship reliable

    apps
  17. • move fast • keep codebase healthy • ship reliable

    apps
  18. None
  19. None
  20. • started from scratch • months in development • millions

    of users
  21. iOS Crash Complaints (avg per Week) 0 35 70 105

    140 April May June July August SoundCloud community team
  22. None
  23. How to get started?

  24. Start simple and iterate

  25. • run unit test suite • α and β versions

    • sign .app builds
  26. None
  27. Rake

  28. require 'rake' ! desc 'say Hello World!' task :hello do

    puts 'Hello World!' end
  29. require 'rake' ! desc 'say Hello World!' task :hello do

    puts 'Hello World!' end
  30. require 'rake' ! desc 'say Hello World!' task :hello do

    puts 'Hello World!' end
  31. require 'rake' ! desc 'say Hello World!' task :hello do

    puts 'Hello World!' end
  32. Running the unit tests

  33. use xcodebuild or xctool

  34. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)" test
  35. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)" test
  36. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)" test
  37. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)" test
  38. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)" test
  39. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)" clean
  40. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)" build
  41. rake tasks • rake clean • rake build • rake

    test
  42. α and β versions

  43. None
  44. User-defined build settings

  45. None
  46. None
  47. None
  48. None
  49. None
  50. Override user-defined build settings when running xcodebuild

  51. xcodebuild -workspace ci.xcworkspace -scheme ci -configuration Debug -sdk iphonesimulator7.1 -destination

    "platform=iOS Simulator,name=iPhone Retina (4-inch)” build
  52. xcodebuild (…) VG_BUNDLE_ID=com.company.app-alpha build

  53. xcodebuild (…) VG_BUNDLE_ID=com.company.app-alpha VG_DISPLAY_NAME=α build

  54. xcodebuild (…) VG_BUNDLE_ID=com.company.app-alpha VG_DISPLAY_NAME=α VG_CODE_SIGNING="'iPhone Distribution: Vincent Garrigues'" build

  55. Where can we store those settings?

  56. We use a YAML file

  57. defaults: &DEFAULTS app: &APP_DEFAULTS workspace: ci.xcworkspace scheme: ci configuration: Release

    display_name: ci-test bundle_identifier: com.company.ci-test codesign: &CODESIGN_DEFAULTS signing_identity: '"iPhone Distribution: Vincent Garrigues (X73EL3QFZB)"' provisioning_profile: provisioning/app_store.mobileprovision
  58. defaults: &DEFAULTS app: &APP_DEFAULTS workspace: ci.xcworkspace scheme: ci configuration: Release

    display_name: ci-test bundle_identifier: com.company.ci-test codesign: &CODESIGN_DEFAULTS signing_identity: '"iPhone Distribution: Vincent Garrigues (X73EL3QFZB)"' provisioning_profile: provisioning/app_store.mobileprovision
  59. defaults: &DEFAULTS app: &APP_DEFAULTS workspace: ci.xcworkspace scheme: ci configuration: Release

    display_name: ci-test bundle_identifier: com.company.ci-test codesign: &CODESIGN_DEFAULTS signing_identity: '"iPhone Distribution: Vincent Garrigues (X73EL3QFZB)"' provisioning_profile: provisioning/app_store.mobileprovision
  60. defaults: &DEFAULTS app: &APP_DEFAULTS workspace: ci.xcworkspace scheme: ci configuration: Release

    display_name: ci-test bundle_identifier: com.company.ci-test codesign: &CODESIGN_DEFAULTS signing_identity: '"iPhone Distribution: Vincent Garrigues (X73EL3QFZB)"' provisioning_profile: provisioning/app_store.mobileprovision
  61. defaults: &DEFAULTS app: &APP_DEFAULTS workspace: ci.xcworkspace scheme: ci configuration: Release

    display_name: ci-test bundle_identifier: com.company.ci-test codesign: &CODESIGN_DEFAULTS signing_identity: '"iPhone Distribution: Vincent Garrigues (X73EL3QFZB)"' provisioning_profile: provisioning/app_store.mobileprovision
  62. defaults: &DEFAULTS app: &APP_DEFAULTS workspace: ci.xcworkspace scheme: ci configuration: Release

    display_name: ci-test bundle_identifier: com.company.ci-test codesign: &CODESIGN_DEFAULTS signing_identity: '"iPhone Distribution: Vincent Garrigues (X73EL3QFZB)"' provisioning_profile: provisioning/app_store.mobileprovision
  63. defaults: (..) ! alpha: <<: *DEFAULTS app: <<: *APP_DEFAULTS display_name:

    α bundle_identifier: com.company.ci-test-alpha codesign: <<: *CODESIGN_DEFAULTS provisioning_profile: provisioning/alpha.mobileprovision
  64. defaults: (..) ! alpha: <<: *DEFAULTS app: <<: *APP_DEFAULTS display_name:

    α bundle_identifier: com.company.ci-test-alpha codesign: <<: *CODESIGN_DEFAULTS provisioning_profile: provisioning/alpha.mobileprovision
  65. defaults: (..) ! alpha: <<: *DEFAULTS app: <<: *APP_DEFAULTS display_name:

    α bundle_identifier: com.company.ci-test-alpha codesign: <<: *CODESIGN_DEFAULTS provisioning_profile: provisioning/alpha.mobileprovision
  66. rake build BUILD_ENV=alpha rake build BUILD_ENV=beta rake build BUILD_ENV=adhoc rake

    build BUILD_ENV=release
  67. rake build BUILD_ENV=alpha rake build BUILD_ENV=beta rake build BUILD_ENV=adhoc rake

    build BUILD_ENV=release
  68. rake build BUILD_ENV=alpha rake build BUILD_ENV=beta rake build BUILD_ENV=adhoc rake

    build BUILD_ENV=appstore
  69. rake build creates a .app

  70. We need a .ipa

  71. xcrun

  72. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  73. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  74. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  75. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  76. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  77. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  78. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  79. xcrun -v -sdk iphoneos PackageApplication ci-test-alpha.app -o ci-test-alpha.ipa --sign "iPhone

    Distribution: Vincent Garrigues" --embed provisioning/alpha.mobileprovision
  80. rake sign

  81. • rake clean • rake test • rake build •

    rake sign
  82. How can we use those rake tasks?

  83. None
  84. • rake build sign BUILD_ENV=alpha • rake build sign BUILD_ENV=beta

    • rake build sign BUILD_ENV=adhoc
  85. • rake build sign BUILD_ENV=alpha • rake build sign BUILD_ENV=beta

    • rake build sign BUILD_ENV=adhoc
  86. • rake build sign BUILD_ENV=alpha • rake build sign BUILD_ENV=beta

    • rake build sign BUILD_ENV=adhoc
  87. • rake build sign BUILD_ENV=alpha • rake build sign BUILD_ENV=beta

    • rake build sign BUILD_ENV=adhoc • rake build sign BUILD_ENV=appstore
  88. Ad-Hoc and AppStore builds have the same bundle id

  89. rake build BUILD_ENV=appstore ci-test-appstore.app ci-test-appstore.dSYM ci-test-appstore.dSYM.zip

  90. rake build BUILD_ENV=appstore rake ipa BUILD_ENV=appstore ci-test-appstore.app ci-test-appstore.dSYM ci-test-appstore.dSYM.zip ci-test-appstore.ipa

  91. rake build BUILD_ENV=appstore rake ipa BUILD_ENV=appstore rake ipa BUILD_ENV=adhoc ci-test-appstore.app

    ci-test-appstore.dSYM ci-test-appstore.dSYM.zip ci-test-appstore.ipa ci-test-adhoc.ipa
  92. rake build BUILD_ENV=appstore rake ipa BUILD_ENV=appstore rake ipa BUILD_ENV=adhoc ci-test-appstore.app

    ci-test-appstore.dSYM ci-test-appstore.dSYM.zip ci-test-appstore.ipa ci-test-adhoc.ipa
  93. ci-test-appstore.app ci-test-appstore.dSYM ci-test-appstore.dSYM.zip ci-test-appstore.ipa ci-test-adhoc.ipa rake build BUILD_ENV=appstore rake ipa

    BUILD_ENV=appstore rake ipa BUILD_ENV=adhoc
  94. The same binary, signed twice

  95. Sample project ! github.com/garriguv/ci-test

  96. None
  97. • linter (Xcode project, data model, acceptance tests, ruby, ObjC)

    • static analysis • unit tests (ruby and ObjC) • acceptance tests • custom app icon • deploy to HockeyApp • …
  98. None
  99. None
  100. Thank you! http://github.com/garriguv/ci-talk ! ! Vincent Garrigues @garriguv