Building iOS apps at scale

Slide 1

Slide 1 text

Building iOS apps at scale Yusei Nishiyama @yuseinishiyama

Slide 2

Slide 2 text

• iOS, Android, Web • 67 countries • 21 languages • 60M monthly users (Japan) • 30M+ monthly users   (Outside Japan) https://info.cookpad.com/en https://cookpad.com/en

Slide 3

Slide 3 text

Distributed Team

Slide 4

Slide 4 text

Working at Cookpad is … • Working in a large team • Working with large code base

Slide 5

Slide 5 text

Working at Cookpad is … • Working in a large team • Working with a large code base

Slide 6

Slide 6 text

• Someone broke the build • We can’t release today because {name} is on vacation • Code review never ﬁnishes

Slide 7

Slide 7 text

Automation with CI • Testing • Nitpicking • Beta distribution • Release management

Slide 8

Slide 8 text

CI Server

Slide 9

Slide 9 text

CI Server

Slide 10

Slide 10 text

CI Server

Slide 11

Slide 11 text

CI Server

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

Fastlane in Cookpad • 7 Fastﬁles • 28 lanes • 740 lines • 5 plugins

Slide 14

Slide 14 text

Sync dSYMs desc "Refresh dSYMs" lane :refresh_dsyms do version = get_version_number build_number = get_build_number # Download dSYM files from iTC into a temporary directory download_dsyms(version: version, build_number: build_number) # Upload them to Crashlytics upload_symbols_to_crashlytics # Delete the temporary dSYM files clean_build_artifacts end

Slide 15

Slide 15 text

Release the App Spaceship::Tunes.login app_identifier = CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier) edit_version = Spaceship::Application.find(app_identifier).edit_version version_to_release = edit_version.version UI.user_error! "No pending verion to release" unless edit_version.app_status == "Pending Developer Release" edit_version.release! UI.success "Version #{version_to_release} is released " slack( channel: "#release", attachment_properties: { pretext: ":appleinc: *Cookpad iOS #{version_to_release} - Released*” }) refresh_dsyms

Slide 16

Slide 16 text

Triggers beta distribution

Slide 17

Slide 17 text

Check Labels desc "Check if pr is marked as beta distribution" private_lane :beta_distribution do labels.any? { |label| label == 'beta distribution' } end desc "Get labels of pull request" private_lane :labels do pr_number = ENV['PR_NUMBER'] repo = ENV['GITHUB_PULL_REQUEST_REPO'] result = github_api( http_method: 'GET', path: "/repos/#{repo}/issues/#{pr_number}/labels" ) result[:json].map { |label| label['name'] } end

Slide 18

Slide 18 text

Notify a New Build desc "Notify new beta distribution" private_lane :notify_build do messages = "A new version is available" pr_number = ENV['PR_NUMBER'] repo = ENV['GITHUB_PULL_REQUEST_REPO'] github_api( http_method: 'POST', path: "/repos/#{repo}/issues/#{pr_number}/comments", body: { "body": message } ) end

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

CI Server

Slide 23

Slide 23 text

Danger Danger runs during your CI process, and gives teams the chance to automate common code review chores. This provides another logical step in your build, through this Danger can help lint your rote tasks in daily code review. You can use Danger to codify your teams norms. Leaving humans to think about harder problems. She does this by leaving messages inside your PRs based on rules that you create with the Ruby scripting language. Over time, as rules are adhered to, the message is amended to reflect the current state of the code review http://danger.systems/ruby/

Slide 24

Slide 24 text

Dangerﬁle github.dismiss_out_of_range_messages({ error: false, warning: true, message: true, markdown: true }) xcode_summary.inline_mode = true xcode_summary.ignored_files = ['Pods/**'] xcode_summary.ignored_results { |result| result.message.start_with? 'ld' } log = File.join('logs', 'test.json') xcode_summary.report log if File.exist?(log) swiftlint.binary_path = './Pods/SwiftLint/swiftlint' swiftlint.lint_files inline_mode: true

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

CI Server

Slide 27

Slide 27 text

InﬂuxDB • Only purpose-built platform for any time-based data • Written in Go • Built-in HTTP API • SQL-like query language • Answer queries in real-time • Rich client libraries

Slide 28

Slide 28 text

Post Code Coverage lane :post_coverage do rate = export_coverage if rate influxdb(table_name: table_name, values: {coverage: rate}) slack(message: "Code coverage: #{rate.to_f.round(4) * 100}%", channel: “#ios- notifications") end end private_lane :export_coverage do fastlane_require 'nokogiri' output_directory = 'coverage_report' reports = Dir.glob(File.join(ROOT_PATH, output_directory, '*.xml')) slather(use_bundle_exec: true, output_directory: output_directory) if reports.empty? nil else xml = Nokogiri::XML(File.open(reports.first)) coverage = xml.xpath('//coverage').first coverage.attr("line-rate").to_f end end

Slide 29

Slide 29 text

CI Server

Slide 30

Slide 30 text

Grafana • Rich metrics dashboard and editor • Fast rendering • Support InﬂuxDB as a data source • Query editor

Slide 31

Slide 31 text

Visualize Code Coverage

Slide 32

Slide 32 text

Other metrics

Slide 33

Slide 33 text

Working at Cookpad is … • Working in a large team • Working with a large code base

Slide 34

Slide 34 text

Let’s improve the registration screen…

Slide 35

Slide 35 text

Let’s improve the registration screen… • Implement

Slide 36

Slide 36 text

Let’s improve the registration screen… • Implement • Tweak constraints

Slide 37

Slide 37 text

Let’s improve the registration screen… • Implement • Tweak constraints • Test with older iOS versions

Slide 38

Slide 38 text

Let’s improve the registration screen… • Implement • Tweak constraints • Test with older iOS versions • Fix tests

Slide 39

Slide 39 text

Let’s improve the registration screen… • Implement • Tweak constraints • Test with older iOS versions • Fix tests • Wait for CI

Slide 40

Slide 40 text

How many times   did you build so far?

Slide 41

Slide 41 text

Reduce Build Time • Measurement • Improvement

Slide 42

Slide 42 text

What I will “not” talk about • Annotate all types explicitly • Write as much as code in a single ﬁle • Get your Mac Pro

Slide 43

Slide 43 text

Reduce Build Time • Measurement • Improvement

Slide 44

Slide 44 text

“Premature optimization is the root of all evil.”    — Donald Knuth —

Slide 45

Slide 45 text

Display build duration $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

Slide 46

Slide 46 text

Build Process • Dependency Manager (CocoaPods, Carthage) • Compile source ﬁles • Run Scripts (Linter, Formatter)

Slide 47

Slide 47 text

fastlane/report.xml …

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Build Process • Dependency Manager (CocoaPods, Carthage) • Compile source ﬁles • Run Scripts (Linter, Formatter)

Slide 52

Slide 52 text

Build Process • Dependency Manager (CocoaPods, Carthage) • Compile source ﬁles • Function A • Function B • … • Runs Scripts (Linter, Formatter)

Slide 53

Slide 53 text

Warn long function bodies

Slide 54

Slide 54 text

Time function bodies $ pbpaste | grep '.[0-9]ms' | sort -nr | head -20

Slide 55

Slide 55 text

xcactivitylog • You can access build logs under DerivedData • xcactivitylog is just gzip. You can check the content as a plain text just by unzipping it

Slide 56

Slide 56 text

giginet/danger-xcproﬁler (in Dangerfile)    xcprofiler.thresholds = { warn: 50, fail: 500 } xcprofiler.report ‘ProductName'

Slide 57

Slide 57 text

Reduce Build Time • Measurement • Improvement

Slide 58

Slide 58 text

Test with Real Data $ cloc    github.com/AlDanial/cloc v 1.72 T=3.40 s (153.3 files/s, 16646.6 lines/s) ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- Swift 497 8944 3122 43831 JSON 18 0 0 522 Objective C 3 37 5 166 C/C++ Header 4 19 6 27 ------------------------------------------------------------------------------- SUM: 522 9000 3133 44546 -------------------------------------------------------------------------------

Slide 59

Slide 59 text

Test with Real Data • 40,000 lines of Swift code • CocoaPods • 38 direct dependencies and 60 libraries • Linter in build phases • CI archives the app and distributes it for internal testing

Slide 60

Slide 60 text

(Local clean build: 330s)

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

Whole Module Optimization

Slide 63

Slide 63 text

WMO without Optimization

Slide 64

Slide 64 text

(Local clean build: 160s)

Slide 65

Slide 65 text

https://swift.org/blog/whole-module-optimizations/

Slide 66

Slide 66 text

Time Function Bodies

Slide 67

Slide 67 text

Slide 68

Slide 68 text

Carthage + github "ReactiveX/RxSwift" ~> 3.4 + github "RxSwiftCommunity/RxRealm" + github "realm/realm-cocoa" ~> 2.4 + github "danielgindi/Charts" ~> 3.0.2 - pod 'RxSwift', '3.3.1' - pod 'RxCocoa', '3.3.1' - pod 'RxRealm', '0.5.2' - pod 'RealmSwift', '2.4.4' Cartﬁle Podﬁle

Slide 69

Slide 69 text

(Local clean build: 68s)

Slide 70

Slide 70 text

Do you really need to build your entire app?

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

Prototyping with Playground • Test your idea in Playground   (interface, design, reproducing a bug, etc.) • Once your idea becomes concrete, move those code into the application project

Slide 73

Slide 73 text

Playground

Slide 74

Slide 74 text

Playground

Slide 75

Slide 75 text

You need “existing” code   for prototyping class ViewController: UITableViewController { let myView = MyView() override func viewDidLoad() { super.viewDidLoad() view.backgroundColor = .myColor view.addSubview(myView) } }

Slide 76

Slide 76 text

Code Modularization import UIKit @testable import YourUIKit let color = UIColor.myGreen

Slide 77

Slide 77 text

Code Modularization Feature A Design Feature B Feature C Network Persistence Core

Slide 78

Slide 78 text

Microservice Architecture https://martinfowler.com/articles/microservices.html

Slide 79

Slide 79 text

Beneﬁts • Strict access control • Faster incremental build/indexing • Clear separation

Slide 80

Slide 80 text

Repository Strategy • Single Repository • Multiple Repositories

Slide 81

Slide 81 text

Single repository • Simple workﬂow • Can’t open-source

Slide 82

Slide 82 text

Multiple Repositories • Complex workﬂow • Code Review Process • Easy to open-source

Slide 83

Slide 83 text

Dependency Management • CocoaPods • git submodule • Carthage • git subtree

Slide 84

Slide 84 text

CocoaPods • Easy in use • Black box in nature

Slide 85

Slide 85 text

git submodule • Simple (!= Easy) • Full control • Manual works • Complex workﬂow

Slide 86

Slide 86 text

Carthage • Flexible • Pre-build • Not all libraries support it • ABI stability

Slide 87

Slide 87 text

https://github.com/apple/swift-evolution#development-major-version--swift-50

Slide 88

Slide 88 text

git subtree • An alternative to git submodule • Clones a subproject and merge it into the parent project • `git subtree push` goes through the commits and picking the changes that should go to a subproject

Slide 89

Slide 89 text

git subtree

Slide 90

Slide 90 text

git subtree • Requires no change to workﬂow • Need to learn a new merge strategy • Contributing back to upstream is a bit tricky

Slide 91

Slide 91 text

Lesson Learned • Breaking existing implicit dependencies is hard • Starting from the single repository • Once modules become stable, consider • managing them in separated repositories • using pre-build versions

Slide 92

Slide 92 text

Case Study: Single repo + CocoaPods

Slide 93

Slide 93 text

Directory Structure . "## MainApp $ "## MainApp $ $ "## AppDelegate.swift $ $ "## Info.plist $ $ # ViewController.swift $ "## MainApp.xcodeproj $ # MainAppTests $ "## Info.plist $ # MainAppTests.swift "## Library $ "## Library $ $ "## Info.plist $ $ "## Library.h $ $ # Library.swift $ "## Library.xcodeproj $ # LibraryTests $ "## Info.plist $ # LibraryTests.swift "## MainApp.xcworkspace "## Podfile "## Podfile.lock "## Pods # README.md

Slide 94

Slide 94 text

Podﬁle source 'https://github.com/CocoaPods/Specs.git' use_frameworks! platform :ios, '9.3' workspace 'MainApp.xcworkspace' target 'MainApp' do project 'MainApp/MainApp.xcodeproj' # Dependencies of the MainApp end target 'Library' do project 'Library/Library.xcodeproj' # Dependencies of the Library end

Slide 95

Slide 95 text

Workspace Structure

Slide 96

Slide 96 text

Link it!

Slide 97

Slide 97 text

Find Implicit Dependencies

Slide 98

Slide 98 text

Pitfall

Slide 99

Slide 99 text

dylib Loading Time Total pre-main time: 4.2 seconds (100.0%) dylib loading time: 4.0 seconds (94.4%) rebase/binding time: 93.32 milliseconds (2.1%) ObjC setup time: 44.64 milliseconds (1.0%) initializer time: 98.30 milliseconds (2.3%)

Slide 100

Slide 100 text

Pitfall

Slide 101

Slide 101 text

Xcode 9 supports Swift Static Library https://download.developer.apple.com/Developer_Tools/Xcode_9_beta_4/ Xcode_9_beta_4_Release_Notes.pdf

Slide 102

Slide 102 text

Recap • Automate daily workﬂows with CI and Fastlane • Reduce workload of code review with Danger • Visualize code metrics you want to improve • Revising project settings may improve build time dramatically • Code modularization improves build/index time and helps you getting well structured code/teams

Slide 103

Slide 103 text

The presentation is available on  https://speakerdeck.com/yuseinishiyama/building-ios-apps-at-scale

Slide 104

Slide 104 text

Thank you! Yusei Nishiyama @yuseinishiyama