2021: CI for Mobile: State of The Art

Slide 1

Slide 1 text

2021: CI for Mobile - State of the Art - 1

Slide 2

Slide 2 text

Plan 1. The role of the CI 2. Key principles 3. Simple Workﬂow 4. On-Premise 5. IaaS 6. SaaS 7. Best Practices 8. Static Analysis 9. Metrics 2

Slide 3

Slide 3 text

Slide 4

Slide 4 text

The role of the CI 1. Lints and performs static analysis 2. Tests (Unit, Integration) 3. Calculates and publishes metrics 4. Interacts with the repository (publishes status, ...) 5. Interacts with the team (Slack, mail, ...) 6. Builds 7. Prepares and/or sends artifacts for the CD 4

Slide 5

Slide 5 text

Key principles for a good CI — Automated — Idempotent — Isolation — Fast — Secure — Observable 5

Slide 6

Slide 6 text

Why automating? 1. Avoid human errors 2. Enforces the workﬂow steps (tests, linting, etc) 3. Frozen conﬁguration 4. Security: avoids sharing admin credentials 5. Time saving: CI runs while we perform other tasks 6

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

2 Categories 1. On Premise (Self Hosted) 2. Cloud (IaaS or SaaS) 3. (Mix of the two above) 12

Slide 13

Slide 13 text

On Premise - (Self Hosted) - 13

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Configuration — Main Only — Main + Agent / Runner 15

Slide 16

Slide 16 text

Configuration — Main Only — Main + Agent / Runner 16

Slide 17

Slide 17 text

Configuration — Jobs described via scripting (Groovy in Jenkins / YML + Bash in GitLab CI) — Environment variables managed in the Jenkins / GitLab CI console — Secrets managed in the Jenkins console or other tools 17

Slide 18

Slide 18 text

Jenkinsfile node { def server = Artifactory.server "SERVER_ID" def buildInfo stage('Clone sources') { git url: 'https://github.com/jfrogdev/project-examples.git' } stage('Gradle build') { buildInfo = rtGradle.run rootDir: "gradle-examples/4/gradle-example-ci-server/", buildFile: 'build.gradle', tasks: 'clean artifactoryPublish' } stage('Publish build info') { server.publishBuildInfo buildInfo } } 18

Slide 19

Slide 19 text

.gitlab-ci.yml build-job: stage: build script: - echo "Hello, $GITLAB_USER_LOGIN!" test-job1: stage: test script: - echo "This job tests something" test-job2: stage: test script: - echo "This job tests something, but takes more time than test-job1." - sleep 20 deploy-prod: stage: deploy script: - echo "This job deploys something from the $CI_COMMIT_BRANCH branch." 19

Slide 20

Slide 20 text

Main Advantages 1. Fast 2. Machines are in an internal & protected network 3. Fully conﬁgurable (supports custom Swift toolchains, special tooling such as mitm-proxy, ...) 20

Slide 21

Slide 21 text

Main Disadvantages 1. Manual management of project conﬁguration (certiﬁcates, keystore, secrets, ...) 2. Manual management of the machine (hardware, OS versions, tool versions, security patches, ...) 3. Parallel execution is hard 4. Isolation is hard 21

Slide 22

Slide 22 text

Some solutions — Android: Docker — iOS: Fastlane Match (Certiﬁcate management) — iOS: xcversion for easily install Xcode versions — iOS: macOS Virtualization (e.g. VMware ESXi, Orka, Veertu) 22

Slide 23

Slide 23 text

+ New Anka "Develop" Free version 23

Slide 24

Slide 24 text

Cloud 24

Slide 25

Slide 25 text

2 Solutions: IaaS OR SaaS 25

Slide 26

Slide 26 text

IaaS 26

Slide 27

Slide 27 text

IaaS — AWS EC2 — iOS: MacStadium 27

Slide 28

Slide 28 text

Key Concepts — Physical or virtual machines managed by providers — Choose between a range of available OSs — Tooling not pre-installed — Access remotely for conﬁguration — Pay per hour/day/month of usage — Pricing may vary (from 5$/month (EC2) to ∞€) 28

Slide 29

Slide 29 text

Advantages — Easy OS provisioning — Generally fast (bare metal) Disadvantages — Manual management of project conﬁguration (certiﬁcates, keystore, secrets, ...) — Less hardware choice than On-Premise 29

Slide 30

Slide 30 text

Focus on macOS on EC2 — Dedicated instance: Mac Mini bare metal — Core i7 3.2Ghz - 32GB RAM — minimum period of 24 hours — $1.083 per hour => 779$ / month — Fully integrated with AWS infrastructure (VPC, AZ, etc) — macOS 10.14 and 10.15, M1 coming soon 30

Slide 31

Slide 31 text

SaaS 31

Slide 32

Slide 32 text

SaaS — Bitrise — Circle CI — Visual Studio App Center — GitHub Actions 32

Slide 33

Slide 33 text

Key Concepts 1. Mobile-Oriented 2. Almost anything is pre-installed 3. Perfect isolation & good security: One job = one VM Virtual machines are spawned for every new job and killed when the job ends 4. Project conﬁguration can be administered via dashboards / API / code 5. (In some cases) workﬂow can be edited from within the solution 33

Slide 34

Slide 34 text

Slide 35

Slide 35 text

.github/workflows/main.yml name: CI on: pull_request: branches: [ master ] jobs: build: runs-on: macos-11.0 steps: - uses: actions/checkout@v2 - name: Run a one-line script run: echo Hello, world! 35

Slide 36

Slide 36 text

Advantages — Fully managed — Quite easy to setup — Easy to switch between conﬁgurations 36

Slide 37

Slide 37 text

Disadvantages — Usually Slow — Risk of Vendor lock-in — Harder to debug the build process (kind of a black box) 37

Slide 38

Slide 38 text

Configuration Best Practices 38

Slide 39

Slide 39 text

Best Practices Keep your CI code readable 39

Slide 40

Slide 40 text

Best Practices Rely on standalone automation tools rather than on proprietary CI steps (e.g. use fastlane or Gradle) 40

Slide 41

Slide 41 text

Rely on standalone automation tools rather than on proprietary CI steps — Reduces risk of vendor lock-in — Allows local debugging of the CI steps — Does not require speciﬁc CI knowledge 41

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Best Practices Commit the dependency versions you're using 43

Slide 44

Slide 44 text

Commit the dependency version you're using Use dependency managers when possible and commit the version ﬁles. — For Ruby, use rbenv (or rvm) + Bundler and don't forget to commit — .ruby-version — Gemfile and Gemfile.lock — With JavaScript, use npm (or yarn) and commit the node_modules folder (or at least package-lock.json) 44

Slide 45

Slide 45 text

Commit the dependency version you're using — On Android, commit — gradlew — gradle-wrapper.properties — gradle-wrapper.jar — On iOS, commit — Podfile AND Podfile.lock (or, better, the whole Pods directory) — Package.resolved 45

Slide 46

Slide 46 text

Best Practices Commit the CI steps and conﬁguration alongside your project 46

Slide 47

Slide 47 text

Commit the CI steps and configuration alongside your project — Commit Jenkinsfile, bitrise.yml, etc in your project repo — allows for coherence between CI and project structure — reduces risk of breaking changes 47

Slide 48

Slide 48 text

Best Practices Retrieve secrets from a secondary repo or service (vault), accessible from the CI only 48

Slide 49

Slide 49 text

Best Practices No "one size ﬁts all". If you're dealing with a huge project use the appropriate tools. 49

Slide 50

Slide 50 text

If you're dealing with a huge project use the appropriate tools If building a huge multi-module project, favor: — monorepo conﬁgurations (when possible) — distributed building tools such as: — Bazel — Buck 50

Slide 51

Slide 51 text

Focus on Static Analysis 51

Slide 52

Slide 52 text

Static Analysis Performs automated static checks on your code, such as: — Linting — via SwiftLint (or Taylor) — gradle lint — Changelog verification — Commit message checks — Suspicious file modifications — ... 52

Slide 53

Slide 53 text

Static Analysis A good practice involves publishing the static analysis result to the Pull Request page. A number of off-the-shelf solutions exist: — GitLab Push Rules (GitLab only) — Hound — Danger 53

Slide 54

Slide 54 text

Hound (Based on SwiftLint) 54

Slide 55

Slide 55 text

Danger 55

Slide 56

Slide 56 text

Danger Examples # Ensure a clean commits history if git.commits.any? { |c| c.message =~ /^Merge branch/ } fail('Please rebase to get rid of the merge commits in this PR') end # Don't let testing shortcuts get into master by accident fail("fit left in tests") if `grep -r "fit" Demo/Tests/`.length > 1 56

Slide 57

Slide 57 text

Danger Examples # Did you make analytics changes? # Please also include a change to our analytics spec made_analytics_changes = modified_files.include?("AppDelegate+Analytics.swift") made_analytics_specs_changes = modified_files.include?("AppAnalyticsSpec.swift") if made_analytics_changes && !made_analytics_specs_changes fail("Analytics changes should have reflected specs changes") end 57

Slide 58

Slide 58 text

Bonus: SwiftInfo 58

Slide 59

Slide 59 text

Focus on Metrics 59

Slide 60

Slide 60 text

Which metrics? — Lines of Code — Coverage — Cyclomatic complexity — Duplication — Documentation (when needed) — Code smells (cf Static Analysis) 60

Slide 61

Slide 61 text

How to track them? — SonarQube — Codecov — Code Climate 61

Slide 62

Slide 62 text

Slide 63

Slide 63 text

The key to a successful DevOps practice on mobile projects 63

Slide 64

Slide 64 text

The keys to a successful practice on mobile projects — No CI-people — Shared knowledge — You build it, you run it — CI as readable code — We are developers — Code is documentation — Don't underestimate the time spent on CI: — time is $$: a simple yet expensive setup can be much cheaper than a complex one 64

Slide 65

Slide 65 text

Thank you! 65

Slide 66

Slide 66 text

Credits — https://about.gitlab.com/stages-devops-lifecycle/ continuous-integration/ — https://www.ingloriousmind.com/blog/continuous- integration-using-bitrise/ 66