2021: CI for Mobile: State of The Art

Transcript

1. 2021: CI for Mobile - State of the Art - 1
2. Plan 1. The role of the CI 2. Key principles 3. Simple Workflow 4. On-Premise 5. IaaS 6. SaaS 7. Best Practices 8. Static Analysis 9. Metrics 2
4. The role of the CI 1. Lints and performs static analysis 2. Tests (Unit, Integration) 3. Calculates and publishes metrics 4. Interacts with the repository (publishes status, ...) 5. Interacts with the team (Slack, mail, ...) 6. Builds 7. Prepares and/or sends artifacts for the CD 4
5. Key principles for a good CI — Automated — Idempotent — Isolation — Fast — Secure — Observable 5
6. Why automating? 1. Avoid human errors 2. Enforces the workflow steps (tests, linting, etc) 3. Frozen configuration 4. Security: avoids sharing admin credentials 5. Time saving: CI runs while we perform other tasks 6
12. 2 Categories 1. On Premise (Self Hosted) 2. Cloud (IaaS or SaaS) 3. (Mix of the two above) 12
13. On Premise - (Self Hosted) - 13
15. Configuration — Main Only — Main + Agent / Runner 15
16. Configuration — Main Only — Main + Agent / Runner 16
17. Configuration — Jobs described via scripting (Groovy in Jenkins / YML + Bash in GitLab CI) — Environment variables managed in the Jenkins / GitLab CI console — Secrets managed in the Jenkins console or other tools 17
18. Jenkinsfile node { def server = Artifactory.server "SERVER_ID" def buildInfo stage('Clone sources') { git url: 'https://github.com/jfrogdev/project-examples.git' } stage('Gradle build') { buildInfo = rtGradle.run rootDir: "gradle-examples/4/gradle-example-ci-server/", buildFile: 'build.gradle', tasks: 'clean artifactoryPublish' } stage('Publish build info') { server.publishBuildInfo buildInfo } } 18
19. .gitlab-ci.yml build-job: stage: build script: - echo "Hello, $GITLAB_USER_LOGIN!" test-job1: stage: test script: - echo "This job tests something" test-job2: stage: test script: - echo "This job tests something, but takes more time than test-job1." - sleep 20 deploy-prod: stage: deploy script: - echo "This job deploys something from the $CI_COMMIT_BRANCH branch." 19
20. Main Advantages 1. Fast 2. Machines are in an internal & protected network 3. Fully configurable (supports custom Swift toolchains, special tooling such as mitm-proxy, ...) 20
21. Main Disadvantages 1. Manual management of project configuration (certificates, keystore, secrets, ...) 2. Manual management of the machine (hardware, OS versions, tool versions, security patches, ...) 3. Parallel execution is hard 4. Isolation is hard 21
22. Some solutions — Android: Docker — iOS: Fastlane Match (Certificate management) — iOS: xcversion for easily install Xcode versions — iOS: macOS Virtualization (e.g. VMware ESXi, Orka, Veertu) 22
23. + New Anka "Develop" Free version 23
24. Cloud 24
25. 2 Solutions: IaaS OR SaaS 25
26. IaaS 26
27. IaaS — AWS EC2 — iOS: MacStadium 27
28. Key Concepts — Physical or virtual machines managed by providers — Choose between a range of available OSs — Tooling not pre-installed — Access remotely for configuration — Pay per hour/day/month of usage — Pricing may vary (from 5$/month (EC2) to ∞€) 28
29. Advantages — Easy OS provisioning — Generally fast (bare metal) Disadvantages — Manual management of project configuration (certificates, keystore, secrets, ...) — Less hardware choice than On-Premise 29
30. Focus on macOS on EC2 — Dedicated instance: Mac Mini bare metal — Core i7 3.2Ghz - 32GB RAM — minimum period of 24 hours — $1.083 per hour => 779$ / month — Fully integrated with AWS infrastructure (VPC, AZ, etc) — macOS 10.14 and 10.15, M1 coming soon 30
31. SaaS 31
32. SaaS — Bitrise — Circle CI — Visual Studio App Center — GitHub Actions 32
33. Key Concepts 1. Mobile-Oriented 2. Almost anything is pre-installed 3. Perfect isolation & good security: One job = one VM Virtual machines are spawned for every new job and killed when the job ends 4. Project configuration can be administered via dashboards / API / code 5. (In some cases) workflow can be edited from within the solution 33
35. .github/workflows/main.yml name: CI on: pull_request: branches: [ master ] jobs: build: runs-on: macos-11.0 steps: - uses: actions/checkout@v2 - name: Run a one-line script run: echo Hello, world! 35
36. Advantages — Fully managed — Quite easy to setup — Easy to switch between configurations 36
37. Disadvantages — Usually Slow — Risk of Vendor lock-in — Harder to debug the build process (kind of a black box) 37
38. Configuration Best Practices 38
39. Best Practices Keep your CI code readable 39
40. Best Practices Rely on standalone automation tools rather than on proprietary CI steps (e.g. use fastlane or Gradle) 40
41. Rely on standalone automation tools rather than on proprietary CI steps — Reduces risk of vendor lock-in — Allows local debugging of the CI steps — Does not require specific CI knowledge 41
43. Best Practices Commit the dependency versions you're using 43
44. Commit the dependency version you're using Use dependency managers when possible and commit the version files. — For Ruby, use rbenv (or rvm) + Bundler and don't forget to commit — .ruby-version — Gemfile and Gemfile.lock — With JavaScript, use npm (or yarn) and commit the node_modules folder (or at least package-lock.json) 44
45. Commit the dependency version you're using — On Android, commit — gradlew — gradle-wrapper.properties — gradle-wrapper.jar — On iOS, commit — Podfile AND Podfile.lock (or, better, the whole Pods directory) — Package.resolved 45
46. Best Practices Commit the CI steps and configuration alongside your project 46
47. Commit the CI steps and configuration alongside your project — Commit Jenkinsfile, bitrise.yml, etc in your project repo — allows for coherence between CI and project structure — reduces risk of breaking changes 47
48. Best Practices Retrieve secrets from a secondary repo or service (vault), accessible from the CI only 48
49. Best Practices No "one size fits all". If you're dealing with a huge project use the appropriate tools. 49
50. If you're dealing with a huge project use the appropriate tools If building a huge multi-module project, favor: — monorepo configurations (when possible) — distributed building tools such as: — Bazel — Buck 50
51. Focus on Static Analysis 51
52. Static Analysis Performs automated static checks on your code, such as: — Linting — via SwiftLint (or Taylor) — gradle lint — Changelog verification — Commit message checks — Suspicious file modifications — ... 52
53. Static Analysis A good practice involves publishing the static analysis result to the Pull Request page. A number of off-the-shelf solutions exist: — GitLab Push Rules (GitLab only) — Hound — Danger 53
54. Hound (Based on SwiftLint) 54
55. Danger 55
56. Danger Examples # Ensure a clean commits history if git.commits.any? { |c| c.message =~ /^Merge branch/ } fail('Please rebase to get rid of the merge commits in this PR') end # Don't let testing shortcuts get into master by accident fail("fit left in tests") if `grep -r "fit" Demo/Tests/`.length > 1 56
57. Danger Examples # Did you make analytics changes? # Please also include a change to our analytics spec made_analytics_changes = modified_files.include?("AppDelegate+Analytics.swift") made_analytics_specs_changes = modified_files.include?("AppAnalyticsSpec.swift") if made_analytics_changes && !made_analytics_specs_changes fail("Analytics changes should have reflected specs changes") end 57
58. Bonus: SwiftInfo 58
59. Focus on Metrics 59
60. Which metrics? — Lines of Code — Coverage — Cyclomatic complexity — Duplication — Documentation (when needed) — Code smells (cf Static Analysis) 60
61. How to track them? — SonarQube — Codecov — Code Climate 61
63. The key to a successful DevOps practice on mobile projects 63
64. The keys to a successful practice on mobile projects — No CI-people — Shared knowledge — You build it, you run it — CI as readable code — We are developers — Code is documentation — Don't underestimate the time spent on CI: — time is $$: a simple yet expensive setup can be much cheaper than a complex one 64
65. Thank you! 65
66. Credits — https://about.gitlab.com/stages-devops-lifecycle/ continuous-integration/ — https://www.ingloriousmind.com/blog/continuous- integration-using-bitrise/ 66