Scalable Continuous Integration for iOS

Swift Heroes 2024 talk.

https://swiftheroes.com/2024/speakers/

Transcript

1. Swift Heroes 2024 Scalable Continuous Integration for iOS Alberto De Bortoli
2. 🙋 Alberto De Bortoli 🇮🇹 Italian DNA 🇬🇧 Living in London 👨💻 IC at JET 9+ years 📱 iOS ♾ DevOps 🍕 JustEatTakeaway.com About me
3. What’s this about? 🧑💻 Let’s implement a Continuous Integration infrastructure in-house! 🤓 using cool tech ⬆ to scale 🏎 for performance 🤩 for fun 📚 to learn
4. Agenda 🤌 What is a Continuous Integration & Delivery system? 🛍 Existing CI/CD solutions 😏 Focus on a setup using AWS EC2 + GitHub Actions 🤖 Implementation using DevOps tools 🤔 Final considerations
5. What is this CI/CD thing?
6. CI/CD system A system that executes tasks on your code to allow integration and deployment of software… …continuously (if you’re lucky)
7. CI/CD system 🚢 Continuous Delivery Deploy the software so it can be used (internally and/or publicly) • Build • Archive • Upload to QA/staging/ production distribution channels 🤖 Continuous Integration Make sure the changes proposed are safe to merge into the default branch • Run validations • Run tests • Generate artifacts
12. Existing Solutions
13. CI/CD solutions Server + Workers
14. CI/CD solutions Server + Workers Server only
15. CI/CD solutions Server + Workers Server only Workers only
16. CI/CD solutions Server + Workers Server only Workers only
17. CI/CD solutions Server + Workers Server only Workers only Apple hardware is required here
18. CI/CD solutions Server + Workers Server only Workers only Apple hardware is required here not here
19. • Ready out-of-the-box • Workers are virtualized • Can take time to update to new Xcode versions • Xcode Cloud special snow fl ake • Appropriate for small-mid apps CI/CD solutions Server + Workers
20. CI/CD solutions Server only • Powerful and fl exible build systems • Usually require a high degree of customisation facebook/buck
21. CI/CD solutions Workers only • On premise • Cloud-hosted
22. Workers solutions On premise 👷 • Painful • Cannot replicate environments • Manual software updates • Hosting • Power supply • Connectivity
23. Workers solutions Cloud ☁ • MacStadium have a huge fl eet of Apple hardware
24. Workers solutions Cloud ☁ • The only cloud player offering bare metal Apple hardware • Offer Intel, M1, M2, M2 Pro mac minis
25. Let’s go!
26. ClickOps vs IaC
27. 👍 IaC Infrastructure as Code • Predictable • Reviewable • Auditable • Point&Click in the UI • Error-prone • Intuitive 👎 ClickOps
28. Setup via ClickOps
29. aws.amazon.com
30. aws.amazon.com
31. aws.amazon.com
32. aws.amazon.com
33. aws.amazon.com
34. aws.amazon.com
41. aws.amazon.com
51. Setup via IaC
52. Hashicorp tools IaC Tool Machine Image Creator
53. AMI definition in Packer
57. AMI creation in Packer AWS credential required export AWS_ACCESS_KEY_ID=“your_access_key"
 export AWS_SECRET_ACCESS_KEY=“your_secret_key" Or set them in ~/.aws/credentials
60. IaC in Terraform
65. Deploying/Decommissioning
66. Deploying/Decommissioning
67. Dedicated hosts via AWS CLI
68. Dedicated hosts via AWS CLI
70. 🔌 Connect to GitHub Actions… need a join token 🤫 Secrets are best saved in Secrets Manager 🕵 Set up permissions to access Secrets Manager via IAM 🏁 Pass the secret ID (ARN) via user data when creating the EC2 instance 🍏 Use AWS ec2-macos-init to handle user data for startup operations
71. Recap
72. Recap Dedicated host EC2 AMI (image) EC2 instance EC2 AMI (image) EC2 instance Dedicated host
73. EC2 instances Recap Dedicated host EC2 AMI (image) EC2 instance EC2 AMI (image) Dedicated hosts
74. Recap … Self-hosted runners
75. Console in GHA
76. Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts
77. Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts
78. Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts
79. Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts
80. Final considerations
81. Taking it further 🤔 • Automate scaling • CloudWatch monitoring • IaC all the things • Optimise your pipelines 🫵
82. Virtualization 🤹 • Leveraging the Apple Virtualization.framework • Max 2 VMs per host • Solutions • Tart.run • Anka on AWS • Lower performance
83. Virtualization 🤹 • Don’t be fooled by virtualization • One team at JET moved from VMs to bare metal cutting PR times from 34 minutes down to 9 minutes. • E.g. 1 machine, 5 devs, 10 PRs per day on average • VMs: ~3h • Bare metal: 1h 30m • The impact is real, run your numbers! • Big app? Not gonna cut it.
84. Costs 💸 • Comparing pricing across solutions is dif fi cult • The more you scale the more you save • Prices vary per AWS region • Smart scaling mac2.metal
 (M1) mac2-m2.metal (M2) mac2- m2pro.metal (M2 Pro) us-east-1 (N. Virginia) $0.65 $0.878 $1.56 eu-west-1 (Ireland) $0.716
85. Costs 💸 • Back-of-the-envelope calculations Orka on AWS Anka on AWS In-house infra In-house infra w/ smart scaling Mac minis on premise 5 instances per month $3,995 $3,348 $2,340 $1,632 🤯 Maintenance pain Virtually none Moderate Initial setup pain,
 joy and satisfaction thereafter High
86. • Company size, team size, codebase size… • Pull requests and CD jobs per hour • Existence of a DevOps team or in-house knowledge • Any desire to skill up? • No solution fi ts all, things evolve over time. Buy vs. Build? 🛠
87. Say hi! 👋 albertodebortoli.com @albertodebo