Tweet
Tweet

Slide 1

Slide 1 text

Swift Heroes 2024 Scalable Continuous Integration for iOS Alberto De Bortoli

Slide 2

Slide 2 text

🙋 Alberto De Bortoli 🇮🇹 Italian DNA 🇬🇧 Living in London 👨💻 IC at JET 9+ years 📱 iOS ♾ DevOps 🍕 JustEatTakeaway.com About me

Slide 3

Slide 3 text

What’s this about? 🧑💻 Let’s implement a Continuous Integration infrastructure in-house! 🤓 using cool tech ⬆ to scale 🏎 for performance 🤩 for fun 📚 to learn

Slide 4

Slide 4 text

Agenda 🤌 What is a Continuous Integration & Delivery system? 🛍 Existing CI/CD solutions 😏 Focus on a setup using AWS EC2 + GitHub Actions 🤖 Implementation using DevOps tools 🤔 Final considerations

Slide 5

Slide 5 text

What is this CI/CD thing?

Slide 6

Slide 6 text

CI/CD system A system that executes tasks on your code to allow integration and deployment of software… …continuously (if you’re lucky)

Slide 7

Slide 7 text

CI/CD system 🚢 Continuous Delivery Deploy the software so it can be used (internally and/or publicly) • Build • Archive • Upload to QA/staging/ production distribution channels 🤖 Continuous Integration Make sure the changes proposed are safe to merge into the default branch • Run validations • Run tests • Generate artifacts

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Existing Solutions

Slide 13

Slide 13 text

CI/CD solutions Server + Workers

Slide 14

Slide 14 text

CI/CD solutions Server + Workers Server only

Slide 15

Slide 15 text

CI/CD solutions Server + Workers Server only Workers only

Slide 16

Slide 16 text

CI/CD solutions Server + Workers Server only Workers only

Slide 17

Slide 17 text

CI/CD solutions Server + Workers Server only Workers only Apple hardware is required here

Slide 18

Slide 18 text

CI/CD solutions Server + Workers Server only Workers only Apple hardware is required here not here

Slide 19

Slide 19 text

• Ready out-of-the-box • Workers are virtualized • Can take time to update to new Xcode versions • Xcode Cloud special snow fl ake • Appropriate for small-mid apps CI/CD solutions Server + Workers

Slide 20

Slide 20 text

CI/CD solutions Server only • Powerful and fl exible build systems • Usually require a high degree of customisation facebook/buck

Slide 21

Slide 21 text

CI/CD solutions Workers only • On premise • Cloud-hosted

Slide 22

Slide 22 text

Workers solutions On premise 👷 • Painful • Cannot replicate environments • Manual software updates • Hosting • Power supply • Connectivity

Slide 23

Slide 23 text

Workers solutions Cloud ☁ • MacStadium have a huge fl eet of Apple hardware

Slide 24

Slide 24 text

Workers solutions Cloud ☁ • The only cloud player offering bare metal Apple hardware • Offer Intel, M1, M2, M2 Pro mac minis

Slide 25

Slide 25 text

Let’s go!

Slide 26

Slide 26 text

ClickOps vs IaC

Slide 27

Slide 27 text

👍 IaC Infrastructure as Code • Predictable • Reviewable • Auditable • Point&Click in the UI • Error-prone • Intuitive 👎 ClickOps

Slide 28

Slide 28 text

Setup via ClickOps

Slide 29

Slide 29 text

aws.amazon.com

Slide 30

Slide 30 text

aws.amazon.com

Slide 31

Slide 31 text

aws.amazon.com

Slide 32

Slide 32 text

aws.amazon.com

Slide 33

Slide 33 text

aws.amazon.com

Slide 34

Slide 34 text

aws.amazon.com

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

aws.amazon.com

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Setup via IaC

Slide 52

Slide 52 text

Hashicorp tools IaC Tool Machine Image Creator

Slide 53

Slide 53 text

AMI definition in Packer

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

AMI creation in Packer AWS credential required export AWS_ACCESS_KEY_ID=“your_access_key"
 export AWS_SECRET_ACCESS_KEY=“your_secret_key" Or set them in ~/.aws/credentials

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

IaC in Terraform

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

No content

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

Deploying/Decommissioning

Slide 66

Slide 66 text

Deploying/Decommissioning

Slide 67

Slide 67 text

Dedicated hosts via AWS CLI

Slide 68

Slide 68 text

Dedicated hosts via AWS CLI

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

🔌 Connect to GitHub Actions… need a join token 🤫 Secrets are best saved in Secrets Manager 🕵 Set up permissions to access Secrets Manager via IAM 🏁 Pass the secret ID (ARN) via user data when creating the EC2 instance 🍏 Use AWS ec2-macos-init to handle user data for startup operations

Slide 71

Slide 71 text

Recap

Slide 72

Slide 72 text

Recap Dedicated host EC2 AMI (image) EC2 instance EC2 AMI (image) EC2 instance Dedicated host

Slide 73

Slide 73 text

EC2 instances Recap Dedicated host EC2 AMI (image) EC2 instance EC2 AMI (image) Dedicated hosts

Slide 74

Slide 74 text

Recap … Self-hosted runners

Slide 75

Slide 75 text

Console in GHA

Slide 76

Slide 76 text

Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts

Slide 77

Slide 77 text

Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts

Slide 78

Slide 78 text

Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts

Slide 79

Slide 79 text

Console in GHA The operations we want to automate: 1. The creation of the AMI 2. The deployment and decommission of the infrastructure 3. The allocation/release of the dedicated hosts

Slide 80

Slide 80 text

Final considerations

Slide 81

Slide 81 text

Taking it further 🤔 • Automate scaling • CloudWatch monitoring • IaC all the things • Optimise your pipelines 🫵

Slide 82

Slide 82 text

Virtualization 🤹 • Leveraging the Apple Virtualization.framework • Max 2 VMs per host • Solutions • Tart.run • Anka on AWS • Lower performance

Slide 83

Slide 83 text

Virtualization 🤹 • Don’t be fooled by virtualization • One team at JET moved from VMs to bare metal cutting PR times from 34 minutes down to 9 minutes. • E.g. 1 machine, 5 devs, 10 PRs per day on average • VMs: ~3h • Bare metal: 1h 30m • The impact is real, run your numbers! • Big app? Not gonna cut it.

Slide 84

Slide 84 text

Costs 💸 • Comparing pricing across solutions is dif fi cult • The more you scale the more you save • Prices vary per AWS region • Smart scaling mac2.metal
 (M1) mac2-m2.metal (M2) mac2- m2pro.metal (M2 Pro) us-east-1 (N. Virginia) $0.65 $0.878 $1.56 eu-west-1 (Ireland) $0.716

Slide 85

Slide 85 text

Costs 💸 • Back-of-the-envelope calculations Orka on AWS Anka on AWS In-house infra In-house infra w/ smart scaling Mac minis on premise 5 instances per month $3,995 $3,348 $2,340 $1,632 🤯 Maintenance pain Virtually none Moderate Initial setup pain,
 joy and satisfaction thereafter High

Slide 86

Slide 86 text

• Company size, team size, codebase size… • Pull requests and CD jobs per hour • Existence of a DevOps team or in-house knowledge • Any desire to skill up? • No solution fi ts all, things evolve over time. Buy vs. Build? 🛠

Slide 87

Slide 87 text

Say hi! 👋 albertodebortoli.com @albertodebo