The intrepid adventure of scaling a mobile app

Slides from the talk that I gave in T3chfest 2018 about my experience helping to scale mobile apps codebases

Transcript

1. The intrepid adventure of scaling a mobile app @pepibumur 1
2. About me — Production Engineer at Shopify. — Building tools for mobile developers. — Born and raised in the ❤ Murcia. — Suffering the ☔ in Berlin. — # ppinera.es — $ @pepibumur — % pedro.pinera@shopify.com 2
3. Scaling a mobile codebase "Scalability is the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged to accommodate that growth" - Wikipedia @pepibumur 3
4. The codebase doesn't scale when... — 20 engineers produce the same as 10 — You spend more time fighting the tools — You can prepare & drink a coffee while CI runs — Random issues arise in a daily basis — There are conflicts very often @pepibumur 4
5. Why? Apple and Google don't optimise the tools for large codebases @pepibumur 5
6. If you are not experimenting scalability issues, they will come up sooner or later (don't build for scale, think for scale) @pepibumur 6
7. Changing environment — Larger products — More APIs — More platforms to support — New programming languages — People joining/leaving the team @pepibumur 7
8. Scaling mobile codebases is all about challenging standards and tools that we are given @pepibumur 8
9. ! SoundCloud — Compilation times. — Inconsistencies in settings. ! Shopify — Multiple mobile apps. — Historically no shared code. — Compilation times. @pepibumur 9
10. Scalability areas 1. Code 2. UI 3. Tools/Projects 4. Teams @pepibumur 10
11. 1. Code @pepibumur 11
12. Key points 1. Make features independent (from UI to Data) — Reduce friction 2. Reduce dependencies between features (atomicity) — Allow autonomy 3. Share as much code as possible — Save time @pepibumur 12
13. Storing data — Collections ➡ SQLite / Core Data / Realm — Secure ➡ Keychain / AccountManager — Other ➡ UserDefaults / SharedPreferences @pepibumur 13
14. Stores contain shared state leads to unpredictable state changes @pepibumur 14
15. Example — Feature A: Deletes a token from the Keychain — Feature B: Expects the token to be there to work. — Result: Feature B behaving unexpectedly @pepibumur 15
16. Make your stores observable And let other features know about the state changes SecureStore.instance.observe(.token) { [weak self] token in if token == nil { self?.deleteData() } } @pepibumur 16
17. If you serialize/save content into disk. Control how the disk space is used @pepibumur 17
18. Example — Feature A: Saves /Documents/orders.json. — Feature B: Cleans the folder /Documents. — Feature A: Tries to access /Documents/orders.json. — Result: Unexpected behaviour from feature A @pepibumur 18
19. Be!er approach // A store to save orders related data let store = StoreManager.instance.store(for: .orders) // We interact with our own disk space try! store.save(json, name: "orders.json") try! store.cleanAll() store.observe("orders.json") { orders in // Orders changed } @pepibumur 19
20. Relational databases — Observing data. — Accessing relationships. — Performance. — ORMs for free. @pepibumur 20
21. SQL creates implicit dependencies at the data layer Unpredictable behaviours (bugs) Dependencies between teams @pepibumur 21
22. Relationships class Product: NSManagedObject {} class Order: NSManagedObject { @dynamic var products: [Product] } class Client: NSManagedObject { @dynamic var orders: [Order] } — Can we safely delete an order? — Do I have to update the order delivery address if the client address changes? — Do I have to update an order if a product gets removed? @pepibumur 22
23. Remove any kind of implicitness @pepibumur 23
24. Explicit APIs — If you use SQL, don't use relationships — Aim for APIs that other features can consume. let order = ordersRepository.new(clientId: clientId) let token = ordersRepository.observe(clientId: clientId) { orders in // Client orders changed } @pepibumur 24
25. Maybe you don't need a SQL store ! . Just serialize objects into disk @pepibumur 25
26. Leverage modules and access levels to decouple your code defining public interfaces @pepibumur 26
27. // Module Core public class Client { public fund execute(request: Request) -> Response private fund privateMethod() { /* some code */ } } // Module App class ProfileInteractor { let client: Client func sync() { client.privateMethod() // Compiler error! let response = client.execute(request: User.me) // All right } } @pepibumur 27
28. As part of your implementations. You should design APIs @pepibumur 28
29. Default access levels — Kotlin: public ! — Swift: internal " @pepibumur 29
30. Other programming paradigms might have benefits, but its usage in large teams is questionable @pepibumur 30
31. Use case: Reactive programming — Everyone in your team needs to learn it. — They need to learn it properly. — You'll most likely rely on a third party dependency (RxSwift, RxJava...) @pepibumur 31
32. Abstractions only if they are really necessary Prefer pa!erns from the system frameworks @pepibumur 32
33. Because... — !" know about the system patterns. — # You get updates/improvements from the platform. — $ Building an abstraction that scale is hard in practice. — % The fewer dependencies, the better. @pepibumur 33
34. Minimize dependencies — Dependencies save time initially, but need to be maintained. — Do you need dependency X: — Yes: Use it but with an abstraction layer. — No: Then don't use it or build just what you need. @pepibumur 34
35. 2. UI @pepibumur 35
36. Composition over extension @pepibumur 36
37. 1. I start defining my layout 2. I add this button here. 3. I add this list of images there. 4. I introduce this new button between these two elements 5. ... 6. ... 7. ... 8. No one can modify the XML/XIB/storyboard anymore @pepibumur 37
38. Use your intuition if the layout is growing considerably then split it in small layouts @pepibumur 38
39. Composition means Reusability Easy maintenance @pepibumur 39
40. 3. Tools/Projects @pepibumur 40
41. Automate the automatable Code linting Project sanity checks (danger.systems) App releases @pepibumur 41
42. Don't spend your developers' time There are great tools already built to help you with automation (e.g. Fastlane) @pepibumur 42
43. Build times The more code you have, the more it takes to compile (unless you use React Native) @pepibumur 43
44. Gradle & Xcode Build system build incrementally Until you clean the build ! And I'm sure it's the first thing you do when your project doesn't compile. @pepibumur 44
45. Let's do some maths — Clean build: 10 minutes — Cleans per day per developer: 10 — Developers in the team: 30 Build time spent per month - 1000 hours - 6.25 developers - 21% of your team @pepibumur 45
46. Make your app modular — Cleaning should only affect the current module. — Developers should only compile the module they are working on (and its dependencies). — Reference: Microfeatures @pepibumur 46
47. You can go further And try other build systems that build incrementally across developers in a team Buck from Facebook Bazel from Google @pepibumur 47
48. If clean builds are slow... So they are on CI Continuous integration services/platforms Build from a clean environment @pepibumur 48
49. Can we speed up clean builds on CI? — Selective build/run tests based on the files that were modified. — Parallelize work. — Cache build artifacts across builds (e.g. Pods, Bundler dependencies) — Use Buck/Bazel. @pepibumur 49
50. Measure ! Before optimising, we need to have some values to compare with @pepibumur 50
51. What to measure? ! — Compilation time of each of the modules. — App startup time. — Flaky tests consistently failing. — Failing builds and the reason. @pepibumur 51
52. Build a dashboard With all the project metrics grafana.com @pepibumur 52
53. Reproducible builds "My build is failing on CI but it doesn't fail locally" @pepibumur 53
54. How to make your builds reproducible — Avoid custom logic that only runs on CI. — Pin your dependencies versions. — Bundle your dependencies into the repository and avoid access the shared system environment. — Use Nix @pepibumur 54
55. 4. Teams @pepibumur 55
56. If a team grows, it gets spli!ed into smaller teams @pepibumur 56
57. Feature teams / Squads ! They are independent. " They might duplicate code. " Patterns/style might diverge a lot. @pepibumur 57
58. Be consistent — Build settings. — Tools. — Processes. — Patterns. — Programming languages. @pepibumur 58
59. Mobops team !" — Has an overview of the project as a whole. — Identifies patterns across teams and extracts reusable components. — Builds tools to make teams productive. — Provides teams with communication tools/channels. — Removes any friction between teams making them as atomic as possible. @pepibumur 59
60. Collective meetings Where all the iOS/Android developers from different teams come together to talk about the vision of the project @pepibumur 60
61. Conclusions @pepibumur 61
62. — If you are not facing the challenges I talked about, you'll do sooner or later. — Don't let yourself be influenced by hype technologies. — Automate as much as you can. — Don't expect Apple/Google to optimise the tools for scale. — An awesome product is the result of motivated developers. @pepibumur 62
63. Thanks Questions? Slides: h!ps://goo.gl/2dbsej @pepibumur 63