Micro Modular Architecture with Bazel

iOSDC Japan 2020 09/21 11:30 Track E
「Bazelを利用したMicro Modular Architecture」

Transcript

1. Micro Modular Architecture
   with Bazel
   Ryo Aoyama
   iOSDC Japan 2020
   

   View Slide

2. Ryo Aoyama
   Mercari
   Software Engineer
   Architect iOS Tech Lead
   @ra1028
   @ra1028fe5
   

   View Slide

3. Agenda
   • Drawbacks of Monolith
   • Micro Modular Architecture
   • What is Bazel?
   • The power of Bazel with micro modules
   

   View Slide

4. Drawbacks of Monolith
   

   View Slide

5. Disclaimer:
   Monolithic application is NOT an absolute
   bad thing.
   

   View Slide

6. FeatureA FeatureB FeatureC
   FeatureD FeatureZ
   …
   Networking
   DesignSystem
   Analytics
   Database
   Architecture
   etc…
   Entity
   Monolithic Application
   All features are concentrated
   in one or a few modules.
   

   View Slide

7. Monolithic Application
   Build time increases as the product grows:
   Build time can increase linearly or exponentially.
   

   View Slide

8. Coupling complexity between features:
   Monolithic Application
   Coupling complexity between features
   gradually increases and there is no going back.
   

   View Slide

9. Barriers to evolutionary potential:
   Monolithic Application
   The introduction of new technology affects the entire app
   and is pretty costly.
   And also, it does not have the ability to evolve progressively.
   

   View Slide

10. So we need to tackle more advanced architecture.
    

    View Slide

11. Micro Modular Architecture
    

    View Slide

12. Micro Modular Architecture
    Our coined term showing a highly modularized
    application structure.
    =
    

    View Slide

13. The idea is to split our application into
    hundreds of smaller modules.
    

    View Slide

14. Concepts
    One module has only one screen (aka view controller).
    All the layers needed for the screen are also in the
    same module.
    • Module by feature
    

    View Slide

15. Concepts
    The screens do not depend on each other and work
    independently.
    • Independent between features
    

    View Slide

16. Concepts
    Create one module for just one functionality.
    Larger functionalities consist of a composition of
    these smaller modules.
    • Fine-grained shared functionality modules
    

    View Slide

17. Monolith
    Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Concepts
    

    View Slide

18. Looks great, but will the modules
    depend on each other complexly?
    

    View Slide

19. Modularization Group
    Applications Features
    Libraries
    Domain
    Grouping modules to give order to the way
    they depend on each other.
    

    View Slide

20. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    The modules in this group contain
    an App Delegate, and that's it.
    It might also contain DI to feature
    modules.
    Applications
    

    View Slide

21. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    Features
    Each vertical feature module is
    independent and can only depend
    on modules in the Domain and
    Libraries groups.
    

    View Slide

22. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    Domain (Library)
    Contains modules of functionality
    shared by several feature modules.
    It also allows modules to have
    service-specific information.
    

    View Slide

23. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    Libraries
    Contains modules of generalized
    functionality.
    It cannot have service-specific
    information.
    

    View Slide

24. Modularization Group
    Applications
    Features
    Libraries
    Domain
    Grouping modules and make them
    unidirectional dependencies.
    Circular dependency
    Complexity
    

    View Slide

25. Next, let's stop into Bazel for more advanced
    micro modular architecture.
    

    View Slide

26. What is Bazel?
    

    View Slide

27. Bazel
    • Fast + Reliable + Scalable + Extensible
    • Multiple language support
    • Starlark language for build scripts
    • Remote Caching
    • Open Source https://github.com/bazelbuild/bazel
    

    View Slide

28. Example of Benefits
    My product completes clean build, unit test, and
    land to testers in around 1m 30s in the shortest.
    But it's still too early stage for a sample.
    

    View Slide

29. Example of Benefits
    Pinterest
    https://medium.com/pinterest-engineering/developing-fast-reliable-ios-builds-at-pinterest-part-one-cb1810407b92
    Local Build:
    CI Build:
    Beta Distribution:
    Build Success Rate:
    4m 38s -> 3m 38s, 21% improved
    10m 24s -> 7m 34s, 27% improved
    14m 32s -> 7m 52s, 45% improved
    80% -> 97%-100%
    

    View Slide

30. Tokopedia
    Example of Benefits
    https://medium.com/tokopedia-engineering/how-tokopedia-achieved-1000-faster-ios-build-time-7664b2d8ae5
    Local Clean Build: 56m 31s -> 4m 56s, 91% improved
    

    View Slide

31. LINE
    Example of Benefits
    https://engineering.linecorp.com/en/blog/improving-build-performance-line-ios-bazel
    Minimum:
    Maximum:
    Average:
    28m 24s -> 4m 24s, 85% improved
    35m 25s -> 26m 32s, 25% improved
    30m 58s -> 14m 32s, 53% improved
    

    View Slide

32. Example of Benefits
    Amazingly, these improvements have been brought to
    each individual developer by remote caching.
    

    View Slide

33. Build Scripts
    WORKSPACE Defines external dependencies.
    BUILD Contains build scripts for 1~N modules.
    .bazelrc Configuration file for project-specific options.
    xxx.bzl Defines utility extension for build scripts which
    will be loaded from the BUILD.
    

    View Slide

34. Bazel for iOS
    rules_apple
    rules_swift
    apple_support
    https://github.com/bazelbuild/rules_apple
    https://github.com/bazelbuild/rules_swift
    https://github.com/bazelbuild/apple_support
    

    View Slide

35. Bazel for iOS
    for more complicated modules
    rules_apple_line
    https://github.com/line/rules_apple_line
    

    View Slide

36. How to build
    load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
    http_archive(
    name = "build_bazel_rules_apple",
    sha256 = “….",
    url = “https://github.com/bazelbuild/rules_apple/releases/download/x.x.x/rules_apple.x.x.x.tar.gz”,
    )
    load(
    "@build_bazel_rules_apple//apple:repositories.bzl",
    "apple_rules_dependencies",
    )
    apple_rules_dependencies()
    load(
    "@build_bazel_rules_swift//swift:repositories.bzl",
    "swift_rules_dependencies",
    )
    swift_rules_dependencies()
    load(
    "@build_bazel_apple_support//lib:repositories.bzl",
    "apple_support_dependencies",
    )
    apple_support_dependencies()
    WORKSPACE
    

    View Slide

37. Library
    How to build
    load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")
    swift_library(
    name = "Sample",
    srcs = glob(["Sources/**/*.swift"]),
    data = glob(["Resources/**/*"]),
    visibility = ["//visibility:public"],
    )
    BUILD
    

    View Slide

38. Test
    How to build
    load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")
    load("@build_bazel_rules_apple//apple:ios.bzl", "ios_unit_test")
    swift_library(
    name = "TestLib",
    srcs = glob(["Tests/**/*.swift"]),
    )
    ios_unit_test(
    name = "Test",
    deps = [":TestLib"],
    )
    BUILD
    

    View Slide

39. Application
    How to build
    load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application")
    load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")
    swift_library(
    name = "AppLib",
    srcs = glob(["App/**/*.swift"]),
    )
    ios_application(
    name = "App",
    app_icons = glob(["Assets.xcassets/AppIcon.appiconset/**"]),
    bundle_id = "com.example.app",
    families = [
    "iphone",
    "ipad",
    ],
    infoplists = ["Info.plist"],
    launch_storyboard = "Launch.storyboard",
    minimum_os_version = "14.0",
    deps = [":AppLib"],
    )
    BUILD
    

    View Slide

40. How to build
    $ bazel build //…
    Build all targets
    $ bazel build //package-name:target-name
    Build specified target
    

    View Slide

41. Carthage
    CocoaPods
    • pinterest/PodToBUILD
    • bazel-ios/cocoapods-bazel
    • with build scripts: swift_library, objc_library
    • with build scripts: apple_dynamic_framework_import
    Our Choice
    Third-party Library
    

    View Slide

42. Bazel + Xcode
    BUILD xcodeproj
    
    

    View Slide

43. Tulsi by Google
    https://github.com/bazelbuild/tulsi
    XCHammer by Pinterest
    https://github.com/pinterest/xchammer
    Bazel + Xcode
    

    View Slide

44. index-import by Lyft
    https://github.com/lyft/index-import
    Bazel + Xcode
    Replaces the paths in the index file created by Bazel
    so that it can be used for Xcode.
    

    View Slide

45. Downside
    • High adoption cost
    • High learning cost
    • iOS is not officially supported
    • Some Xcode features have been disabled
    

    View Slide

46. Downside
    Wrong setting will rather increase the build time.
    

    View Slide

47. Alternatives
    Buck by Facebook
    https://github.com/facebook/buck
    Gradle
    https://github.com/gradle/gradle
    

    View Slide

48. The power of Bazel with micro modules
    

    View Slide

49. Aggressive Caching
    Once a module is built, it won't be rebuilt
    until the code in that module is changed.
    

    View Slide

50. Aggressive Caching
    Caches are created per module.
    Micro modular architecture maximizes cache efficiency.
    =
    

    View Slide

51. Test Results Caching
    Bazel will not run already succeeded tests.
    

    View Slide

52. Test Results Caching
    That’s can be ignored by
    bazel test //… --nocache_test_results
    

    View Slide

53. Remote Caching
    Once it's built somewhere, all developers
    don't have to rebuild it.
    Build once, reuse everywhere:
    

    View Slide

54. Remote Caching
    Remote
    Cache
    Client
    CI
    Client
    Client
    Cache
    Cache
    Push
    

    View Slide

55. Remote Caching
    Caches can be minimized by
    bazel build //… ——remote_download_minimal
    

    View Slide

56. Remote Caching
    Now, all we need is ample network bandwidth
    instead of Mac Pro.
    

    View Slide

57. Static Linking
    Too many dynamic frameworks can have a negative
    impact on the launch speed of your app.
    

    View Slide

58. Static Linking
    Bazel compiles all modules into static libraries,
    so micro modules has no negative impacts.
    

    View Slide

59. Dependency Diagrams
    Bazel can track all dependency graphs and parse them
    with the bazel query command.
    It can be output as a graphical diagram with Graphviz.
    

    View Slide

60. Dependency Diagrams
    https://github.com/kastiglione/bazel-xcode-demo-swift-driver
    

    View Slide

61. Conclusion
    

    View Slide

62. • Micro Modular Architecture maximizes the synergy
    with Bazel, and vice versa.
    Conclusion
    • Build speed has room for improvement in software
    architecture as well as hardware performance.
    • Bazel will dramatically improve our developer
    experience.
    

    View Slide

63. References
    • Bazel Document
    • Faster Together: Uber Engineering’s iOS Monorepo
    • How Tokopedia Achieved 1000% Faster iOS Build Time
    • Improving Build Performance of LINE for iOS with Bazel
    • Developing fast & reliable iOS builds at Pinterest (Part one)
    • kastiglione/bazel-xcode-demo-swift-driver
    https://eng.uber.com/ios-monorepo
    https://engineering.linecorp.com/en/blog/improving-build-performance-line-ios-bazel
    https://medium.com/tokopedia-engineering/how-tokopedia-achieved-1000-faster-ios-build-time-7664b2d8ae5
    https://medium.com/pinterest-engineering/developing-fast-reliable-ios-builds-at-pinterest-part-one-cb1810407b92
    https://bazel.build
    https://github.com/kastiglione/bazel-xcode-demo-swift-driver
    

    View Slide

64. Thank you
    Ryo Aoyama
    iOSDC Japan 2020
    

    View Slide