$30 off During Our Annual Pro Sale. View Details »

Micro Modular Architecture with Bazel

Ryo Aoyama
September 21, 2020

Micro Modular Architecture with Bazel

iOSDC Japan 2020 09/21 11:30 Track E
「Bazelを利用したMicro Modular Architecture」

Ryo Aoyama

September 21, 2020
Tweet

More Decks by Ryo Aoyama

Other Decks in Programming

Transcript

  1. Micro Modular Architecture
    with Bazel
    Ryo Aoyama
    iOSDC Japan 2020

    View Slide

  2. Ryo Aoyama
    Mercari
    Software Engineer
    Architect iOS Tech Lead
    @ra1028
    @ra1028fe5

    View Slide

  3. Agenda
    • Drawbacks of Monolith
    • Micro Modular Architecture
    • What is Bazel?
    • The power of Bazel with micro modules

    View Slide

  4. Drawbacks of Monolith

    View Slide

  5. Disclaimer:
    Monolithic application is NOT an absolute
    bad thing.

    View Slide

  6. FeatureA FeatureB FeatureC
    FeatureD FeatureZ

    Networking
    DesignSystem
    Analytics
    Database
    Architecture
    etc…
    Entity
    Monolithic Application
    All features are concentrated
    in one or a few modules.

    View Slide

  7. Monolithic Application
    Build time increases as the product grows:
    Build time can increase linearly or exponentially.

    View Slide

  8. Coupling complexity between features:
    Monolithic Application
    Coupling complexity between features
    gradually increases and there is no going back.

    View Slide

  9. Barriers to evolutionary potential:
    Monolithic Application
    The introduction of new technology affects the entire app
    and is pretty costly.
    And also, it does not have the ability to evolve progressively.

    View Slide

  10. So we need to tackle more advanced architecture.

    View Slide

  11. Micro Modular Architecture

    View Slide

  12. Micro Modular Architecture
    Our coined term showing a highly modularized
    application structure.
    =

    View Slide

  13. The idea is to split our application into
    hundreds of smaller modules.

    View Slide

  14. Concepts
    One module has only one screen (aka view controller).
    All the layers needed for the screen are also in the
    same module.
    • Module by feature

    View Slide

  15. Concepts
    The screens do not depend on each other and work
    independently.
    • Independent between features

    View Slide

  16. Concepts
    Create one module for just one functionality.
    Larger functionalities consist of a composition of
    these smaller modules.
    • Fine-grained shared functionality modules

    View Slide

  17. Monolith
    Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Concepts

    View Slide

  18. Looks great, but will the modules
    depend on each other complexly?

    View Slide

  19. Modularization Group
    Applications Features
    Libraries
    Domain
    Grouping modules to give order to the way
    they depend on each other.

    View Slide

  20. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    The modules in this group contain
    an App Delegate, and that's it.
    It might also contain DI to feature
    modules.
    Applications

    View Slide

  21. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    Features
    Each vertical feature module is
    independent and can only depend
    on modules in the Domain and
    Libraries groups.

    View Slide

  22. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    Domain (Library)
    Contains modules of functionality
    shared by several feature modules.
    It also allows modules to have
    service-specific information.

    View Slide

  23. Application
    Feature
    Feature
    Feature
    Feature
    Feature
    Common Common
    Common Common
    Modularization Group
    Libraries
    Contains modules of generalized
    functionality.
    It cannot have service-specific
    information.

    View Slide

  24. Modularization Group
    Applications
    Features
    Libraries
    Domain
    Grouping modules and make them
    unidirectional dependencies.
    Circular dependency
    Complexity

    View Slide

  25. Next, let's stop into Bazel for more advanced
    micro modular architecture.

    View Slide

  26. What is Bazel?

    View Slide

  27. Bazel
    • Fast + Reliable + Scalable + Extensible
    • Multiple language support
    • Starlark language for build scripts
    • Remote Caching
    • Open Source https://github.com/bazelbuild/bazel

    View Slide

  28. Example of Benefits
    My product completes clean build, unit test, and
    land to testers in around 1m 30s in the shortest.
    But it's still too early stage for a sample.

    View Slide

  29. Example of Benefits
    Pinterest
    https://medium.com/pinterest-engineering/developing-fast-reliable-ios-builds-at-pinterest-part-one-cb1810407b92
    Local Build:
    CI Build:
    Beta Distribution:
    Build Success Rate:
    4m 38s -> 3m 38s, 21% improved
    10m 24s -> 7m 34s, 27% improved
    14m 32s -> 7m 52s, 45% improved
    80% -> 97%-100%

    View Slide

  30. Tokopedia
    Example of Benefits
    https://medium.com/tokopedia-engineering/how-tokopedia-achieved-1000-faster-ios-build-time-7664b2d8ae5
    Local Clean Build: 56m 31s -> 4m 56s, 91% improved

    View Slide

  31. LINE
    Example of Benefits
    https://engineering.linecorp.com/en/blog/improving-build-performance-line-ios-bazel
    Minimum:
    Maximum:
    Average:
    28m 24s -> 4m 24s, 85% improved
    35m 25s -> 26m 32s, 25% improved
    30m 58s -> 14m 32s, 53% improved

    View Slide

  32. Example of Benefits
    Amazingly, these improvements have been brought to
    each individual developer by remote caching.

    View Slide

  33. Build Scripts
    WORKSPACE Defines external dependencies.
    BUILD Contains build scripts for 1~N modules.
    .bazelrc Configuration file for project-specific options.
    xxx.bzl Defines utility extension for build scripts which
    will be loaded from the BUILD.

    View Slide

  34. Bazel for iOS
    rules_apple
    rules_swift
    apple_support
    https://github.com/bazelbuild/rules_apple
    https://github.com/bazelbuild/rules_swift
    https://github.com/bazelbuild/apple_support

    View Slide

  35. Bazel for iOS
    for more complicated modules
    rules_apple_line
    https://github.com/line/rules_apple_line

    View Slide

  36. How to build
    load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
    http_archive(
    name = "build_bazel_rules_apple",
    sha256 = “….",
    url = “https://github.com/bazelbuild/rules_apple/releases/download/x.x.x/rules_apple.x.x.x.tar.gz”,
    )
    load(
    "@build_bazel_rules_apple//apple:repositories.bzl",
    "apple_rules_dependencies",
    )
    apple_rules_dependencies()
    load(
    "@build_bazel_rules_swift//swift:repositories.bzl",
    "swift_rules_dependencies",
    )
    swift_rules_dependencies()
    load(
    "@build_bazel_apple_support//lib:repositories.bzl",
    "apple_support_dependencies",
    )
    apple_support_dependencies()
    WORKSPACE

    View Slide

  37. Library
    How to build
    load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")
    swift_library(
    name = "Sample",
    srcs = glob(["Sources/**/*.swift"]),
    data = glob(["Resources/**/*"]),
    visibility = ["//visibility:public"],
    )
    BUILD

    View Slide

  38. Test
    How to build
    load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")
    load("@build_bazel_rules_apple//apple:ios.bzl", "ios_unit_test")
    swift_library(
    name = "TestLib",
    srcs = glob(["Tests/**/*.swift"]),
    )
    ios_unit_test(
    name = "Test",
    deps = [":TestLib"],
    )
    BUILD

    View Slide

  39. Application
    How to build
    load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application")
    load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")
    swift_library(
    name = "AppLib",
    srcs = glob(["App/**/*.swift"]),
    )
    ios_application(
    name = "App",
    app_icons = glob(["Assets.xcassets/AppIcon.appiconset/**"]),
    bundle_id = "com.example.app",
    families = [
    "iphone",
    "ipad",
    ],
    infoplists = ["Info.plist"],
    launch_storyboard = "Launch.storyboard",
    minimum_os_version = "14.0",
    deps = [":AppLib"],
    )
    BUILD

    View Slide

  40. How to build
    $ bazel build //…
    Build all targets
    $ bazel build //package-name:target-name
    Build specified target

    View Slide

  41. Carthage
    CocoaPods
    • pinterest/PodToBUILD
    • bazel-ios/cocoapods-bazel
    • with build scripts: swift_library, objc_library
    • with build scripts: apple_dynamic_framework_import
    Our Choice
    Third-party Library

    View Slide

  42. Bazel + Xcode
    BUILD xcodeproj

    View Slide

  43. Tulsi by Google
    https://github.com/bazelbuild/tulsi
    XCHammer by Pinterest
    https://github.com/pinterest/xchammer
    Bazel + Xcode

    View Slide

  44. index-import by Lyft
    https://github.com/lyft/index-import
    Bazel + Xcode
    Replaces the paths in the index file created by Bazel
    so that it can be used for Xcode.

    View Slide

  45. Downside
    • High adoption cost
    • High learning cost
    • iOS is not officially supported
    • Some Xcode features have been disabled

    View Slide

  46. Downside
    Wrong setting will rather increase the build time.

    View Slide

  47. Alternatives
    Buck by Facebook
    https://github.com/facebook/buck
    Gradle
    https://github.com/gradle/gradle

    View Slide

  48. The power of Bazel with micro modules

    View Slide

  49. Aggressive Caching
    Once a module is built, it won't be rebuilt
    until the code in that module is changed.

    View Slide

  50. Aggressive Caching
    Caches are created per module.
    Micro modular architecture maximizes cache efficiency.
    =

    View Slide

  51. Test Results Caching
    Bazel will not run already succeeded tests.

    View Slide

  52. Test Results Caching
    That’s can be ignored by
    bazel test //… --nocache_test_results

    View Slide

  53. Remote Caching
    Once it's built somewhere, all developers
    don't have to rebuild it.
    Build once, reuse everywhere:

    View Slide

  54. Remote Caching
    Remote
    Cache
    Client
    CI
    Client
    Client
    Cache
    Cache
    Push

    View Slide

  55. Remote Caching
    Caches can be minimized by
    bazel build //… ——remote_download_minimal

    View Slide

  56. Remote Caching
    Now, all we need is ample network bandwidth
    instead of Mac Pro.

    View Slide

  57. Static Linking
    Too many dynamic frameworks can have a negative
    impact on the launch speed of your app.

    View Slide

  58. Static Linking
    Bazel compiles all modules into static libraries,
    so micro modules has no negative impacts.

    View Slide

  59. Dependency Diagrams
    Bazel can track all dependency graphs and parse them
    with the bazel query command.
    It can be output as a graphical diagram with Graphviz.

    View Slide

  60. Dependency Diagrams
    https://github.com/kastiglione/bazel-xcode-demo-swift-driver

    View Slide

  61. Conclusion

    View Slide

  62. • Micro Modular Architecture maximizes the synergy
    with Bazel, and vice versa.
    Conclusion
    • Build speed has room for improvement in software
    architecture as well as hardware performance.
    • Bazel will dramatically improve our developer
    experience.

    View Slide

  63. References
    • Bazel Document
    • Faster Together: Uber Engineering’s iOS Monorepo
    • How Tokopedia Achieved 1000% Faster iOS Build Time
    • Improving Build Performance of LINE for iOS with Bazel
    • Developing fast & reliable iOS builds at Pinterest (Part one)
    • kastiglione/bazel-xcode-demo-swift-driver
    https://eng.uber.com/ios-monorepo
    https://engineering.linecorp.com/en/blog/improving-build-performance-line-ios-bazel
    https://medium.com/tokopedia-engineering/how-tokopedia-achieved-1000-faster-ios-build-time-7664b2d8ae5
    https://medium.com/pinterest-engineering/developing-fast-reliable-ios-builds-at-pinterest-part-one-cb1810407b92
    https://bazel.build
    https://github.com/kastiglione/bazel-xcode-demo-swift-driver

    View Slide

  64. Thank you
    Ryo Aoyama
    iOSDC Japan 2020

    View Slide