$30 off During Our Annual Pro Sale. View Details »

Majestic Modular Monoliths

Axel Fontaine
February 22, 2018

Majestic Modular Monoliths

Microservices are everywhere. Everyone seems to be either going into that direction or is talking about doing so. But are they really the best choice for you?

Developers! Architects! Buckle up as we're going to cut through the hype. Instead of going all-in on microservices or all-in on big ball of mud, we'll introduce a third choice: the Majestic Modular Monolith! We'll look at what this brings to the table, when this may be a good fit and how it compares to the other two approaches in terms of code organization, productivity, scalability and much more. We'll look at how this can be designed and implemented in practice, we won't shy away from the hard questions.

Video recording: https://vimeo.com/233980163

Axel Fontaine

February 22, 2018
Tweet

More Decks by Axel Fontaine

Other Decks in Technology

Transcript

  1. Majestic
    Modular
    Monoliths
    Photo by Jon Clark from Darwin, NT, Australia - Balls Pyramid pano, CC BY 2.0,
    https://commons.wikimedia.org/w/index.php?curid=41160475
    @axelfontaine

    View Slide

  2. Axel Fontaine
    @axelfontaine
    flywaydb.org
    boxfuse.com

    View Slide

  3. about
    questions

    View Slide

  4. View Slide

  5. 10110111
    00101011
    01011011
    11001...
    MyClass{
    do() {
    run();
    ...

    View Slide

  6. 10110111
    00101011
    01011011
    11001...
    MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?

    View Slide

  7. 10110111
    00101011
    01011011
    11001...
    MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?
    95% / 5%
    Read/Write Ratio

    View Slide

  8. MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?
    95% / 5%
    Read/Write Ratio
    Optimize for this!

    View Slide

  9. MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?
    95% / 5%
    Read/Write Ratio
    ✓ Add Structure
    ✓ Raise Abstraction
    ✓ Reduce Cognitive Load

    View Slide

  10. MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?
    95% / 5%
    Read/Write Ratio
    Seeing the forest
    through the trees!

    View Slide

  11. MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?
    95% / 5%
    Read/Write Ratio
    ✓ Methods
    ✓ Classes
    ✓ Packages
    ✓ Modules
    ✓ Applications

    View Slide

  12. Architecture
    =
    Interactions & Relationships
    between these
    ✓ Methods
    ✓ Classes
    ✓ Packages
    ✓ Modules
    ✓ Applications

    View Slide

  13. Architecture Styles
    Monolith Microservices

    View Slide

  14. Architecture Styles
    based on conference
    agendas
    of the last
    3-4 years
    Monolith Microservices

    View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. Architecture Styles
    Monolith Microservices

    View Slide

  19. Architecture Styles
    Integrated
    System
    Distributed
    System

    View Slide

  20. View Slide

  21. Monolith Microservices
    • One artifact
    • Entanglement risk
    • Simple method calls
    • All parts always up
    and available
    • Easy interface refactoring
    • Application scales as a unit
    • One database
    • Transactions
    • One platform
    • JAR hell risk (forced dependency
    convergence)
    • Limited team parallelization
    • Many individual services
    • Focus on clear small units
    • Unreliable network calls
    • Service discovery + internal load
    balancing + circuit breakers
    • Difficult to refactor
    • Services scale individually
    • Polyglot persistence
    • Eventual consistency
    • Platform choice
    • Works with incompatible library
    versions
    • Easy team parallelization

    View Slide

  22. Very large organizations
    Very complex applications
    Very small organizations
    Very simple applications
    Monolith Microservices
    Is there room for something
    in the middle?

    View Slide

  23. Introducing
    Distributed Monoliths

    View Slide

  24. Monolith Microservices
    Modular Monolith

    View Slide

  25. Majestic
    Modular Monolith

    View Slide

  26. Adding Structure to the Monolith
    Most monoliths

    View Slide

  27. Layered Architecture
    Controllers
    Services
    Repositories
    M
    O
    D
    E
    L

    View Slide

  28. Layered Architecture
    Controllers
    Services
    Repositories
    Customer Invoice Payment

    View Slide

  29. View Slide

  30. View Slide

  31. Layered Architecture
    Controllers
    Services
    Repositories
    Customer Invoice Payment

    View Slide

  32. Aggregate
    Domain Driven Design
    Controllers
    Services
    Repositories
    Customer
    Aggregate
    Invoice
    Aggregate
    Payment

    View Slide

  33. Acyclic Dependencies
    Payment
    Customer
    Invoice
    Changes never
    affect Invoice
    Changes can
    affect Invoice

    View Slide

  34. Acyclic Dependencies
    Unstable concept
    Stable concept
    depends on

    View Slide

  35. Module Boundaries and Structure
    API
    ✓Look for highly cohesive blocks of functionality
    ✓Aim for low coupling to other modules
    ✓Expose a small, well-defined, stable API
    ✓Only expose aggregate root
    ✓Make implementation and other entities private
    ✓Only depend on API (not impl) of other modules
    Impl
    (private)
    Reduces cognitive load!

    View Slide

  36. Only depend on API
    API
    Impl
    (private)
    API
    Impl
    (private)
    API
    Impl
    (private)

    View Slide

  37. Code Isolation
    Separate
    Package
    Maven
    Module
    Separate
    Repo
    External
    Service
    different JAR
    same version
    same app
    different JAR
    different version
    same app
    same JAR
    same version
    same app
    different JAR
    different version
    different app
    Microservices
    Monolith
    Great for sharing
    technical & infra.
    modules between
    apps!

    View Slide

  38. Maven
    Module
    Separate
    Repo
    different JAR
    same version
    same app
    different JAR
    different version
    same app
    ✓Restricts compile classpath (but not reflection!)
    ✓Enforces directed acyclic graph (Maven module build order)
    ✓Only rebuild & test module and its dependents
    mvn clean install -pl mymodule -amd
    ✓Easy overview diagrams with IntelliJ or Maven Graph Plugin
    Seeing the forest
    through the trees!

    View Slide

  39. Avoiding Jar Hell
    myapp
    libxyz 5.3
    libabc 4.0
    mymodule
    libabc 2.0
    ✓Limit your dependencies to absolute minimum
    ✓Prefer libraries with zero transitive dependencies
    ✓Enforce dependency convergence
    ✓Shade if conflict is not resolvable

    View Slide


  40. org.apache.maven.plugins
    maven-enforcer-plugin
    3.0.0-M1


    enforce



    true




    enforce



    Enforcing Dependency Convergence

    View Slide

  41. Enforcing Package Usage
    (Java 9 with named modules)
    module com.hello.payment {
    exports com.hello.payment.api;
    requires com.hello.invoice.api;
    requires java.sql;
    }

    View Slide

  42. Enforcing Package Usage
    (Java 8 with code-assert)
    @Test
    public void dependency() {
    class ComHello extends DependencyRuler {
    DependencyRule paymentApi, paymentImpl, invoiceApi;
    public void defineRules() {
    paymentApi().mayUse(invoiceApi);
    paymentImpl().mayUse(paymentApi, invoiceApi);
    }
    }
    AnalyzerConfig config = AnalyzerConfig.maven().main();
    DependencyRules rules = DependencyRules.denyAll()
    .withRelativeRules(new ComHello()).withExternals("java.sql");
    DependencyResult result = new DependencyAnalyzer(config).rules(rules).analyze();
    assertThat(result, matchesRulesExactly());
    }
    https://github.com/nidi3/code-assert

    View Slide

  43. ✓Referential integrity
    ✓Atomic transactions
    (vs eventual consistency)
    Relational Databases

    View Slide

  44. Relational Databases
    ✓Ensure each module only accesses
    its own tables

    View Slide

  45. Relational Databases
    ✓Ensure each module only accesses
    its own tables
    ✓No sharing of tables
    between modules
    ✓Joins only between tables
    of same module (not across modules!)
    ✓Maintain referential integrity and
    transactions across modules

    View Slide

  46. API
    Impl
    (private)
    API
    Impl
    (private)
    API
    Impl
    (private)
    Schema
    Schema
    Schema

    View Slide

  47. Data Isolation
    Separate
    Table
    Separate
    Schema
    Separate
    Database
    Other
    Persistence
    different schema
    same database
    same RDBMS
    different schema
    different database
    same RDBMS
    same schema
    same database
    same RDBMS
    different schema
    different database
    different (R)DBMS
    Same
    flyway_schema_history
    table

    View Slide

  48. Scaling
    EC2 x1e.32xlarge
    128 vCPUs
    3904 GB RAM
    EC2 t2.nano
    1 vCPU
    0.5 GB RAM

    View Slide

  49. Scaling Asymmetrically
    Executor
    Service
    different capacity
    per module
    ExecutorService paymentExecutor = Executors.newFixedThreadPool(10);
    ExecutorService invoiceExecutor = Executors.newFixedThreadPool(25);

    View Slide

  50. Scaling Asymmetrically
    Executor
    Service
    RDBMS
    Queue
    different number
    of consumers per
    module, persistent
    and participates in
    RDBMS
    transaction
    different capacity
    per module
    id message
    1 {“customer”:”abc”}
    2 {“other”:”xyz”}
    INSERT INTO queue (message) VALUES (?)
    DELETE FROM queue WHERE id =
    (SELECT id FROM queue
    ORDER BY id LIMIT 1
    FOR UPDATE SKIP LOCKED)
    RETURNING message

    View Slide

  51. Scaling Asymmetrically
    Executor
    Service
    RDBMS
    Queue
    Dedicated
    Queue
    different number
    of consumers per
    module, persistent
    and participates in
    RDBMS
    transaction
    different number
    of consumers per
    module, persistent
    and very high
    performance
    different capacity
    per module

    View Slide

  52. Language independence

    View Slide

  53. boxfuse.com
    Continuous Deployment as a Service
    for JVM, Node.js and Go apps
    on AWS
    Deploying
    ✓ Up and running in minutes
    ✓ Deploy with 1 command
    ✓ Focus on development
    ✓ Immutable Infrastructure as Code
    ✓ Minimal images
    ✓ Zero downtime blue/green deployments
    boxfuse run my-majestic-monolith.jar –env=prod

    View Slide

  54. flywaydb.org
    Evolve your relational database schemas
    reliably across all your environments
    for each of your modules and services
    with pleasure and plain SQL
    ✓ Supports all popular RDBMS
    ✓ Millions of users
    ✓ Designed for Continuous Delivery
    ✓ Open-source Community Edition
    and commercial Pro and Enterprise Editions
    ✓ Highly focused and very easy to get started
    Database Schema Evolution

    View Slide

  55. Best, pragmatic, choice for most companies
    combining the simplicity of monoliths
    with structure and focus of microservices
    Summary
    Majestic
    Modular Monolith

    View Slide

  56. Go Modular and
    make your Monolith
    Majestic again!

    View Slide

  57. Axel Fontaine
    @axelfontaine
    flywaydb.org
    boxfuse.com
    Thanks !

    View Slide