Majestic Modular Monoliths

Microservices are everywhere. Everyone seems to be either going into that direction or is talking about doing so. But are they really the best choice for you?

Developers! Architects! Buckle up as we're going to cut through the hype. Instead of going all-in on microservices or all-in on big ball of mud, we'll introduce a third choice: the Majestic Modular Monolith! We'll look at what this brings to the table, when this may be a good fit and how it compares to the other two approaches in terms of code organization, productivity, scalability and much more. We'll look at how this can be designed and implemented in practice, we won't shy away from the hard questions.

Video recording: https://vimeo.com/233980163

Transcript

1. Majestic
   Modular
   Monoliths
   Photo by Jon Clark from Darwin, NT, Australia - Balls Pyramid pano, CC BY 2.0,
   https://commons.wikimedia.org/w/index.php?curid=41160475
   @axelfontaine
   

   View Slide

2. Axel Fontaine
   @axelfontaine
   flywaydb.org
   boxfuse.com
   

   View Slide

3. about
   questions
   

   View Slide

4. View Slide

5. 10110111
   00101011
   01011011
   11001...
   MyClass{
   do() {
   run();
   ...
   

   View Slide

6. 10110111
   00101011
   01011011
   11001...
   MyClass{
   do() {
   run();
   ...
   As code size increases,
   where is the bottleneck?
   

   View Slide

7. 10110111
   00101011
   01011011
   11001...
   MyClass{
   do() {
   run();
   ...
   As code size increases,
   where is the bottleneck?
   95% / 5%
   Read/Write Ratio
   

   View Slide

8. MyClass{
   do() {
   run();
   ...
   As code size increases,
   where is the bottleneck?
   95% / 5%
   Read/Write Ratio
   Optimize for this!
   

   View Slide

9. MyClass{
   do() {
   run();
   ...
   MyClass{
   do() {
   run();
   ...
   MyClass{
   do() {
   run();
   ...
   As code size increases,
   where is the bottleneck?
   95% / 5%
   Read/Write Ratio
   ✓ Add Structure
   ✓ Raise Abstraction
   ✓ Reduce Cognitive Load
   

   View Slide

10. MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?
    95% / 5%
    Read/Write Ratio
    Seeing the forest
    through the trees!
    

    View Slide

11. MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    MyClass{
    do() {
    run();
    ...
    As code size increases,
    where is the bottleneck?
    95% / 5%
    Read/Write Ratio
    ✓ Methods
    ✓ Classes
    ✓ Packages
    ✓ Modules
    ✓ Applications
    

    View Slide

12. Architecture
    =
    Interactions & Relationships
    between these
    ✓ Methods
    ✓ Classes
    ✓ Packages
    ✓ Modules
    ✓ Applications
    

    View Slide

13. Architecture Styles
    Monolith Microservices
    

    View Slide

14. Architecture Styles
    based on conference
    agendas
    of the last
    3-4 years
    Monolith Microservices
    

    View Slide

15. View Slide

16. View Slide

17. View Slide

18. Architecture Styles
    Monolith Microservices
    

    View Slide

19. Architecture Styles
    Integrated
    System
    Distributed
    System
    

    View Slide

20. View Slide

21. Monolith Microservices
    • One artifact
    • Entanglement risk
    • Simple method calls
    • All parts always up
    and available
    • Easy interface refactoring
    • Application scales as a unit
    • One database
    • Transactions
    • One platform
    • JAR hell risk (forced dependency
    convergence)
    • Limited team parallelization
    • Many individual services
    • Focus on clear small units
    • Unreliable network calls
    • Service discovery + internal load
    balancing + circuit breakers
    • Difficult to refactor
    • Services scale individually
    • Polyglot persistence
    • Eventual consistency
    • Platform choice
    • Works with incompatible library
    versions
    • Easy team parallelization
    

    View Slide

22. Very large organizations
    Very complex applications
    Very small organizations
    Very simple applications
    Monolith Microservices
    Is there room for something
    in the middle?
    

    View Slide

23. Introducing
    Distributed Monoliths
    

    View Slide

24. Monolith Microservices
    Modular Monolith
    

    View Slide

25. Majestic
    Modular Monolith
    

    View Slide

26. Adding Structure to the Monolith
    Most monoliths
    

    View Slide

27. Layered Architecture
    Controllers
    Services
    Repositories
    M
    O
    D
    E
    L
    

    View Slide

28. Layered Architecture
    Controllers
    Services
    Repositories
    Customer Invoice Payment
    

    View Slide

29. View Slide

30. View Slide

31. Layered Architecture
    Controllers
    Services
    Repositories
    Customer Invoice Payment
    

    View Slide

32. Aggregate
    Domain Driven Design
    Controllers
    Services
    Repositories
    Customer
    Aggregate
    Invoice
    Aggregate
    Payment
    

    View Slide

33. Acyclic Dependencies
    Payment
    Customer
    Invoice
    Changes never
    affect Invoice
    Changes can
    affect Invoice
    

    View Slide

34. Acyclic Dependencies
    Unstable concept
    Stable concept
    depends on
    

    View Slide

35. Module Boundaries and Structure
    API
    ✓Look for highly cohesive blocks of functionality
    ✓Aim for low coupling to other modules
    ✓Expose a small, well-defined, stable API
    ✓Only expose aggregate root
    ✓Make implementation and other entities private
    ✓Only depend on API (not impl) of other modules
    Impl
    (private)
    Reduces cognitive load!
    

    View Slide

36. Only depend on API
    API
    Impl
    (private)
    API
    Impl
    (private)
    API
    Impl
    (private)
    

    View Slide

37. Code Isolation
    Separate
    Package
    Maven
    Module
    Separate
    Repo
    External
    Service
    different JAR
    same version
    same app
    different JAR
    different version
    same app
    same JAR
    same version
    same app
    different JAR
    different version
    different app
    Microservices
    Monolith
    Great for sharing
    technical & infra.
    modules between
    apps!
    

    View Slide

38. Maven
    Module
    Separate
    Repo
    different JAR
    same version
    same app
    different JAR
    different version
    same app
    ✓Restricts compile classpath (but not reflection!)
    ✓Enforces directed acyclic graph (Maven module build order)
    ✓Only rebuild & test module and its dependents
    mvn clean install -pl mymodule -amd
    ✓Easy overview diagrams with IntelliJ or Maven Graph Plugin
    Seeing the forest
    through the trees!
    

    View Slide

39. Avoiding Jar Hell
    myapp
    libxyz 5.3
    libabc 4.0
    mymodule
    libabc 2.0
    ✓Limit your dependencies to absolute minimum
    ✓Prefer libraries with zero transitive dependencies
    ✓Enforce dependency convergence
    ✓Shade if conflict is not resolvable
    

    View Slide

40. 
    org.apache.maven.plugins
    maven-enforcer-plugin
    3.0.0-M1
    
    
    enforce
    
    
    
    true
    
    
    
    
    enforce
    
    
    
    Enforcing Dependency Convergence
    

    View Slide

41. Enforcing Package Usage
    (Java 9 with named modules)
    module com.hello.payment {
    exports com.hello.payment.api;
    requires com.hello.invoice.api;
    requires java.sql;
    }
    

    View Slide

42. Enforcing Package Usage
    (Java 8 with code-assert)
    @Test
    public void dependency() {
    class ComHello extends DependencyRuler {
    DependencyRule paymentApi, paymentImpl, invoiceApi;
    public void defineRules() {
    paymentApi().mayUse(invoiceApi);
    paymentImpl().mayUse(paymentApi, invoiceApi);
    }
    }
    AnalyzerConfig config = AnalyzerConfig.maven().main();
    DependencyRules rules = DependencyRules.denyAll()
    .withRelativeRules(new ComHello()).withExternals("java.sql");
    DependencyResult result = new DependencyAnalyzer(config).rules(rules).analyze();
    assertThat(result, matchesRulesExactly());
    }
    https://github.com/nidi3/code-assert
    

    View Slide

43. ✓Referential integrity
    ✓Atomic transactions
    (vs eventual consistency)
    Relational Databases
    

    View Slide

44. Relational Databases
    ✓Ensure each module only accesses
    its own tables
    

    View Slide

45. Relational Databases
    ✓Ensure each module only accesses
    its own tables
    ✓No sharing of tables
    between modules
    ✓Joins only between tables
    of same module (not across modules!)
    ✓Maintain referential integrity and
    transactions across modules
    

    View Slide

46. API
    Impl
    (private)
    API
    Impl
    (private)
    API
    Impl
    (private)
    Schema
    Schema
    Schema
    

    View Slide

47. Data Isolation
    Separate
    Table
    Separate
    Schema
    Separate
    Database
    Other
    Persistence
    different schema
    same database
    same RDBMS
    different schema
    different database
    same RDBMS
    same schema
    same database
    same RDBMS
    different schema
    different database
    different (R)DBMS
    Same
    flyway_schema_history
    table
    

    View Slide

48. Scaling
    EC2 x1e.32xlarge
    128 vCPUs
    3904 GB RAM
    EC2 t2.nano
    1 vCPU
    0.5 GB RAM
    

    View Slide

49. Scaling Asymmetrically
    Executor
    Service
    different capacity
    per module
    ExecutorService paymentExecutor = Executors.newFixedThreadPool(10);
    ExecutorService invoiceExecutor = Executors.newFixedThreadPool(25);
    

    View Slide

50. Scaling Asymmetrically
    Executor
    Service
    RDBMS
    Queue
    different number
    of consumers per
    module, persistent
    and participates in
    RDBMS
    transaction
    different capacity
    per module
    id message
    1 {“customer”:”abc”}
    2 {“other”:”xyz”}
    INSERT INTO queue (message) VALUES (?)
    DELETE FROM queue WHERE id =
    (SELECT id FROM queue
    ORDER BY id LIMIT 1
    FOR UPDATE SKIP LOCKED)
    RETURNING message
    

    View Slide

51. Scaling Asymmetrically
    Executor
    Service
    RDBMS
    Queue
    Dedicated
    Queue
    different number
    of consumers per
    module, persistent
    and participates in
    RDBMS
    transaction
    different number
    of consumers per
    module, persistent
    and very high
    performance
    different capacity
    per module
    

    View Slide

52. Language independence
    

    View Slide

53. boxfuse.com
    Continuous Deployment as a Service
    for JVM, Node.js and Go apps
    on AWS
    Deploying
    ✓ Up and running in minutes
    ✓ Deploy with 1 command
    ✓ Focus on development
    ✓ Immutable Infrastructure as Code
    ✓ Minimal images
    ✓ Zero downtime blue/green deployments
    boxfuse run my-majestic-monolith.jar –env=prod
    

    View Slide

54. flywaydb.org
    Evolve your relational database schemas
    reliably across all your environments
    for each of your modules and services
    with pleasure and plain SQL
    ✓ Supports all popular RDBMS
    ✓ Millions of users
    ✓ Designed for Continuous Delivery
    ✓ Open-source Community Edition
    and commercial Pro and Enterprise Editions
    ✓ Highly focused and very easy to get started
    Database Schema Evolution
    

    View Slide

55. Best, pragmatic, choice for most companies
    combining the simplicity of monoliths
    with structure and focus of microservices
    Summary
    Majestic
    Modular Monolith
    

    View Slide

56. Go Modular and
    make your Monolith
    Majestic again!
    

    View Slide

57. Axel Fontaine
    @axelfontaine
    flywaydb.org
    boxfuse.com
    Thanks !
    

    View Slide