Majestic Modular Monoliths

Microservices are everywhere. Everyone seems to be either going into that direction or is talking about doing so. But are they really the best choice for you?

Developers! Architects! Buckle up as we're going to cut through the hype. Instead of going all-in on microservices or all-in on big ball of mud, we'll introduce a third choice: the Majestic Modular Monolith! We'll look at what this brings to the table, when this may be a good fit and how it compares to the other two approaches in terms of code organization, productivity, scalability and much more. We'll look at how this can be designed and implemented in practice, we won't shy away from the hard questions.

Video recording: https://vimeo.com/233980163

Transcript

1. Majestic Modular Monoliths Photo by Jon Clark from Darwin, NT, Australia - Balls Pyramid pano, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=41160475 @axelfontaine
2. Axel Fontaine @axelfontaine flywaydb.org boxfuse.com
3. about questions
5. 10110111 00101011 01011011 11001... MyClass{ do() { run(); ...
6. 10110111 00101011 01011011 11001... MyClass{ do() { run(); ... As code size increases, where is the bottleneck?
7. 10110111 00101011 01011011 11001... MyClass{ do() { run(); ... As code size increases, where is the bottleneck? 95% / 5% Read/Write Ratio
8. MyClass{ do() { run(); ... As code size increases, where is the bottleneck? 95% / 5% Read/Write Ratio Optimize for this!
9. MyClass{ do() { run(); ... MyClass{ do() { run(); ... MyClass{ do() { run(); ... As code size increases, where is the bottleneck? 95% / 5% Read/Write Ratio ✓ Add Structure ✓ Raise Abstraction ✓ Reduce Cognitive Load
10. MyClass{ do() { run(); ... MyClass{ do() { run(); ... MyClass{ do() { run(); ... As code size increases, where is the bottleneck? 95% / 5% Read/Write Ratio Seeing the forest through the trees!
11. MyClass{ do() { run(); ... MyClass{ do() { run(); ... MyClass{ do() { run(); ... As code size increases, where is the bottleneck? 95% / 5% Read/Write Ratio ✓ Methods ✓ Classes ✓ Packages ✓ Modules ✓ Applications
12. Architecture = Interactions & Relationships between these ✓ Methods ✓ Classes ✓ Packages ✓ Modules ✓ Applications
13. Architecture Styles Monolith Microservices
14. Architecture Styles based on conference agendas of the last 3-4 years Monolith Microservices
18. Architecture Styles Monolith Microservices
19. Architecture Styles Integrated System Distributed System
21. Monolith Microservices • One artifact • Entanglement risk • Simple method calls • All parts always up and available • Easy interface refactoring • Application scales as a unit • One database • Transactions • One platform • JAR hell risk (forced dependency convergence) • Limited team parallelization • Many individual services • Focus on clear small units • Unreliable network calls • Service discovery + internal load balancing + circuit breakers • Difficult to refactor • Services scale individually • Polyglot persistence • Eventual consistency • Platform choice • Works with incompatible library versions • Easy team parallelization
22. Very large organizations Very complex applications Very small organizations Very simple applications Monolith Microservices Is there room for something in the middle?
23. Introducing Distributed Monoliths
24. Monolith Microservices Modular Monolith
25. Majestic Modular Monolith
26. Adding Structure to the Monolith Most monoliths
27. Layered Architecture Controllers Services Repositories M O D E L
28. Layered Architecture Controllers Services Repositories Customer Invoice Payment
31. Layered Architecture Controllers Services Repositories Customer Invoice Payment
32. Aggregate Domain Driven Design Controllers Services Repositories Customer Aggregate Invoice Aggregate Payment
33. Acyclic Dependencies Payment Customer Invoice Changes never affect Invoice Changes can affect Invoice
34. Acyclic Dependencies Unstable concept Stable concept depends on
35. Module Boundaries and Structure API ✓Look for highly cohesive blocks of functionality ✓Aim for low coupling to other modules ✓Expose a small, well-defined, stable API ✓Only expose aggregate root ✓Make implementation and other entities private ✓Only depend on API (not impl) of other modules Impl (private) Reduces cognitive load!
36. Only depend on API API Impl (private) API Impl (private) API Impl (private)
37. Code Isolation Separate Package Maven Module Separate Repo External Service different JAR same version same app different JAR different version same app same JAR same version same app different JAR different version different app Microservices Monolith Great for sharing technical & infra. modules between apps!
38. Maven Module Separate Repo different JAR same version same app different JAR different version same app ✓Restricts compile classpath (but not reflection!) ✓Enforces directed acyclic graph (Maven module build order) ✓Only rebuild & test module and its dependents mvn clean install -pl mymodule -amd ✓Easy overview diagrams with IntelliJ or Maven Graph Plugin Seeing the forest through the trees!
39. Avoiding Jar Hell myapp libxyz 5.3 libabc 4.0 mymodule libabc 2.0 ✓Limit your dependencies to absolute minimum ✓Prefer libraries with zero transitive dependencies ✓Enforce dependency convergence ✓Shade if conflict is not resolvable
40. <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>3.0.0-M1</version> <executions> <execution> <id>enforce</id> <configuration> <rules> <dependencyConvergence> <uniqueVersions>true</uniqueVersions> </dependencyConvergence> </rules> </configuration> <goals> <goal>enforce</goal> </goals> </execution> </executions> Enforcing Dependency Convergence
41. Enforcing Package Usage (Java 9 with named modules) module com.hello.payment { exports com.hello.payment.api; requires com.hello.invoice.api; requires java.sql; }
42. Enforcing Package Usage (Java 8 with code-assert) @Test public void dependency() { class ComHello extends DependencyRuler { DependencyRule paymentApi, paymentImpl, invoiceApi; public void defineRules() { paymentApi().mayUse(invoiceApi); paymentImpl().mayUse(paymentApi, invoiceApi); } } AnalyzerConfig config = AnalyzerConfig.maven().main(); DependencyRules rules = DependencyRules.denyAll() .withRelativeRules(new ComHello()).withExternals("java.sql"); DependencyResult result = new DependencyAnalyzer(config).rules(rules).analyze(); assertThat(result, matchesRulesExactly()); } https://github.com/nidi3/code-assert
43. ✓Referential integrity ✓Atomic transactions (vs eventual consistency) Relational Databases
44. Relational Databases ✓Ensure each module only accesses its own tables
45. Relational Databases ✓Ensure each module only accesses its own tables ✓No sharing of tables between modules ✓Joins only between tables of same module (not across modules!) ✓Maintain referential integrity and transactions across modules
46. API Impl (private) API Impl (private) API Impl (private) Schema Schema Schema
47. Data Isolation Separate Table Separate Schema Separate Database Other Persistence different schema same database same RDBMS different schema different database same RDBMS same schema same database same RDBMS different schema different database different (R)DBMS Same flyway_schema_history table
48. Scaling EC2 x1e.32xlarge 128 vCPUs 3904 GB RAM EC2 t2.nano 1 vCPU 0.5 GB RAM
49. Scaling Asymmetrically Executor Service different capacity per module ExecutorService paymentExecutor = Executors.newFixedThreadPool(10); ExecutorService invoiceExecutor = Executors.newFixedThreadPool(25);
50. Scaling Asymmetrically Executor Service RDBMS Queue different number of consumers per module, persistent and participates in RDBMS transaction different capacity per module id message 1 {“customer”:”abc”} 2 {“other”:”xyz”} INSERT INTO queue (message) VALUES (?) DELETE FROM queue WHERE id = (SELECT id FROM queue ORDER BY id LIMIT 1 FOR UPDATE SKIP LOCKED) RETURNING message
51. Scaling Asymmetrically Executor Service RDBMS Queue Dedicated Queue different number of consumers per module, persistent and participates in RDBMS transaction different number of consumers per module, persistent and very high performance different capacity per module
52. Language independence
53. boxfuse.com Continuous Deployment as a Service for JVM, Node.js and Go apps on AWS Deploying ✓ Up and running in minutes ✓ Deploy with 1 command ✓ Focus on development ✓ Immutable Infrastructure as Code ✓ Minimal images ✓ Zero downtime blue/green deployments boxfuse run my-majestic-monolith.jar –env=prod
54. flywaydb.org Evolve your relational database schemas reliably across all your environments for each of your modules and services with pleasure and plain SQL ✓ Supports all popular RDBMS ✓ Millions of users ✓ Designed for Continuous Delivery ✓ Open-source Community Edition and commercial Pro and Enterprise Editions ✓ Highly focused and very easy to get started Database Schema Evolution
55. Best, pragmatic, choice for most companies combining the simplicity of monoliths with structure and focus of microservices Summary Majestic Modular Monolith
56. Go Modular and make your Monolith Majestic again!
57. Axel Fontaine @axelfontaine flywaydb.org boxfuse.com Thanks !