From Monolith to Microservices

In this talk, We will focus on how to split up an old enterprise application which has become a monolith into microservices. We will examine the pros and cons of doing so and also the best design practices and architecture decisions that will ensure this process is a success.

We will mainly explore how to identify services in your monolithic application, identify seams in your monolith. We will also investigate how to decouple microservices from monoliths and the best practices when doing so. We will also focus on microservices development and DevOps tips.

We will be examining how to extract a monolith application using example PHP microservices that were pulled out of a larger legacy PHP application. I will also investigate some war stories on how Superbalist achieved this and the lessons learned along the way.

At the end of the talk, you will know better how to refactor your monolith and improve your own architecture.

Transcript

1. From Monolith to Microservices Liam Norman, Superbalist
2. Hi, I'm Liam @liamjnorman https:/ /www.liamnorman.com Software Engineer @ Superbalist.com Organiser of Cape Town PHP
3. About You
4. About You • Don't know what microservices are?
5. About You • Don't know what microservices are? • Have a monolith that you want to migrate?
6. About You • Don't know what microservices are? • Have a monolith that you want to migrate? • In the process of migrating a monolith?
7. About You • Don't know what microservices are? • Have a monolith that you want to migrate? • In the process of migrating a monolith? • Building a new architecture with microservices
8. About You • Don't know what microservices are? • Have a monolith that you want to migrate? • In the process of migrating a monolith? • Building a new architecture with microservices • This talk has a little bit for all of you
9. What are Microservices? Microservices at their core are small, independent, autonomous services that work together.
10. What are Microservices? Microservices at their core are small, independent, autonomous services that work together. • Microservices should be small.
11. What are Microservices? Microservices at their core are small, independent, autonomous services that work together. • Microservices should be small. • Microservices should be independent.
12. Autonomous
13. Autonomous • Microservices should be separate entities.
14. Autonomous • Microservices should be separate entities. • Tight Coupling is when you have services which rely on one another.
15. Autonomous • Microservices should be separate entities. • Tight Coupling is when you have services which rely on one another. • Avoid Tight Coupling
16. Autonomous • Microservices should be separate entities. • Tight Coupling is when you have services which rely on one another. • Avoid Tight Coupling • Microservices should communicate with one another via network calls.
17. "Can you make a change to a microservice and deploy it by itself without changing anything else?"
18. Benefits of Microservices Technology Freedom
19. Benefits of Microservices Technology Freedom • Can select the right tool for the job
20. Benefits of Microservices Technology Freedom • Can select the right tool for the job • No need to use a one-size-fits all approach as we often use for a monolith.
21. Benefits of Microservices Technology Freedom • Can select the right tool for the job • No need to use a one-size-fits all approach as we often use for a monolith. • Improved performance with correct tool!
22. Benefits of Microservices Resilience
23. Benefits of Microservices Resilience • Avoids cascading failures
24. Benefits of Microservices Resilience • Avoids cascading failures • Isolated services
25. Benefits of Microservices Resilience • Avoids cascading failures • Isolated services • Separates our application and can degrade gracefully if handled correctly.
26. Benefits of Microservices Scaling
27. Benefits of Microservices Scaling • Easier to scale as we can scale smaller independent services.
28. Benefits of Microservices Scaling • Easier to scale as we can scale smaller independent services. • Superbalist moved to microservices as we found it was important to ensure we could handle the load when a large sale occurred or Black Friday occurred.
29. Benefits of Microservices Ease of Deployment
30. Benefits of Microservices Ease of Deployment • Deploying monoliths is dangerous as they are large-impact, high risk deployments.
31. Benefits of Microservices Ease of Deployment • Deploying monoliths is dangerous as they are large-impact, high risk deployments. • Deploying microservices is safer as we deploy it independently and if we encounter a bug, we know exactly which service is causing it and can rollback.
33. Benefits of Microservices Composability A system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia
34. Benefits of Microservices Composability A system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia • Composable
35. Benefits of Microservices Composability A system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia • Composable • We no longer develop for one application.
36. No Silver Bullet
37. No Silver Bullet • Microservices are no silver bullet and don't make a good choice for everything.
38. No Silver Bullet • Microservices are no silver bullet and don't make a good choice for everything. • Microservices are great for many scenarios but they come with added complexity.
39. No Silver Bullet • Microservices are no silver bullet and don't make a good choice for everything. • Microservices are great for many scenarios but they come with added complexity. • It depends...
40. Cons of Microservices
41. Cons of Microservices • Added complexity in deployments
42. Cons of Microservices • Added complexity in deployments • Added complexity in managing logging and monitoring.
43. Cons of Microservices • Added complexity in deployments • Added complexity in managing logging and monitoring. • How to properly prevent cascading failure and degrade gracefully.
44. Modelling Microservices Let's begin!
45. Setting the Scene
46. Setting the Scene • Creating a simple order management system with a product microservices.
47. Setting the Scene • Creating a simple order management system with a product microservices. • We will be using Laravel, Docker and Loggly.
48. What Makes A Good Microservice?
49. What Makes A Good Microservice? • Loose coupling
50. What Makes A Good Microservice? • Loose coupling • High cohesion
51. Bounded Contexts Bounded contexts, What does that mean?
52. Bounded Contexts Bounded contexts, What does that mean? • Any given domain consists of multiple bounded contexts
53. Bounded Contexts Bounded contexts, What does that mean? • Any given domain consists of multiple bounded contexts • A bounded context is defined as “a specific responsibility enforced by explicit boundaries.”
54. Bounded Contexts
55. Bounded Contexts • Not sharing internal representations reduces tight coupling
56. Bounded Contexts • Not sharing internal representations reduces tight coupling • Bounded contexts lend themselves well to identifying boundaries, especially in code.
57. Bounded Contexts • Not sharing internal representations reduces tight coupling • Bounded contexts lend themselves well to identifying boundaries, especially in code. • Bounded contexts become excellent candidates for microservices.
58. Splitting the Monolith Where to start?
59. Seams
60. Seams • A seam is a portion of code that can be treated in isolation and worked on without impacting the rest of the codebase.
61. Seams • A seam is a portion of code that can be treated in isolation and worked on without impacting the rest of the codebase. • To break up a monolith, we must find seams.
62. Seams • A seam is a portion of code that can be treated in isolation and worked on without impacting the rest of the codebase. • To break up a monolith, we must find seams. • Bounded contexts make great seams because they are cohesive and loosely coupled.
63. Splitting into Packages
64. Splitting into Packages • Orders - for managing orders, order items, orders placed
65. Splitting into Packages • Orders - for managing orders, order items, orders placed
66. Splitting into Packages • Orders - for managing orders, order items, orders placed • Products - for managing products, product stock
67. Splitting into Packages • Orders - for managing orders, order items, orders placed • Products - for managing products, product stock
68. Splitting into Packages • Orders - for managing orders, order items, orders placed • Products - for managing products, product stock • Users - for managing users and user information
70. Packages in PHP namespace App\Http\Controllers; class ProductController extends Controller { ... }
71. Slow Down
72. Slow Down • This process could be quick or several weeks, months etc...
73. Slow Down • This process could be quick or several weeks, months etc... • Rather look at splitting the codebase accurately into viable packages.
74. Evolutionary Architecture and Microservices
75. Evolutionary Architecture and Microservices • It is difficult to change architecture once implemented.
76. Evolutionary Architecture and Microservices • It is difficult to change architecture once implemented. • Evolutionary architecture is a terminology created by Rebeca Parsons, Patrick Kua and Neal Ford. It refers to architecture which has the ability to change over time.
77. Evolutionary Architecture and Microservices • It is difficult to change architecture once implemented. • Evolutionary architecture is a terminology created by Rebeca Parsons, Patrick Kua and Neal Ford. It refers to architecture which has the ability to change over time. • Microservices enables this in an incremental way.
78. Evolutionary Architecture and Microservices
79. Methodology to Split Out the Microservice
80. Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams
81. Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages
82. Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality
83. Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality 4. Direct traffic to microservices
84. Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality 4. Direct traffic to microservices 5. Retire old code in Monolith
85. Choosing Your First Microservice
86. Choosing Your First Microservice • Pace of change
87. Choosing Your First Microservice • Pace of change • Isolation
88. Choosing Your First Microservice • Pace of change • Isolation • Risk
89. Tangled Dependencies • http:/ /stan4j.com/
90. Splitting Databases
91. Problems with the Monolithic Database Design
92. Problems with the Monolithic Database Design • A monolith DB is a single point of failure
93. Problems with the Monolithic Database Design • A monolith DB is a single point of failure • Bad for data retrieval and performance over time
94. Problems with the Monolithic Database Design • A monolith DB is a single point of failure • Bad for data retrieval and performance over time • Constraints to one type of database
95. Database Implementation in a Microservice-oriented Architecture.
96. Finding seams in your Database
97. Finding seams in your Database • Equally important as seams in application layer
98. Finding seams in your Database • Equally important as seams in application layer • Ensure each microservice has the data they need and doesn't need to make unnecessary API calls
99. Installing SchemaSpy apt-get install graphviz java -jar schemaspy.jar -t mysql -dp /users/liamnorman/sqljdbc4-3.0.jar -db products -host 127.0.0.1 -port 3306 -u testuser -p mygreatpassword -o /users/liamnorman/schemaspyoutput
101. The Database Split
102. The Database Split • Break parts of your database into smaller databases
103. The Database Split • Break parts of your database into smaller databases • Easy to revert, only retire old tables after split and microservice is running
104. The Database Split • Break parts of your database into smaller databases • Easy to revert, only retire old tables after split and microservice is running • Split database first
105. Breaking Foreign Key Relationships
106. Breaking Foreign Key Relationships
107. Data Integrity and Event Sourcing
108. Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc...
109. Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture.
110. Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture. • Multiple PubSub providers such as Google PubSub, AWS SNS, Socket Cluster etc...
111. Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture. • Multiple PubSub providers such as Google PubSub, AWS SNS, Socket Cluster etc...
112. Data Integrity and Event Sourcing
113. DevOps Tips for Microservices
114. DevOps Tips for Microservices • Distributed Logging
115. DevOps Tips for Microservices • Distributed Logging • Correlation IDs
116. DevOps Tips for Microservices
117. DevOps Tips for Microservices • Searchable Logs
118. DevOps Tips for Microservices • Searchable Logs • Monitoring and monitor networks between services.
119. Crafting our Microservices Application
120. Crafting our Microservices Application • We are going to build a simple order management system with a product Microservice in PHP 7 using Laravel 5.
121. Crafting our Microservices Application • We are going to build a simple order management system with a product Microservice in PHP 7 using Laravel 5. • We are going to focus on the product Microservice in the system.
122. App Toolbox
123. App Toolbox • PHP 7
124. App Toolbox • PHP 7 • Composer
125. App Toolbox • PHP 7 • Composer • Laravel 5
126. App Toolbox • PHP 7 • Composer • Laravel 5 • Docker
127. App Toolbox • PHP 7 • Composer • Laravel 5 • Docker • Monolog
128. App Toolbox • PHP 7 • Composer • Laravel 5 • Docker • Monolog • Loggly
129. Setting Up Our Product Microservice composer global require laravel/installer laravel new microshop
130. Setting up Docker version: "3" services: app: image: laravel-www container_name: product-api build: context: . dockerfile: docker/Dockerfile depends_on: - mysql env_file: - .docker.env volumes: - .:/var/www/html networks: - frontend_oms ports: - 8081:80 environment: APP_ENV: local CONTAINER_ROLE: app
131. Setting up Docker mysql: container_name: product-mysql image: mysql:5.7 ports: - 13307:3306 volumes: - mysql:/var/lib/mysql environment: MYSQL_DATABASE: homestead MYSQL_ROOT_PASSWORD: root MYSQL_USER: homestead MYSQL_PASSWORD: secret volumes: mysql: driver: "local" networks: frontend_oms: external: true
132. Docker Up and Running docker-compose build && docker- compose up -d e1576be47250 laravel-www "docker-php-entrypoi…" 4 seconds ago Up 2 seconds 9000/tcp, 0.0.0.0:8081->80/tcp product-api 474a6cb3928f mysql:5.7 "docker-entrypoint.s…" 7 minutes ago Up 4 seconds 0.0.0.0:13307->3306/tcp product-mysql 097314bbe4f9 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 80/tcp, 9000/tcp frontend-scheduler a8fb0eac2298 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 80/tcp, 9000/tcp frontend-queue 2d62d7fa1b7c 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 9000/tcp, 0.0.0.0:8080->80/tcp frontend-www 349b26ec6e4a mysql:5.7 "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:13306->3306/tcp frontend-mysql b2a42fd78a1d redis:4-alpine "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:16379->6379/tcp frontend-redis
133. Product Models php artisan make:model Product -mcr
134. Product Migrations <?php use Illuminate\Support\Facades\Schema; use Illuminate\Database\Schema\Blueprint; use Illuminate\Database\Migrations\Migration; class CreateProductsTable extends Migration { public function up() { Schema::create('products', function (Blueprint $table) { $table->increments('id'); $table->string('name'); $table->string('slug'); $table->bigInteger('price'); $table->integer('quantity'); $table->string('image'); $table->timestamps(); }); } public function down() { Schema::dropIfExists('products'); } }
135. Routes // Routes for Product CRUD Route::get('/products', 'ProductController@index')->name('products.index'); Route::post('/products', 'ProductController@store')->name('products.store'); Route::get('/products/{slug}', 'ProductController@show')->name('products.show'); Route::get('/products/{slug}/delete', 'ProductController@delete')->name('products.delete');
136. Product Controller <?php namespace App\Http\Controllers; use App\Product; use Illuminate\Http\Request; class ProductController extends Controller { /** * Display a listing of the resource. * * @return \Illuminate\Http\Response */ public function index() { $products = Product::all(); return response()->json($products); } }
137. Product Controller /** * Display the specified resource. * * @param string $slug * @return \Illuminate\Http\Response */ public function show($slug) { $product = Product::where('slug', '=', $slug)->get()->first(); if (empty($product)) { return response()->json([ 'message' => 'Product not found', ], 404); } // check product is in stock if ($product->quantity <= 0) { return response()->json([ 'message' => 'Product not in stock', ], 200); } return response()->json($product); }
138. Product Controller /** * Delete a product * @param $slug * @return \Illuminate\Http\JsonResponse */ public function delete($slug) { $product = Product::where('slug', '=', $slug)->get()->first(); if (empty($product)) { return response()->json([ 'message' => 'Product not found', ], 404); } $product->delete(); return response()->json([], 200); }
139. Setting Up Distributed Logging // in config/services.php 'loggly' => [ 'key' => 'MY LOGGLY TOKEN', 'tag' => 'microshop' .strtolower(env('APP_ENV')), ], // in bootstrap/app.php $app->configureMonologUsing(function($monolog) { $handler = new \Monolog\Handler\LogglyHandler(config('services.loggly.key'),\Monolog\Logger::DEBUG); $handler->setTag(config('services.loggly.tag')); $monolog->pushHandler($handler); }); Log::info('received products', $products)
140. Demo Time!
141. Superbalist War Stories
142. Superbalist War Stories • Monitoring microservices is difficult. It creates overhead/ management. For each project you need to have insight, logging and metrics. If you have multiple teams working on different services with different standards then this can be a pain point. It is important to have standardisation.
143. Superbalist War Stories • Monitoring microservices is difficult. It creates overhead/ management. For each project you need to have insight, logging and metrics. If you have multiple teams working on different services with different standards then this can be a pain point. It is important to have standardisation. • The greatest benefit from an application standpoint is separation of concerns, single purpose systems and isolation.
144. Superbalist War Stories
145. Superbalist War Stories • Microservices ultimately take longer to develop and start. A monolith isn't a bad thing when trying to meet a deadline. It is much easier to start with a monolith and migrate to a microservices based architecture.
146. Superbalist War Stories • Microservices ultimately take longer to develop and start. A monolith isn't a bad thing when trying to meet a deadline. It is much easier to start with a monolith and migrate to a microservices based architecture. • Moving to microservices greatly boosted developer productivity as each team had more responsibility over their projects and were less bogged down by obscure code in the monolith.
148. Thanks !