Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Monolith to Microservices

From Monolith to Microservices

In this talk, We will focus on how to split up an old enterprise application which has become a monolith into microservices. We will examine the pros and cons of doing so and also the best design practices and architecture decisions that will ensure this process is a success.

We will mainly explore how to identify services in your monolithic application, identify seams in your monolith. We will also investigate how to decouple microservices from monoliths and the best practices when doing so. We will also focus on microservices development and DevOps tips.

We will be examining how to extract a monolith application using example PHP microservices that were pulled out of a larger legacy PHP application. I will also investigate some war stories on how Superbalist achieved this and the lessons learned along the way.

At the end of the talk, you will know better how to refactor your monolith and improve your own architecture.

Liam Norman

March 28, 2019
Tweet

More Decks by Liam Norman

Other Decks in Technology

Transcript

  1. From Monolith to Microservices Liam Norman, Superbalist

  2. Hi, I'm Liam @liamjnorman https:/ /www.liamnorman.com Software Engineer @ Superbalist.com

    Organiser of Cape Town PHP
  3. About You

  4. About You • Don't know what microservices are?

  5. About You • Don't know what microservices are? • Have

    a monolith that you want to migrate?
  6. About You • Don't know what microservices are? • Have

    a monolith that you want to migrate? • In the process of migrating a monolith?
  7. About You • Don't know what microservices are? • Have

    a monolith that you want to migrate? • In the process of migrating a monolith? • Building a new architecture with microservices
  8. About You • Don't know what microservices are? • Have

    a monolith that you want to migrate? • In the process of migrating a monolith? • Building a new architecture with microservices • This talk has a little bit for all of you
  9. What are Microservices? Microservices at their core are small, independent,

    autonomous services that work together.
  10. What are Microservices? Microservices at their core are small, independent,

    autonomous services that work together. • Microservices should be small.
  11. What are Microservices? Microservices at their core are small, independent,

    autonomous services that work together. • Microservices should be small. • Microservices should be independent.
  12. Autonomous

  13. Autonomous • Microservices should be separate entities.

  14. Autonomous • Microservices should be separate entities. • Tight Coupling

    is when you have services which rely on one another.
  15. Autonomous • Microservices should be separate entities. • Tight Coupling

    is when you have services which rely on one another. • Avoid Tight Coupling
  16. Autonomous • Microservices should be separate entities. • Tight Coupling

    is when you have services which rely on one another. • Avoid Tight Coupling • Microservices should communicate with one another via network calls.
  17. "Can you make a change to a microservice and deploy

    it by itself without changing anything else?"
  18. Benefits of Microservices Technology Freedom

  19. Benefits of Microservices Technology Freedom • Can select the right

    tool for the job
  20. Benefits of Microservices Technology Freedom • Can select the right

    tool for the job • No need to use a one-size-fits all approach as we often use for a monolith.
  21. Benefits of Microservices Technology Freedom • Can select the right

    tool for the job • No need to use a one-size-fits all approach as we often use for a monolith. • Improved performance with correct tool!
  22. Benefits of Microservices Resilience

  23. Benefits of Microservices Resilience • Avoids cascading failures

  24. Benefits of Microservices Resilience • Avoids cascading failures • Isolated

    services
  25. Benefits of Microservices Resilience • Avoids cascading failures • Isolated

    services • Separates our application and can degrade gracefully if handled correctly.
  26. Benefits of Microservices Scaling

  27. Benefits of Microservices Scaling • Easier to scale as we

    can scale smaller independent services.
  28. Benefits of Microservices Scaling • Easier to scale as we

    can scale smaller independent services. • Superbalist moved to microservices as we found it was important to ensure we could handle the load when a large sale occurred or Black Friday occurred.
  29. Benefits of Microservices Ease of Deployment

  30. Benefits of Microservices Ease of Deployment • Deploying monoliths is

    dangerous as they are large-impact, high risk deployments.
  31. Benefits of Microservices Ease of Deployment • Deploying monoliths is

    dangerous as they are large-impact, high risk deployments. • Deploying microservices is safer as we deploy it independently and if we encounter a bug, we know exactly which service is causing it and can rollback.
  32. None
  33. Benefits of Microservices Composability A system design principle that deals

    with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia
  34. Benefits of Microservices Composability A system design principle that deals

    with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia • Composable
  35. Benefits of Microservices Composability A system design principle that deals

    with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia • Composable • We no longer develop for one application.
  36. No Silver Bullet

  37. No Silver Bullet • Microservices are no silver bullet and

    don't make a good choice for everything.
  38. No Silver Bullet • Microservices are no silver bullet and

    don't make a good choice for everything. • Microservices are great for many scenarios but they come with added complexity.
  39. No Silver Bullet • Microservices are no silver bullet and

    don't make a good choice for everything. • Microservices are great for many scenarios but they come with added complexity. • It depends...
  40. Cons of Microservices

  41. Cons of Microservices • Added complexity in deployments

  42. Cons of Microservices • Added complexity in deployments • Added

    complexity in managing logging and monitoring.
  43. Cons of Microservices • Added complexity in deployments • Added

    complexity in managing logging and monitoring. • How to properly prevent cascading failure and degrade gracefully.
  44. Modelling Microservices Let's begin!

  45. Setting the Scene

  46. Setting the Scene • Creating a simple order management system

    with a product microservices.
  47. Setting the Scene • Creating a simple order management system

    with a product microservices. • We will be using Laravel, Docker and Loggly.
  48. What Makes A Good Microservice?

  49. What Makes A Good Microservice? • Loose coupling

  50. What Makes A Good Microservice? • Loose coupling • High

    cohesion
  51. Bounded Contexts Bounded contexts, What does that mean?

  52. Bounded Contexts Bounded contexts, What does that mean? • Any

    given domain consists of multiple bounded contexts
  53. Bounded Contexts Bounded contexts, What does that mean? • Any

    given domain consists of multiple bounded contexts • A bounded context is defined as “a specific responsibility enforced by explicit boundaries.”
  54. Bounded Contexts

  55. Bounded Contexts • Not sharing internal representations reduces tight coupling

  56. Bounded Contexts • Not sharing internal representations reduces tight coupling

    • Bounded contexts lend themselves well to identifying boundaries, especially in code.
  57. Bounded Contexts • Not sharing internal representations reduces tight coupling

    • Bounded contexts lend themselves well to identifying boundaries, especially in code. • Bounded contexts become excellent candidates for microservices.
  58. Splitting the Monolith Where to start?

  59. Seams

  60. Seams • A seam is a portion of code that

    can be treated in isolation and worked on without impacting the rest of the codebase.
  61. Seams • A seam is a portion of code that

    can be treated in isolation and worked on without impacting the rest of the codebase. • To break up a monolith, we must find seams.
  62. Seams • A seam is a portion of code that

    can be treated in isolation and worked on without impacting the rest of the codebase. • To break up a monolith, we must find seams. • Bounded contexts make great seams because they are cohesive and loosely coupled.
  63. Splitting into Packages

  64. Splitting into Packages • Orders - for managing orders, order

    items, orders placed
  65. Splitting into Packages • Orders - for managing orders, order

    items, orders placed
  66. Splitting into Packages • Orders - for managing orders, order

    items, orders placed • Products - for managing products, product stock
  67. Splitting into Packages • Orders - for managing orders, order

    items, orders placed • Products - for managing products, product stock
  68. Splitting into Packages • Orders - for managing orders, order

    items, orders placed • Products - for managing products, product stock • Users - for managing users and user information
  69. None
  70. Packages in PHP namespace App\Http\Controllers; class ProductController extends Controller {

    ... }
  71. Slow Down

  72. Slow Down • This process could be quick or several

    weeks, months etc...
  73. Slow Down • This process could be quick or several

    weeks, months etc... • Rather look at splitting the codebase accurately into viable packages.
  74. Evolutionary Architecture and Microservices

  75. Evolutionary Architecture and Microservices • It is difficult to change

    architecture once implemented.
  76. Evolutionary Architecture and Microservices • It is difficult to change

    architecture once implemented. • Evolutionary architecture is a terminology created by Rebeca Parsons, Patrick Kua and Neal Ford. It refers to architecture which has the ability to change over time.
  77. Evolutionary Architecture and Microservices • It is difficult to change

    architecture once implemented. • Evolutionary architecture is a terminology created by Rebeca Parsons, Patrick Kua and Neal Ford. It refers to architecture which has the ability to change over time. • Microservices enables this in an incremental way.
  78. Evolutionary Architecture and Microservices

  79. Methodology to Split Out the Microservice

  80. Methodology to Split Out the Microservice 1. Establish the bounded

    contexts and the seams
  81. Methodology to Split Out the Microservice 1. Establish the bounded

    contexts and the seams 2. Identify code to split out and group into Packages
  82. Methodology to Split Out the Microservice 1. Establish the bounded

    contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality
  83. Methodology to Split Out the Microservice 1. Establish the bounded

    contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality 4. Direct traffic to microservices
  84. Methodology to Split Out the Microservice 1. Establish the bounded

    contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality 4. Direct traffic to microservices 5. Retire old code in Monolith
  85. Choosing Your First Microservice

  86. Choosing Your First Microservice • Pace of change

  87. Choosing Your First Microservice • Pace of change • Isolation

  88. Choosing Your First Microservice • Pace of change • Isolation

    • Risk
  89. Tangled Dependencies • http:/ /stan4j.com/

  90. Splitting Databases

  91. Problems with the Monolithic Database Design

  92. Problems with the Monolithic Database Design • A monolith DB

    is a single point of failure
  93. Problems with the Monolithic Database Design • A monolith DB

    is a single point of failure • Bad for data retrieval and performance over time
  94. Problems with the Monolithic Database Design • A monolith DB

    is a single point of failure • Bad for data retrieval and performance over time • Constraints to one type of database
  95. Database Implementation in a Microservice-oriented Architecture.

  96. Finding seams in your Database

  97. Finding seams in your Database • Equally important as seams

    in application layer
  98. Finding seams in your Database • Equally important as seams

    in application layer • Ensure each microservice has the data they need and doesn't need to make unnecessary API calls
  99. Installing SchemaSpy apt-get install graphviz java -jar schemaspy.jar -t mysql

    -dp /users/liamnorman/sqljdbc4-3.0.jar -db products -host 127.0.0.1 -port 3306 -u testuser -p mygreatpassword -o /users/liamnorman/schemaspyoutput
  100. None
  101. The Database Split

  102. The Database Split • Break parts of your database into

    smaller databases
  103. The Database Split • Break parts of your database into

    smaller databases • Easy to revert, only retire old tables after split and microservice is running
  104. The Database Split • Break parts of your database into

    smaller databases • Easy to revert, only retire old tables after split and microservice is running • Split database first
  105. Breaking Foreign Key Relationships

  106. Breaking Foreign Key Relationships

  107. Data Integrity and Event Sourcing

  108. Data Integrity and Event Sourcing • There are multiple approaches

    to this problem such as event bus, distributed transactions, syncs etc...
  109. Data Integrity and Event Sourcing • There are multiple approaches

    to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture.
  110. Data Integrity and Event Sourcing • There are multiple approaches

    to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture. • Multiple PubSub providers such as Google PubSub, AWS SNS, Socket Cluster etc...
  111. Data Integrity and Event Sourcing • There are multiple approaches

    to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture. • Multiple PubSub providers such as Google PubSub, AWS SNS, Socket Cluster etc...
  112. Data Integrity and Event Sourcing

  113. DevOps Tips for Microservices

  114. DevOps Tips for Microservices • Distributed Logging

  115. DevOps Tips for Microservices • Distributed Logging • Correlation IDs

  116. DevOps Tips for Microservices

  117. DevOps Tips for Microservices • Searchable Logs

  118. DevOps Tips for Microservices • Searchable Logs • Monitoring and

    monitor networks between services.
  119. Crafting our Microservices Application

  120. Crafting our Microservices Application • We are going to build

    a simple order management system with a product Microservice in PHP 7 using Laravel 5.
  121. Crafting our Microservices Application • We are going to build

    a simple order management system with a product Microservice in PHP 7 using Laravel 5. • We are going to focus on the product Microservice in the system.
  122. App Toolbox

  123. App Toolbox • PHP 7

  124. App Toolbox • PHP 7 • Composer

  125. App Toolbox • PHP 7 • Composer • Laravel 5

  126. App Toolbox • PHP 7 • Composer • Laravel 5

    • Docker
  127. App Toolbox • PHP 7 • Composer • Laravel 5

    • Docker • Monolog
  128. App Toolbox • PHP 7 • Composer • Laravel 5

    • Docker • Monolog • Loggly
  129. Setting Up Our Product Microservice composer global require laravel/installer laravel

    new microshop
  130. Setting up Docker version: "3" services: app: image: laravel-www container_name:

    product-api build: context: . dockerfile: docker/Dockerfile depends_on: - mysql env_file: - .docker.env volumes: - .:/var/www/html networks: - frontend_oms ports: - 8081:80 environment: APP_ENV: local CONTAINER_ROLE: app
  131. Setting up Docker mysql: container_name: product-mysql image: mysql:5.7 ports: -

    13307:3306 volumes: - mysql:/var/lib/mysql environment: MYSQL_DATABASE: homestead MYSQL_ROOT_PASSWORD: root MYSQL_USER: homestead MYSQL_PASSWORD: secret volumes: mysql: driver: "local" networks: frontend_oms: external: true
  132. Docker Up and Running docker-compose build && docker- compose up

    -d e1576be47250 laravel-www "docker-php-entrypoi…" 4 seconds ago Up 2 seconds 9000/tcp, 0.0.0.0:8081->80/tcp product-api 474a6cb3928f mysql:5.7 "docker-entrypoint.s…" 7 minutes ago Up 4 seconds 0.0.0.0:13307->3306/tcp product-mysql 097314bbe4f9 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 80/tcp, 9000/tcp frontend-scheduler a8fb0eac2298 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 80/tcp, 9000/tcp frontend-queue 2d62d7fa1b7c 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 9000/tcp, 0.0.0.0:8080->80/tcp frontend-www 349b26ec6e4a mysql:5.7 "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:13306->3306/tcp frontend-mysql b2a42fd78a1d redis:4-alpine "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:16379->6379/tcp frontend-redis
  133. Product Models php artisan make:model Product -mcr

  134. Product Migrations <?php use Illuminate\Support\Facades\Schema; use Illuminate\Database\Schema\Blueprint; use Illuminate\Database\Migrations\Migration; class

    CreateProductsTable extends Migration { public function up() { Schema::create('products', function (Blueprint $table) { $table->increments('id'); $table->string('name'); $table->string('slug'); $table->bigInteger('price'); $table->integer('quantity'); $table->string('image'); $table->timestamps(); }); } public function down() { Schema::dropIfExists('products'); } }
  135. Routes // Routes for Product CRUD Route::get('/products', 'ProductController@index')->name('products.index'); Route::post('/products', 'ProductController@store')->name('products.store');

    Route::get('/products/{slug}', 'ProductController@show')->name('products.show'); Route::get('/products/{slug}/delete', 'ProductController@delete')->name('products.delete');
  136. Product Controller <?php namespace App\Http\Controllers; use App\Product; use Illuminate\Http\Request; class

    ProductController extends Controller { /** * Display a listing of the resource. * * @return \Illuminate\Http\Response */ public function index() { $products = Product::all(); return response()->json($products); } }
  137. Product Controller /** * Display the specified resource. * *

    @param string $slug * @return \Illuminate\Http\Response */ public function show($slug) { $product = Product::where('slug', '=', $slug)->get()->first(); if (empty($product)) { return response()->json([ 'message' => 'Product not found', ], 404); } // check product is in stock if ($product->quantity <= 0) { return response()->json([ 'message' => 'Product not in stock', ], 200); } return response()->json($product); }
  138. Product Controller /** * Delete a product * @param $slug

    * @return \Illuminate\Http\JsonResponse */ public function delete($slug) { $product = Product::where('slug', '=', $slug)->get()->first(); if (empty($product)) { return response()->json([ 'message' => 'Product not found', ], 404); } $product->delete(); return response()->json([], 200); }
  139. Setting Up Distributed Logging // in config/services.php 'loggly' => [

    'key' => 'MY LOGGLY TOKEN', 'tag' => 'microshop' .strtolower(env('APP_ENV')), ], // in bootstrap/app.php $app->configureMonologUsing(function($monolog) { $handler = new \Monolog\Handler\LogglyHandler(config('services.loggly.key'),\Monolog\Logger::DEBUG); $handler->setTag(config('services.loggly.tag')); $monolog->pushHandler($handler); }); Log::info('received products', $products)
  140. Demo Time!

  141. Superbalist War Stories

  142. Superbalist War Stories • Monitoring microservices is difficult. It creates

    overhead/ management. For each project you need to have insight, logging and metrics. If you have multiple teams working on different services with different standards then this can be a pain point. It is important to have standardisation.
  143. Superbalist War Stories • Monitoring microservices is difficult. It creates

    overhead/ management. For each project you need to have insight, logging and metrics. If you have multiple teams working on different services with different standards then this can be a pain point. It is important to have standardisation. • The greatest benefit from an application standpoint is separation of concerns, single purpose systems and isolation.
  144. Superbalist War Stories

  145. Superbalist War Stories • Microservices ultimately take longer to develop

    and start. A monolith isn't a bad thing when trying to meet a deadline. It is much easier to start with a monolith and migrate to a microservices based architecture.
  146. Superbalist War Stories • Microservices ultimately take longer to develop

    and start. A monolith isn't a bad thing when trying to meet a deadline. It is much easier to start with a monolith and migrate to a microservices based architecture. • Moving to microservices greatly boosted developer productivity as each team had more responsibility over their projects and were less bogged down by obscure code in the monolith.
  147. None
  148. Thanks !