Slide 1

Slide 1 text

From Monolith to Microservices Liam Norman, Superbalist

Slide 2

Slide 2 text

Hi, I'm Liam @liamjnorman https:/ /www.liamnorman.com Software Engineer @ Superbalist.com Organiser of Cape Town PHP

Slide 3

Slide 3 text

About You

Slide 4

Slide 4 text

About You • Don't know what microservices are?

Slide 5

Slide 5 text

About You • Don't know what microservices are? • Have a monolith that you want to migrate?

Slide 6

Slide 6 text

About You • Don't know what microservices are? • Have a monolith that you want to migrate? • In the process of migrating a monolith?

Slide 7

Slide 7 text

About You • Don't know what microservices are? • Have a monolith that you want to migrate? • In the process of migrating a monolith? • Building a new architecture with microservices

Slide 8

Slide 8 text

About You • Don't know what microservices are? • Have a monolith that you want to migrate? • In the process of migrating a monolith? • Building a new architecture with microservices • This talk has a little bit for all of you

Slide 9

Slide 9 text

What are Microservices? Microservices at their core are small, independent, autonomous services that work together.

Slide 10

Slide 10 text

What are Microservices? Microservices at their core are small, independent, autonomous services that work together. • Microservices should be small.

Slide 11

Slide 11 text

What are Microservices? Microservices at their core are small, independent, autonomous services that work together. • Microservices should be small. • Microservices should be independent.

Slide 12

Slide 12 text

Autonomous

Slide 13

Slide 13 text

Autonomous • Microservices should be separate entities.

Slide 14

Slide 14 text

Autonomous • Microservices should be separate entities. • Tight Coupling is when you have services which rely on one another.

Slide 15

Slide 15 text

Autonomous • Microservices should be separate entities. • Tight Coupling is when you have services which rely on one another. • Avoid Tight Coupling

Slide 16

Slide 16 text

Autonomous • Microservices should be separate entities. • Tight Coupling is when you have services which rely on one another. • Avoid Tight Coupling • Microservices should communicate with one another via network calls.

Slide 17

Slide 17 text

"Can you make a change to a microservice and deploy it by itself without changing anything else?"

Slide 18

Slide 18 text

Benefits of Microservices Technology Freedom

Slide 19

Slide 19 text

Benefits of Microservices Technology Freedom • Can select the right tool for the job

Slide 20

Slide 20 text

Benefits of Microservices Technology Freedom • Can select the right tool for the job • No need to use a one-size-fits all approach as we often use for a monolith.

Slide 21

Slide 21 text

Benefits of Microservices Technology Freedom • Can select the right tool for the job • No need to use a one-size-fits all approach as we often use for a monolith. • Improved performance with correct tool!

Slide 22

Slide 22 text

Benefits of Microservices Resilience

Slide 23

Slide 23 text

Benefits of Microservices Resilience • Avoids cascading failures

Slide 24

Slide 24 text

Benefits of Microservices Resilience • Avoids cascading failures • Isolated services

Slide 25

Slide 25 text

Benefits of Microservices Resilience • Avoids cascading failures • Isolated services • Separates our application and can degrade gracefully if handled correctly.

Slide 26

Slide 26 text

Benefits of Microservices Scaling

Slide 27

Slide 27 text

Benefits of Microservices Scaling • Easier to scale as we can scale smaller independent services.

Slide 28

Slide 28 text

Benefits of Microservices Scaling • Easier to scale as we can scale smaller independent services. • Superbalist moved to microservices as we found it was important to ensure we could handle the load when a large sale occurred or Black Friday occurred.

Slide 29

Slide 29 text

Benefits of Microservices Ease of Deployment

Slide 30

Slide 30 text

Benefits of Microservices Ease of Deployment • Deploying monoliths is dangerous as they are large-impact, high risk deployments.

Slide 31

Slide 31 text

Benefits of Microservices Ease of Deployment • Deploying monoliths is dangerous as they are large-impact, high risk deployments. • Deploying microservices is safer as we deploy it independently and if we encounter a bug, we know exactly which service is causing it and can rollback.

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Benefits of Microservices Composability A system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia

Slide 34

Slide 34 text

Benefits of Microservices Composability A system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia • Composable

Slide 35

Slide 35 text

Benefits of Microservices Composability A system design principle that deals with the inter-relationships of components. A highly composable system provides components that can be selected and assembled in various combinations to satisfy specific user requirements. — Wikipedia • Composable • We no longer develop for one application.

Slide 36

Slide 36 text

No Silver Bullet

Slide 37

Slide 37 text

No Silver Bullet • Microservices are no silver bullet and don't make a good choice for everything.

Slide 38

Slide 38 text

No Silver Bullet • Microservices are no silver bullet and don't make a good choice for everything. • Microservices are great for many scenarios but they come with added complexity.

Slide 39

Slide 39 text

No Silver Bullet • Microservices are no silver bullet and don't make a good choice for everything. • Microservices are great for many scenarios but they come with added complexity. • It depends...

Slide 40

Slide 40 text

Cons of Microservices

Slide 41

Slide 41 text

Cons of Microservices • Added complexity in deployments

Slide 42

Slide 42 text

Cons of Microservices • Added complexity in deployments • Added complexity in managing logging and monitoring.

Slide 43

Slide 43 text

Cons of Microservices • Added complexity in deployments • Added complexity in managing logging and monitoring. • How to properly prevent cascading failure and degrade gracefully.

Slide 44

Slide 44 text

Modelling Microservices Let's begin!

Slide 45

Slide 45 text

Setting the Scene

Slide 46

Slide 46 text

Setting the Scene • Creating a simple order management system with a product microservices.

Slide 47

Slide 47 text

Setting the Scene • Creating a simple order management system with a product microservices. • We will be using Laravel, Docker and Loggly.

Slide 48

Slide 48 text

What Makes A Good Microservice?

Slide 49

Slide 49 text

What Makes A Good Microservice? • Loose coupling

Slide 50

Slide 50 text

What Makes A Good Microservice? • Loose coupling • High cohesion

Slide 51

Slide 51 text

Bounded Contexts Bounded contexts, What does that mean?

Slide 52

Slide 52 text

Bounded Contexts Bounded contexts, What does that mean? • Any given domain consists of multiple bounded contexts

Slide 53

Slide 53 text

Bounded Contexts Bounded contexts, What does that mean? • Any given domain consists of multiple bounded contexts • A bounded context is defined as “a specific responsibility enforced by explicit boundaries.”

Slide 54

Slide 54 text

Bounded Contexts

Slide 55

Slide 55 text

Bounded Contexts • Not sharing internal representations reduces tight coupling

Slide 56

Slide 56 text

Bounded Contexts • Not sharing internal representations reduces tight coupling • Bounded contexts lend themselves well to identifying boundaries, especially in code.

Slide 57

Slide 57 text

Bounded Contexts • Not sharing internal representations reduces tight coupling • Bounded contexts lend themselves well to identifying boundaries, especially in code. • Bounded contexts become excellent candidates for microservices.

Slide 58

Slide 58 text

Splitting the Monolith Where to start?

Slide 59

Slide 59 text

Seams

Slide 60

Slide 60 text

Seams • A seam is a portion of code that can be treated in isolation and worked on without impacting the rest of the codebase.

Slide 61

Slide 61 text

Seams • A seam is a portion of code that can be treated in isolation and worked on without impacting the rest of the codebase. • To break up a monolith, we must find seams.

Slide 62

Slide 62 text

Seams • A seam is a portion of code that can be treated in isolation and worked on without impacting the rest of the codebase. • To break up a monolith, we must find seams. • Bounded contexts make great seams because they are cohesive and loosely coupled.

Slide 63

Slide 63 text

Splitting into Packages

Slide 64

Slide 64 text

Splitting into Packages • Orders - for managing orders, order items, orders placed

Slide 65

Slide 65 text

Splitting into Packages • Orders - for managing orders, order items, orders placed

Slide 66

Slide 66 text

Splitting into Packages • Orders - for managing orders, order items, orders placed • Products - for managing products, product stock

Slide 67

Slide 67 text

Splitting into Packages • Orders - for managing orders, order items, orders placed • Products - for managing products, product stock

Slide 68

Slide 68 text

Splitting into Packages • Orders - for managing orders, order items, orders placed • Products - for managing products, product stock • Users - for managing users and user information

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

Packages in PHP namespace App\Http\Controllers; class ProductController extends Controller { ... }

Slide 71

Slide 71 text

Slow Down

Slide 72

Slide 72 text

Slow Down • This process could be quick or several weeks, months etc...

Slide 73

Slide 73 text

Slow Down • This process could be quick or several weeks, months etc... • Rather look at splitting the codebase accurately into viable packages.

Slide 74

Slide 74 text

Evolutionary Architecture and Microservices

Slide 75

Slide 75 text

Evolutionary Architecture and Microservices • It is difficult to change architecture once implemented.

Slide 76

Slide 76 text

Evolutionary Architecture and Microservices • It is difficult to change architecture once implemented. • Evolutionary architecture is a terminology created by Rebeca Parsons, Patrick Kua and Neal Ford. It refers to architecture which has the ability to change over time.

Slide 77

Slide 77 text

Evolutionary Architecture and Microservices • It is difficult to change architecture once implemented. • Evolutionary architecture is a terminology created by Rebeca Parsons, Patrick Kua and Neal Ford. It refers to architecture which has the ability to change over time. • Microservices enables this in an incremental way.

Slide 78

Slide 78 text

Evolutionary Architecture and Microservices

Slide 79

Slide 79 text

Methodology to Split Out the Microservice

Slide 80

Slide 80 text

Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams

Slide 81

Slide 81 text

Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages

Slide 82

Slide 82 text

Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality

Slide 83

Slide 83 text

Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality 4. Direct traffic to microservices

Slide 84

Slide 84 text

Methodology to Split Out the Microservice 1. Establish the bounded contexts and the seams 2. Identify code to split out and group into Packages 3. Create the Microservice to accomplish same functionality 4. Direct traffic to microservices 5. Retire old code in Monolith

Slide 85

Slide 85 text

Choosing Your First Microservice

Slide 86

Slide 86 text

Choosing Your First Microservice • Pace of change

Slide 87

Slide 87 text

Choosing Your First Microservice • Pace of change • Isolation

Slide 88

Slide 88 text

Choosing Your First Microservice • Pace of change • Isolation • Risk

Slide 89

Slide 89 text

Tangled Dependencies • http:/ /stan4j.com/

Slide 90

Slide 90 text

Splitting Databases

Slide 91

Slide 91 text

Problems with the Monolithic Database Design

Slide 92

Slide 92 text

Problems with the Monolithic Database Design • A monolith DB is a single point of failure

Slide 93

Slide 93 text

Problems with the Monolithic Database Design • A monolith DB is a single point of failure • Bad for data retrieval and performance over time

Slide 94

Slide 94 text

Problems with the Monolithic Database Design • A monolith DB is a single point of failure • Bad for data retrieval and performance over time • Constraints to one type of database

Slide 95

Slide 95 text

Database Implementation in a Microservice-oriented Architecture.

Slide 96

Slide 96 text

Finding seams in your Database

Slide 97

Slide 97 text

Finding seams in your Database • Equally important as seams in application layer

Slide 98

Slide 98 text

Finding seams in your Database • Equally important as seams in application layer • Ensure each microservice has the data they need and doesn't need to make unnecessary API calls

Slide 99

Slide 99 text

Installing SchemaSpy apt-get install graphviz java -jar schemaspy.jar -t mysql -dp /users/liamnorman/sqljdbc4-3.0.jar -db products -host 127.0.0.1 -port 3306 -u testuser -p mygreatpassword -o /users/liamnorman/schemaspyoutput

Slide 100

Slide 100 text

No content

Slide 101

Slide 101 text

The Database Split

Slide 102

Slide 102 text

The Database Split • Break parts of your database into smaller databases

Slide 103

Slide 103 text

The Database Split • Break parts of your database into smaller databases • Easy to revert, only retire old tables after split and microservice is running

Slide 104

Slide 104 text

The Database Split • Break parts of your database into smaller databases • Easy to revert, only retire old tables after split and microservice is running • Split database first

Slide 105

Slide 105 text

Breaking Foreign Key Relationships

Slide 106

Slide 106 text

Breaking Foreign Key Relationships

Slide 107

Slide 107 text

Data Integrity and Event Sourcing

Slide 108

Slide 108 text

Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc...

Slide 109

Slide 109 text

Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture.

Slide 110

Slide 110 text

Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture. • Multiple PubSub providers such as Google PubSub, AWS SNS, Socket Cluster etc...

Slide 111

Slide 111 text

Data Integrity and Event Sourcing • There are multiple approaches to this problem such as event bus, distributed transactions, syncs etc... • I recommend an Event Bus (event sourcing) approach - each service sends events for its changes and the interested parties listen and save the info as needed. This is often implemented with PubSub and events-based architecture. • Multiple PubSub providers such as Google PubSub, AWS SNS, Socket Cluster etc...

Slide 112

Slide 112 text

Data Integrity and Event Sourcing

Slide 113

Slide 113 text

DevOps Tips for Microservices

Slide 114

Slide 114 text

DevOps Tips for Microservices • Distributed Logging

Slide 115

Slide 115 text

DevOps Tips for Microservices • Distributed Logging • Correlation IDs

Slide 116

Slide 116 text

DevOps Tips for Microservices

Slide 117

Slide 117 text

DevOps Tips for Microservices • Searchable Logs

Slide 118

Slide 118 text

DevOps Tips for Microservices • Searchable Logs • Monitoring and monitor networks between services.

Slide 119

Slide 119 text

Crafting our Microservices Application

Slide 120

Slide 120 text

Crafting our Microservices Application • We are going to build a simple order management system with a product Microservice in PHP 7 using Laravel 5.

Slide 121

Slide 121 text

Crafting our Microservices Application • We are going to build a simple order management system with a product Microservice in PHP 7 using Laravel 5. • We are going to focus on the product Microservice in the system.

Slide 122

Slide 122 text

App Toolbox

Slide 123

Slide 123 text

App Toolbox • PHP 7

Slide 124

Slide 124 text

App Toolbox • PHP 7 • Composer

Slide 125

Slide 125 text

App Toolbox • PHP 7 • Composer • Laravel 5

Slide 126

Slide 126 text

App Toolbox • PHP 7 • Composer • Laravel 5 • Docker

Slide 127

Slide 127 text

App Toolbox • PHP 7 • Composer • Laravel 5 • Docker • Monolog

Slide 128

Slide 128 text

App Toolbox • PHP 7 • Composer • Laravel 5 • Docker • Monolog • Loggly

Slide 129

Slide 129 text

Setting Up Our Product Microservice composer global require laravel/installer laravel new microshop

Slide 130

Slide 130 text

Setting up Docker version: "3" services: app: image: laravel-www container_name: product-api build: context: . dockerfile: docker/Dockerfile depends_on: - mysql env_file: - .docker.env volumes: - .:/var/www/html networks: - frontend_oms ports: - 8081:80 environment: APP_ENV: local CONTAINER_ROLE: app

Slide 131

Slide 131 text

Setting up Docker mysql: container_name: product-mysql image: mysql:5.7 ports: - 13307:3306 volumes: - mysql:/var/lib/mysql environment: MYSQL_DATABASE: homestead MYSQL_ROOT_PASSWORD: root MYSQL_USER: homestead MYSQL_PASSWORD: secret volumes: mysql: driver: "local" networks: frontend_oms: external: true

Slide 132

Slide 132 text

Docker Up and Running docker-compose build && docker- compose up -d e1576be47250 laravel-www "docker-php-entrypoi…" 4 seconds ago Up 2 seconds 9000/tcp, 0.0.0.0:8081->80/tcp product-api 474a6cb3928f mysql:5.7 "docker-entrypoint.s…" 7 minutes ago Up 4 seconds 0.0.0.0:13307->3306/tcp product-mysql 097314bbe4f9 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 80/tcp, 9000/tcp frontend-scheduler a8fb0eac2298 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 80/tcp, 9000/tcp frontend-queue 2d62d7fa1b7c 80c639ead8df "docker-php-entrypoi…" 8 minutes ago Up 8 minutes 9000/tcp, 0.0.0.0:8080->80/tcp frontend-www 349b26ec6e4a mysql:5.7 "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:13306->3306/tcp frontend-mysql b2a42fd78a1d redis:4-alpine "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 0.0.0.0:16379->6379/tcp frontend-redis

Slide 133

Slide 133 text

Product Models php artisan make:model Product -mcr

Slide 134

Slide 134 text

Product Migrations increments('id'); $table->string('name'); $table->string('slug'); $table->bigInteger('price'); $table->integer('quantity'); $table->string('image'); $table->timestamps(); }); } public function down() { Schema::dropIfExists('products'); } }

Slide 135

Slide 135 text

Routes // Routes for Product CRUD Route::get('/products', 'ProductController@index')->name('products.index'); Route::post('/products', 'ProductController@store')->name('products.store'); Route::get('/products/{slug}', 'ProductController@show')->name('products.show'); Route::get('/products/{slug}/delete', 'ProductController@delete')->name('products.delete');

Slide 136

Slide 136 text

Product Controller json($products); } }

Slide 137

Slide 137 text

Product Controller /** * Display the specified resource. * * @param string $slug * @return \Illuminate\Http\Response */ public function show($slug) { $product = Product::where('slug', '=', $slug)->get()->first(); if (empty($product)) { return response()->json([ 'message' => 'Product not found', ], 404); } // check product is in stock if ($product->quantity <= 0) { return response()->json([ 'message' => 'Product not in stock', ], 200); } return response()->json($product); }

Slide 138

Slide 138 text

Product Controller /** * Delete a product * @param $slug * @return \Illuminate\Http\JsonResponse */ public function delete($slug) { $product = Product::where('slug', '=', $slug)->get()->first(); if (empty($product)) { return response()->json([ 'message' => 'Product not found', ], 404); } $product->delete(); return response()->json([], 200); }

Slide 139

Slide 139 text

Setting Up Distributed Logging // in config/services.php 'loggly' => [ 'key' => 'MY LOGGLY TOKEN', 'tag' => 'microshop' .strtolower(env('APP_ENV')), ], // in bootstrap/app.php $app->configureMonologUsing(function($monolog) { $handler = new \Monolog\Handler\LogglyHandler(config('services.loggly.key'),\Monolog\Logger::DEBUG); $handler->setTag(config('services.loggly.tag')); $monolog->pushHandler($handler); }); Log::info('received products', $products)

Slide 140

Slide 140 text

Demo Time!

Slide 141

Slide 141 text

Superbalist War Stories

Slide 142

Slide 142 text

Superbalist War Stories • Monitoring microservices is difficult. It creates overhead/ management. For each project you need to have insight, logging and metrics. If you have multiple teams working on different services with different standards then this can be a pain point. It is important to have standardisation.

Slide 143

Slide 143 text

Superbalist War Stories • Monitoring microservices is difficult. It creates overhead/ management. For each project you need to have insight, logging and metrics. If you have multiple teams working on different services with different standards then this can be a pain point. It is important to have standardisation. • The greatest benefit from an application standpoint is separation of concerns, single purpose systems and isolation.

Slide 144

Slide 144 text

Superbalist War Stories

Slide 145

Slide 145 text

Superbalist War Stories • Microservices ultimately take longer to develop and start. A monolith isn't a bad thing when trying to meet a deadline. It is much easier to start with a monolith and migrate to a microservices based architecture.

Slide 146

Slide 146 text

Superbalist War Stories • Microservices ultimately take longer to develop and start. A monolith isn't a bad thing when trying to meet a deadline. It is much easier to start with a monolith and migrate to a microservices based architecture. • Moving to microservices greatly boosted developer productivity as each team had more responsibility over their projects and were less bogged down by obscure code in the monolith.

Slide 147

Slide 147 text

No content

Slide 148

Slide 148 text

Thanks !