Tweet
Tweet

Slide 1

Slide 1 text

Distributed microservices in the real world By Imraan Parker

Slide 2

Slide 2 text

Python 3.6.6 [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print(objective) print(about) >>> The purpose of this presentation is to pass on our knowledge regarding out implementation of microservices to empower you to make an informed decision based on your own situation. Hi PyConZA. My role is Head of Product. I code to make a difference.

Slide 3

Slide 3 text

@imraanparker Background  CareerJunction (https://www.careerjunction.co.za) is 21 this year  We build products to bring together those looking for jobs and those supplying them  We work primarily in Python to  Perform data capture, mining and analysis  Provide powerful tools for recruitment  Our technology stack is diverse

Slide 4

Slide 4 text

@imraanparker Agenda  Our journey from monolith to microservices  Modern software development principles  Why choose a microservices architecture?  Explore the CareerJunction microservices framework  Requirements  Conceptual architecture  Framework components  Features by coding and running a service  The benefits and considerations  IT and business benefits  What to consider and avoid?

Slide 5

Slide 5 text

@imraanparker Modern Software Development  Resources  How teams are organised  Reporting structure  Processes  How to respond to requirements  How to react to changes  Architecture  How to build software  Standards & guidelines

Slide 6

Slide 6 text

Modern Software Development Architecture Processes Resources Deliver often React faster to change Small autonomous teams

Slide 7

Slide 7 text

Architecture?

Slide 8

Slide 8 text

@imraanparker Application Monolithic Architecture  Easy to  Develop  Test  Deploy  Scale  Over time  Codebase grew  We saw diminishing returns Web UI Web UI Search Module Search Module Job Module Job Module User Module User Module DB HTML HTTP API

Slide 9

Slide 9 text

Monolithic Architecture Monolithic Architecture Processes Resources Deliver often React faster to change Small autonomous teams

Slide 10

Slide 10 text

@imraanparker Monolithic Architecture  What to do?  Break up the monolithic application, and  Implement microservices  Unfortunately we have to keep the planes in the air  Feature development trumps architecture development  We investigated different options, but ultimately came up short  We tried an approach that ultimately failed to live up to expectations  Broke up the monolithic application, into  Smaller applications

Slide 11

Slide 11 text

@imraanparker Web UI Web UI DB HTML HTTP API Search Search Job Job User User HTTP API No control over application data Extra overhead

Slide 12

Slide 12 text

@imraanparker Application Layers User Pylons WSGI Apache

Slide 13

Slide 13 text

@imraanparker Application Layers  Other Issues  Setup and configuration  Request monitoring  Troubleshooting User Pylons WSGI Apache

Slide 14

Slide 14 text

“ ” Insanity… Doing the same thing over and over again and expecting different results. It took us two years to finally bite the bullet and take the time to do it properly

Slide 15

Slide 15 text

@imraanparker Microservices Architecture  We dedicated resources to the project  Investigated available options in the market  Score each based on our requirements  Pickings were slim  A few were feature rich, but expensive and/or verbose  Others had some of the features, meaning we needed two or more systems combined

Slide 16

Slide 16 text

Build it and they will come

Slide 17

Slide 17 text

@imraanparker Microservices Architecture Web UI Web UI DB HTML RPC DB DB Search Service Search Service Job Service Job Service User Service User Service Gateway Gateway HTTP API RPC RPC

Slide 18

Slide 18 text

Microservices Architecture Microservices Architecture Processes Resources Deliver often React faster to change Small autonomous teams  

Slide 19

Slide 19 text

@imraanparker Summary – Our Journey  The monolith is not a bad idea  In the scope of a large scale application, microservices is better  If you are not prepared to do it right the first time, be prepared to do it twice.

Slide 20

Slide 20 text

CareerJunction Framework

Slide 21

Slide 21 text

@imraanparker Requirements  Developer adoption  Increase productivity  Language agnostic  Scalable  Highly available  Discoverable services

Slide 22

Slide 22 text

@imraanparker Requirements  Message contracts  Automated documentation  Task scheduling  Monitoring  Debugging

Slide 23

Slide 23 text

@imraanparker Conceptual Architecture  Technologies  Python 3 (3.4+)  ZeroMQ (http://zeromq.org/)  Messaging  Reliable request-reply dialog  Based on the Majordomo pattern (https://rfc.zeromq.org/spec:7/MDP/)  JSON for serialisation

Slide 24

Slide 24 text

@imraanparker Conceptual Architecture Client Client Client Broker Broker Broker Broker Worker Worker tea Worker Worker coffee Worker Worker water

Slide 25

Slide 25 text

@imraanparker Conceptual Architecture Client Client Client Broker Broker Broker Broker Worker Worker tea Worker Worker coffee Worker Worker water I want tea

Slide 26

Slide 26 text

@imraanparker Conceptual Architecture Client Client Client Broker Broker Broker Broker Worker Worker tea Worker Worker coffee Worker Worker water I want tea

Slide 27

Slide 27 text

@imraanparker Conceptual Architecture Client Client Client Broker Broker Broker Broker Worker Worker tea Worker Worker coffee Worker Worker water I want tea

Slide 28

Slide 28 text

@imraanparker Conceptual Architecture Client Client Client Broker Broker Broker Broker Worker Worker tea Worker Worker coffee Worker Worker water I want tea

Slide 29

Slide 29 text

@imraanparker Conceptual Architecture Client Client Client Broker Broker Broker Broker Worker Worker tea Worker Worker coffee Worker Worker water I want tea

Slide 30

Slide 30 text

@imraanparker Components - Broker  Responsible for  Communication between clients and workers  Exposing services via HTTP  Distribution of requests to workers  Logging of requests

Slide 31

Slide 31 text

@imraanparker Components - Broker  To accomplish this the broker consists of four sub systems  Transport  Queue  Logger  Heartbeat Client Worker Worker tea Broker Transport Queue Logger Heartbeat

Slide 32

Slide 32 text

@imraanparker Components - Broker  Managing connections  Handling data transfer  Broadcasting broker’s presence Client Worker Worker tea Broker Transport Queue Logger Heartbeat

Slide 33

Slide 33 text

@imraanparker Components - Broker  Request distribution  Round-robin fashion  Handles queues for services Client Worker Worker tea Broker Transport Queue Logger Heartbeat

Slide 34

Slide 34 text

@imraanparker Components - Broker  Records all requests  Received -> processing -> complete  Exposes a dashboard for monitoring and debugging Client Worker Worker tea Broker Transport Queue Logger Heartbeat

Slide 35

Slide 35 text

@imraanparker Components - Broker  Monitors connections between clients and workers  Disconnects peers on heartbeat failure Client Worker Worker tea Broker Transport Queue Logger Heartbeat

Slide 36

Slide 36 text

@imraanparker Components - Worker  Responsible for  Connecting to brokers by listening for multicast messages  Registering services (including documentation) with brokers  Support scaling using threads  Processing requests

Slide 37

Slide 37 text

@imraanparker Components - Client  Responsible for  Exposing a RPC type syntax for calling services  Encode and decoding language specific data types  Sending requests to the broker and handling replies  Raising exceptions in a language specific manner on failure

Slide 38

Slide 38 text

Coding a Service

Slide 39

Slide 39 text

@imraanparker Definitions  A worker is an instance of a service  A service is a collection of microservices  A microservice is a method

Slide 40

Slide 40 text

Service Definition

Slide 41

Slide 41 text

Service Definition

Slide 42

Slide 42 text

Calling the Service

Slide 43

Slide 43 text

Service Scaling

Slide 44

Slide 44 text

Asynchronous

Slide 45

Slide 45 text

Minimum Timeout

Slide 46

Slide 46 text

Exposing via HTTP

Slide 47

Slide 47 text

Scheduled Jobs

Slide 48

Slide 48 text

@imraanparker Dashboard  Overall health of the system  Monitor and debug requests  Requests  Summary  Failed  In Progress  Successful

Slide 49

Slide 49 text

@imraanparker Summary

Slide 50

Slide 50 text

@imraanparker Failed Requests

Slide 51

Slide 51 text

Request Information

Slide 52

Slide 52 text

Request Information

Slide 53

Slide 53 text

Request Information

Slide 54

Slide 54 text

@imraanparker In Progress

Slide 55

Slide 55 text

@imraanparker Successful

Slide 56

Slide 56 text

@imraanparker Service Documentation

Slide 57

Slide 57 text

@imraanparker Scheduled Jobs

Slide 58

Slide 58 text

@imraanparker HTTP API

Slide 59

Slide 59 text

Demo

Slide 60

Slide 60 text

@imraanparker Wrap-up  Benefits  Four fold increase in productivity  Lower IT cost  Infrastructure  Maintenance  Human Resources  Reflections  Microservices is best suited for large scale applications  Do it right the first time  Learn from your mistakes, but importantly from ours

Slide 61

Slide 61 text

@imraanparker Getting Involved https://github.com/ careerjunction

Slide 62

Slide 62 text

@imraanparker Final Thoughts  How to deploy services?  How to deal with distributed data?  How to manage feature development that extends over multiple teams?

Slide 63

Slide 63 text

Thank you

Slide 64

Slide 64 text

This will all end in tears. I just know it.