Tweet
Tweet

Slide 1

Slide 1 text

BUILDING MICROSERVICE ARCHITECTURES IN GO @mattheath

Slide 2

Slide 2 text

@mattheath

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Mondo

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

1895

Slide 7

Slide 7 text

monoliths traditional dev

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

You're doing quite well

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

DATABASE APPLICATION

Slide 15

Slide 15 text

DATABASE APPLICATION

Slide 16

Slide 16 text

DATABASE DATABASES APPLICATION

Slide 17

Slide 17 text

DATABASE DATABASES APPLICATION SEARCH

Slide 18

Slide 18 text

DATABASE DATABASES APPLICATION CACHE SEARCH

Slide 19

Slide 19 text

DATABASE DATABASES APPLICATION CACHE SEARCH CAT GIFS

Slide 20

Slide 20 text

ALL HAIL THE MONOLITH

Slide 21

Slide 21 text

DO NOT WANT

Slide 22

Slide 22 text

DATABASE DATABASES MONOLITH CACHE SEARCH CAT GIFS

Slide 23

Slide 23 text

SCALE
 BREAKS
 HARDWARE

Slide 24

Slide 24 text

SPEED
 BREAKS
 SOFTWARE

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

BECOME
 CLOUD NATIVE

Slide 27

Slide 27 text

MONOLITH

Slide 28

Slide 28 text

MONOLITH

Slide 29

Slide 29 text

MONOLITH

Slide 30

Slide 30 text

MONOLITH

Slide 31

Slide 31 text

MONOLITH

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

Mondo

Slide 36

Slide 36 text

SINGLE RESPONSIBILITY BOUNDED CONTEXT WELL DEFINED INTERFACES COMPOSABILITY

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

message Profile { string user_id = 1; string name = 2; string preferred_name = 3; string phone_number = 4; string email = 5; Address address = 6; string date_of_birth = 7; string gender = 8; map metadata = 10; } PROTOBUF DEFINITIONS

Slide 39

Slide 39 text

message Profile { string user_id = 1; string name = 2; string preferred_name = 3; string phone_number = 4; string email = 5; Address address = 6; string date_of_birth = 7; string gender = 8; map metadata = 10; }

Slide 40

Slide 40 text

SERVICE SERVICE SERVICE LOAD BALANCER API / ROUTING LAYER MESSAGE BUS DATABASE

Slide 41

Slide 41 text

Service Service Service Load Balancer API / Routing Layer datacentre 1 RabbitMQ Message Bus Cassandra Cassandra Service Service Service Load Balancer API / Routing Layer RabbitMQ Message Bus datacentre n Service L AP Rabb

Slide 42

Slide 42 text

LOGIN
 API AUTH SERVICE PERMS SERVICE LOAD BALANCER API / ROUTING LAYER

Slide 43

Slide 43 text

LOGIN
 API AUTH SERVICE PERMS SERVICE LOAD BALANCER API / ROUTING LAYER ACCOUNT SERVICE FEED SERVICE

Slide 44

Slide 44 text

? ? ? LOAD BALANCER API / ROUTING LAYER ? ?

Slide 45

Slide 45 text

Logic Handler Storage Service

Slide 46

Slide 46 text

Library for building services that talk Protobuf via RMQ library Logic Handler Storage Service

Slide 47

Slide 47 text

Library for building services that talk Protobuf via RMQ Self-configuring external service adapters library Logic Handler Storage Service libraries

Slide 48

Slide 48 text

service.Register( &Endpoint{ Name: "foo", Handler: handler.Foo, Request: fooproto.Request, Response: fooproto.Response, } )

Slide 49

Slide 49 text

package handler import ( "..." ) func Foo(ctx context.Context, req server.Request) (protobuf.Message, error) { }

Slide 50

Slide 50 text

deliveries, err := server.Listen() for delivery := range deliveries { go handleDelivery(delivery) }

Slide 51

Slide 51 text

func handleDelivery(delivery amqp.Delivery) { // Do something for this request response, err := executeRequest(delivery) // Send Response server.Respond(response) }

Slide 52

Slide 52 text

var tokens = make(chan bool, 1000) 
 func handleDelivery(delivery amqp.Delivery) { // Throttle requests // error handling omitted... select { case <-tokens: response, err := executeRequest(delivery) tokens <- true // replace token case <-time.After(100 * time.Millisecond): response, err := timeoutResponse() } // Send Response server.Respond(response) }

Slide 53

Slide 53 text

var tokens = make(chan bool, 1000) 
 func handleDelivery(delivery amqp.Delivery) { // Throttle requests // error handling omitted... select { case <-tokens: response, err := executeRequest(delivery) tokens <- true // replace token case <-time.After(100 * time.Millisecond): response, err := timeoutResponse() } // Send Response server.Respond(response) }

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

• Provisioning • Service discovery • Configuration • Monitoring • Authentication/authorisation • AB testing • Self configuring connectivity 
 to third-party services Library for building services that talk Protobuf via RMQ Self-configuring external service adapters library Logic Handler Storage Service libraries

Slide 56

Slide 56 text

Mesos Slave Mesos Slave Mesos Slave CI Pipeline (Janky/Jenkins) Amazon S3 Provisioning Scheduler - eg. Marathon

Slide 57

Slide 57 text

DEALING WITH 
 COMPLEXITY

Slide 58

Slide 58 text

TESTING

Slide 59

Slide 59 text

LOAD FAILURE DEGRADATION

Slide 60

Slide 60 text

MONITORING

Slide 61

Slide 61 text

IN-BAND vs OUT-OF-BAND

Slide 62

Slide 62 text

AUTOMATIC HEALTHCHECK REGISTRATION Provisioning Service Rabbit MQ Monitoring
 Service Provisioning Service 
 Service Publish Healthchecks

Slide 63

Slide 63 text

MONITOR YOUR BUSINESS LOGIC

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

DISTRIBUTED
 TRACING

Slide 66

Slide 66 text

api api.customer service.customer api api.customer service.customer

Slide 67

Slide 67 text

api api.customer service.customer api api.customer service.customer REQ REP REQ REP IN OUT IN OUT

Slide 68

Slide 68 text

{ "timestamp": 1410262798416053450, "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765", "type": "REQ", "messageId": "6404dd1e-c995-48a9-73dc-9edb1380f0bf", "parentMessageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120", "from": "api.customer", "to": "service.customer", "hostname": "ip-10-13-2-251", "handlerInstanceId": “server-18bd089e-8ef1-4ca1-75cb-8...c”, "az": "eu-west-1a", "payload": "..." }

Slide 69

Slide 69 text

{ "timestamp": 1410262798416053450, "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765", "type": "REQ", "messageId": "6404dd1e-c995-48a9-73dc-9edb1380f0bf", "parentMessageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120", "from": "api.customer", "to": "service.customer", "hostname": "ip-10-13-2-251", "handlerInstanceId": “server-18bd089e-8ef1-4ca1-75cb-8...c”, "az": "eu-west-1a", "payload": "..." }

Slide 70

Slide 70 text

Tracing: 33eda743-f124-435c-71fc-3c872bbc98e6 2015-06-03 02:20:19.867 [/] [START] → - 2015-06-03 02:20:19.867 [eu-west-1a/ip-10-11-3-51] [REQ] api → api.customer.login - 2015-06-03 02:20:19.867 [eu-west-1a/ip-10-11-2-203] [IN] api → api.customer.login - 2015-06-03 02:20:19.868 [eu-west-1a/ip-10-11-2-203] [REQ] api.customer → service.feature-flags.features - 2015-06-03 02:20:19.869 [eu-west-1a/ip-10-11-3-111] [IN] api.customer → service.feature-flags.features - 2015-06-03 02:20:19.876 [eu-west-1a/ip-10-11-3-111] [REQ] service.feature-flags → service.experiment.list - 2015-06-03 02:20:19.877 [eu-west-1a/ip-10-11-3-168] [IN] service.experiment → service.config.compile - 2015-06-03 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [IN] service.feature-flags → service.experiment.list - 2015-06-03 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [REQ] service.experiment → service.config.compile - 2015-06-03 02:20:19.883 [eu-west-1a/ip-10-11-3-168] [OUT] service.experiment → service.config.compile - 5.59 ms 2015-06-03 02:20:19.886 [eu-west-1a/ip-10-11-3-111] [REP] service.experiment → service.config.compile - 8.40 ms 2015-06-03 02:20:19.887 [eu-west-1a/ip-10-11-3-111] [OUT] service.feature-flags → service.experiment.list - 9.72 ms 2015-06-03 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [REP] service.feature-flags → service.experiment.list - 13.23 ms 2015-06-03 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [OUT] api.customer → service.feature-flags.features - 20.58 ms 2015-06-03 02:20:19.890 [eu-west-1a/ip-10-11-2-203] [REP] api.customer → service.feature-flags.features - 22.59 ms 2015-06-03 02:20:19.902 [eu-west-1a/ip-10-11-2-203] [REQ] api.customer → service.customer.read - 2015-06-03 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] api.customer → service.customer.read - 2015-06-03 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] api.customer → service.customer.read - 2015-06-03 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [IN] api.customer → service.customer.read - 2015-06-03 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [OUT] api.customer → service.customer.read - 0.36 ms 2015-06-03 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REP] api.customer → service.customer.read - 1.97 ms 2015-06-03 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [IN] api.customer → service.customer.read - 2015-06-03 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REQ] api.customer → service.transaction.search - 2015-06-03 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [OUT] api.customer → service.customer.base - 0.10 ms 2015-06-03 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [IN] api.customer → service.customer.base - 2015-06-03 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [OUT] api.customer → service.customer.base - 0.06 ms 2015-06-03 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [IN] api.customer → service.feed.search - 2015-06-03 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [REQ] service.feed → service.zoning.search - 2015-06-03 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [IN] service.feed → service.zoning.search - 2015-06-03 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [OUT] service.feed → service.zoning.search - 0.20 ms 2015-06-03 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REP] service.feed → service.zoning.search - 2.25 ms 2015-06-03 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REQ] service.feed → service.raziel.multisearch - 2015-06-03 02:20:19.912 [eu-west-1a/ip-10-11-3-227] [IN] service.feed → service.raziel.multisearch - 2015-06-03 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REP] service.feed → service.raziel.multisearch - 9.46 ms 2015-06-03 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REQ] service.feed → service.location.multisearch - 2015-06-03 02:20:19.919 [eu-west-1a/ip-10-11-3-227] [OUT] service.feed → service.raziel.multisearch - 7.58 ms 2015-06-03 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [IN] service.feed → service.location.multisearch - 2015-06-03 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [OUT] service.feed → service.location.multisearch - 0.06 ms 2015-06-03 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [REP] service.feed → service.location.multisearch - 1.77 ms 2015-06-03 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [OUT] api.customer → service.feed.search - 14.02 ms 2015-06-03 02:20:19.921 [eu-west-1a/ip-10-11-2-203] [REP] api.customer → service.feed.search - 15.48 ms 2015-06-03 02:20:19.941 [eu-west-1a/ip-10-11-2-203] [REQ] api.customer → service.experiment.readlastupdated - 2015-06-03 02:20:19.945 [eu-west-1a/ip-10-11-2-214] [IN] api.customer → service.experiment.readlastupdated - 2015-06-03 02:20:19.947 [eu-west-1a/ip-10-11-2-214] [OUT] api.customer → service.experiment.readlastupdated - 1.82 ms 2015-06-03 02:20:19.947 [eu-west-1a/ip-10-11-2-203] [REP] api.customer → service.experiment.readlastupdated - 6.01 ms 2015-06-03 02:20:19.948 [eu-west-1a/ip-10-11-2-203] [OUT] api → api.customer.login - 80.46 ms 2015-06-03 02:20:19.950 [eu-west-1a/ip-10-11-3-51] [REP] api → api.customer.login - 82.71 ms

Slide 71

Slide 71 text

api api.customer service.customer api api.customer service.customer REQ REP REQ REP IN OUT IN OUT

Slide 72

Slide 72 text

api api.customer service.customer api api.customer service.customer REQ REP REQ REP IN OUT IN OUT Phosphor

Slide 73

Slide 73 text

api api.customer service.customer api api.customer service.customer REQ REP REQ REP IN OUT IN OUT Phosphor

Slide 74

Slide 74 text

PhosphorD Host Instances Publish Service A Trace Library goroutine chan UDP Service B Trace Library goroutine chan UDP Phosphor In-memory Aggregates Optional
 persistant storage Dashboards & Monitoring

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

SO, GO?

Slide 79

Slide 79 text

SMALL SIMPLE EASY TO LEARN

Slide 80

Slide 80 text

CONCURRENCY & INTERFACES

Slide 81

Slide 81 text

EASY DEPLOYMENT & MANAGEMENT IN PRODUCTION AND AT SCALE

Slide 82

Slide 82 text

Image Credits ATM: Thomas Hawk
 IBM System/360: IBM Orbital Ion Cannon: www.rom.ac Go Gophers: Renee French Duopia: Jonny Hughes Control Room: NASA Microchips: Santi ATM Failure: George Redgrave THANKS!