SCALING MICRO- SERVICES WITH GO HIGH LOAD STRATEGY 2015 @mattheath 

S TA R L I N G 

monoliths traditional dev 

Database Application 

Database App App App Database App Load Balancer App App 

SLOW
 DEVELOPMENT 

SLOW
 RELEASE CYCLES 

COMPLEX
 SOFTWARE 

HARD
 TO SCALE 

SCALE
 BREAKS
 HARDWARE 

SPEED
 BREAKS
 SOFTWARE 

Communication paths with two programmers Communication paths with three programmers Communication paths with four programmers Communication paths with five programmers Communication paths with ten programmers 

No content

“construct a highly agile and highly available service from ephemeral and assumed broken components”- Adrian Cockcroſt 

A CLOUDY
 FUTURE 

MICROSERVICES 

transport layers COMMUNICATION 

Database Service Service Service Load Balancer API / Routing Layer datacentre 1 Service Service Service Load Balancer API / Routing Layer RabbitMQ Message Bus RabbitMQ Message Bus Database datacentre n 

No content

package handler import ( "..." ) func Foo(req *server.Request) (protobuf.Message, error) { } 

service.Register( &service.Endpoint{ Name: "foo", Handler: handler.Foo, } ) 

deliveries, err := server.Listen() for delivery := range deliveries { go handleDelivery(delivery) } 

func handleDelivery(delivery amqp.Delivery) { // Do something for this request response, err := executeRequest(delivery) // Send Response server.Respond(response) } 

var tokens = make(chan bool, 1000) 
 func handleDelivery(delivery amqp.Delivery) { // Throttle requests // error handling omitted... select { case <-tokens: response, err := executeRequest(delivery) tokens <- true // replace token case <-time.After(500 * time.Millisecond): response, err := timeoutResponse() } // Send Response server.Respond(response) } 

Logic Handler Storage Service 

platform-library Library for building services that talk Protobuf via RMQ Logic Handler Storage Service 

service-library Library for building services that talk Protobuf via RMQ Self-configuring external service adapters platform-library Logic Handler Storage Service 

Services get for free: • Provisioning • Service discovery • Configuration • Monitoring • Authentication/authorisation • AB testing • Self configuring connectivity 
 to third-party services Service Library for building services that talk Protobuf via RMQ service-library platform-library Logic Handler Storage Self-configuring external service adapters 

No content

Provisioning Daemon Provisioning Daemon Provisioning Daemon CI Pipeline (Janky/Jenkins) Amazon S3 Provisioning Scheduler - eg. Marathon on Mesos 

DEALING WITH COMPLEXITY 

LOOSE
 COUPLING 

ASYNC EVENT
 DRIVEN 

TESTING 

LOAD FAILURE DEGRADATION 

MONITORING 

DISTRIBUTED
 TRACING 

api api.customer service.customer api api.customer service.customer 

api api.customer service.customer api api.customer service.customer REQ REP REQ REP IN OUT IN OUT 

{ "timestamp": 1410262798427145176, "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765", "type": "OUT", "messageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120", "parentMessageId": "", "from": “uk.co.starlingbank.webui”, "to": “uk.co.starlingbank.api.customer", "hostname": "ip-10-13-2-251", "az": "eu-west-1a", "handlerInstanceId": “server-18bd089e-8ef1-4ca1-75cb-8...c”, "duration": 11222094 } { "timestamp": 1410262798416053450, "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765", "type": "REQ", "messageId": "6404dd1e-c995-48a9-73dc-9edb1380f0bf", "parentMessageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120", "from": "uk.co.starlingbank.api.customer", "to": "uk.co.starlingbank.service.customer", "hostname": "ip-10-13-2-251", "az": "eu-west-1a" } 

Tracing: 33eda743-f124-435c-71fc-3c872bbc98e6 2014-09-07 02:20:19.867 [/] [START] → - 2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-3-51] [REQ] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-2-203] [IN] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 2014-09-07 02:20:19.868 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 2014-09-07 02:20:19.869 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 2014-09-07 02:20:19.876 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-168] [IN] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 2014-09-07 02:20:19.883 [eu-west-1a/ip-10-11-3-168] [OUT] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 5.59 ms 2014-09-07 02:20:19.886 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 8.40 ms 2014-09-07 02:20:19.887 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 9.72 ms 2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 13.23 ms 2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 20.58 ms 2014-09-07 02:20:19.890 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 22.59 ms 2014-09-07 02:20:19.902 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.36 ms 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 1.97 ms 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.10 ms
 ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx 2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.06 ms 
 ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx 2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 0.20 ms 2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2.25 ms 2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 2014-09-07 02:20:19.912 [eu-west-1a/ip-10-11-3-227] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 9.46 ms 2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-227] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 7.58 ms 2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 0.06 ms 2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 1.77 ms 2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 14.02 ms 2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 15.48 ms 2014-09-07 02:20:19.941 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 2014-09-07 02:20:19.945 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 1.82 ms 2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 6.01 ms 2014-09-07 02:20:19.948 [eu-west-1a/ip-10-11-2-203] [OUT] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 80.46 ms 2014-09-07 02:20:19.950 [eu-west-1a/ip-10-11-3-51] [REP] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 82.71 ms 

No content

No content

No content

No content

No content

WHY BUILD MICROSERVICES? 

SIMPLE
 INDEPENDENT UNITS 

SIMPLE
 SCALABILITY 

EXPECT
 FAILURE 

AUTOMATE
 EVERYTHING 

CLOUD NATIVE
 ANTIFRAGILITY 

GO? 

SMALL SIMPLE EASY TO READ EASY TO LEARN 

CONCURRENCY PRIMITIVES INTERFACE SUPPORT 

EASY DEPLOYMENT & MANAGEMENT IN PRODUCTION AND AT SCALE 

THANKS! Image Credits
 IBM System/360: IBM Orbital Ion Cannon: www.rom.ac Go Gophers: Renee French Duopia: Jonny Hughes Control Room: NASA Microchips: Santi Antennas: Alex Weimer