Scaling Microservices in Go - High Load Strategy 2015

Presented at High Load Strategy 2015 in Vilnius, Lithuania

Moving to a microservice architecture is a complex challenge, which often requires re-evaluating our approach to technology. Having previously helped transition Hailo to a new microservice platform, built almost entirely in Go, Matt is now working with Starling, using the latest technologies to build a new breed of digital bank. This talk will cover how to develop and migrate to a microservice architecture, the benefits of this approach, common pitfalls to avoid, and lessons learnt when developing high volume, low latency, distributed Go applications.

Transcript

1. SCALING MICRO- SERVICES WITH GO HIGH LOAD STRATEGY 2015 @mattheath
2. S TA R L I N G
3. monoliths traditional dev
4. Database Application
5. Database App App App Database App Load Balancer App App
6. SLOW
 DEVELOPMENT
7. SLOW
 RELEASE CYCLES
8. COMPLEX
 SOFTWARE
9. HARD
 TO SCALE
10. SCALE
 BREAKS
 HARDWARE
11. SPEED
 BREAKS
 SOFTWARE
12. Communication paths with two programmers Communication paths with three programmers Communication paths with four programmers Communication paths with five programmers Communication paths with ten programmers
14. “construct a highly agile and highly available service from ephemeral and assumed broken components”- Adrian Cockcroſt
15. A CLOUDY
 FUTURE
16. MICROSERVICES
17. transport layers COMMUNICATION
18. Database Service Service Service Load Balancer API / Routing Layer datacentre 1 Service Service Service Load Balancer API / Routing Layer RabbitMQ Message Bus RabbitMQ Message Bus Database datacentre n
20. package handler import ( "..." ) func Foo(req *server.Request) (protobuf.Message, error) { }
21. service.Register( &service.Endpoint{ Name: "foo", Handler: handler.Foo, } )
22. deliveries, err := server.Listen() for delivery := range deliveries { go handleDelivery(delivery) }
23. func handleDelivery(delivery amqp.Delivery) { // Do something for this request response, err := executeRequest(delivery) // Send Response server.Respond(response) }
24. var tokens = make(chan bool, 1000) 
 func handleDelivery(delivery amqp.Delivery) { // Throttle requests // error handling omitted... select { case <-tokens: response, err := executeRequest(delivery) tokens <- true // replace token case <-time.After(500 * time.Millisecond): response, err := timeoutResponse() } // Send Response server.Respond(response) }
25. Logic Handler Storage Service
26. platform-library Library for building services that talk Protobuf via RMQ Logic Handler Storage Service
27. service-library Library for building services that talk Protobuf via RMQ Self-configuring external service adapters platform-library Logic Handler Storage Service
28. Services get for free: • Provisioning • Service discovery • Configuration • Monitoring • Authentication/authorisation • AB testing • Self configuring connectivity 
 to third-party services Service Library for building services that talk Protobuf via RMQ service-library platform-library Logic Handler Storage Self-configuring external service adapters
30. Provisioning Daemon Provisioning Daemon Provisioning Daemon CI Pipeline (Janky/Jenkins) Amazon S3 Provisioning Scheduler - eg. Marathon on Mesos
31. DEALING WITH COMPLEXITY
32. LOOSE
 COUPLING
33. ASYNC EVENT
 DRIVEN
34. TESTING
35. LOAD FAILURE DEGRADATION
36. MONITORING
37. DISTRIBUTED
 TRACING
38. api api.customer service.customer api api.customer service.customer
39. api api.customer service.customer api api.customer service.customer REQ REP REQ REP IN OUT IN OUT
40. { "timestamp": 1410262798427145176, "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765", "type": "OUT", "messageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120", "parentMessageId": "", "from": “uk.co.starlingbank.webui”, "to": “uk.co.starlingbank.api.customer", "hostname": "ip-10-13-2-251", "az": "eu-west-1a", "handlerInstanceId": “server-18bd089e-8ef1-4ca1-75cb-8...c”, "duration": 11222094 } { "timestamp": 1410262798416053450, "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765", "type": "REQ", "messageId": "6404dd1e-c995-48a9-73dc-9edb1380f0bf", "parentMessageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120", "from": "uk.co.starlingbank.api.customer", "to": "uk.co.starlingbank.service.customer", "hostname": "ip-10-13-2-251", "az": "eu-west-1a" }
41. Tracing: 33eda743-f124-435c-71fc-3c872bbc98e6 2014-09-07 02:20:19.867 [/] [START] → - 2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-3-51] [REQ] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-2-203] [IN] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 2014-09-07 02:20:19.868 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 2014-09-07 02:20:19.869 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 2014-09-07 02:20:19.876 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-168] [IN] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 2014-09-07 02:20:19.883 [eu-west-1a/ip-10-11-3-168] [OUT] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 5.59 ms 2014-09-07 02:20:19.886 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 8.40 ms 2014-09-07 02:20:19.887 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 9.72 ms 2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 13.23 ms 2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 20.58 ms 2014-09-07 02:20:19.890 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 22.59 ms 2014-09-07 02:20:19.902 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.36 ms 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 1.97 ms 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.10 ms
 ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx 2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.06 ms 
 ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx 2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 0.20 ms 2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2.25 ms 2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 2014-09-07 02:20:19.912 [eu-west-1a/ip-10-11-3-227] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 9.46 ms 2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-227] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 7.58 ms 2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 0.06 ms 2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 1.77 ms 2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 14.02 ms 2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 15.48 ms 2014-09-07 02:20:19.941 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 2014-09-07 02:20:19.945 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 1.82 ms 2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 6.01 ms 2014-09-07 02:20:19.948 [eu-west-1a/ip-10-11-2-203] [OUT] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 80.46 ms 2014-09-07 02:20:19.950 [eu-west-1a/ip-10-11-3-51] [REP] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 82.71 ms
47. WHY BUILD MICROSERVICES?
48. SIMPLE
 INDEPENDENT UNITS
49. SIMPLE
 SCALABILITY
50. EXPECT
 FAILURE
51. AUTOMATE
 EVERYTHING
52. CLOUD NATIVE
 ANTIFRAGILITY
53. GO?
54. SMALL SIMPLE EASY TO READ EASY TO LEARN
55. CONCURRENCY PRIMITIVES INTERFACE SUPPORT
56. EASY DEPLOYMENT & MANAGEMENT IN PRODUCTION AND AT SCALE
57. THANKS! Image Credits
 IBM System/360: IBM Orbital Ion Cannon: www.rom.ac Go Gophers: Renee French Duopia: Jonny Hughes Control Room: NASA Microchips: Santi Antennas: Alex Weimer