Scaling Microservices in Go - High Load Strategy 2015

Presented at High Load Strategy 2015 in Vilnius, Lithuania

Moving to a microservice architecture is a complex challenge, which often requires re-evaluating our approach to technology. Having previously helped transition Hailo to a new microservice platform, built almost entirely in Go, Matt is now working with Starling, using the latest technologies to build a new breed of digital bank. This talk will cover how to develop and migrate to a microservice architecture, the benefits of this approach, common pitfalls to avoid, and lessons learnt when developing high volume, low latency, distributed Go applications.

Transcript

1. SCALING MICRO-
   SERVICES WITH GO
   HIGH LOAD
   STRATEGY 2015
   @mattheath
   

   View Slide

2. S TA R L I N G
   

   View Slide

3. monoliths
   traditional dev
   

   View Slide

4. Database
   Application
   

   View Slide

5. Database
   App App App
   Database
   App
   Load Balancer
   App App
   

   View Slide

6. SLOW

   DEVELOPMENT
   

   View Slide

7. SLOW

   RELEASE CYCLES
   

   View Slide

8. COMPLEX

   SOFTWARE
   

   View Slide

9. HARD

   TO SCALE
   

   View Slide

10. SCALE

    BREAKS

    HARDWARE
    

    View Slide

11. SPEED

    BREAKS

    SOFTWARE
    

    View Slide

12. Communication paths
    with two programmers
    Communication paths
    with three programmers
    Communication paths
    with four programmers
    Communication paths
    with five programmers
    Communication paths
    with ten programmers
    

    View Slide

13. View Slide

14. “construct a highly agile and
    highly available service from
    ephemeral and assumed broken
    components”- Adrian Cockcroſt
    

    View Slide

15. A CLOUDY

    FUTURE
    

    View Slide

16. MICROSERVICES
    

    View Slide

17. transport layers
    COMMUNICATION
    

    View Slide

18. Database
    Service Service Service
    Load Balancer
    API / Routing Layer
    datacentre 1
    Service Service Service
    Load Balancer
    API / Routing Layer
    RabbitMQ Message Bus RabbitMQ Message Bus
    Database
    datacentre n
    

    View Slide

19. View Slide

20. package handler
    import (
    "..."
    )
    func Foo(req *server.Request) (protobuf.Message, error) {
    }
    

    View Slide

21. service.Register(
    &service.Endpoint{
    Name: "foo",
    Handler: handler.Foo,
    }
    )
    

    View Slide

22. deliveries, err := server.Listen()
    for delivery := range deliveries {
    go handleDelivery(delivery)
    }
    

    View Slide

23. func handleDelivery(delivery amqp.Delivery) {
    // Do something for this request
    response, err := executeRequest(delivery)
    // Send Response
    server.Respond(response)
    }
    

    View Slide

24. var tokens = make(chan bool, 1000)
    

    func handleDelivery(delivery amqp.Delivery) {
    // Throttle requests
    // error handling omitted...
    select {
    case <-tokens:
    response, err := executeRequest(delivery)
    tokens <- true // replace token
    case <-time.After(500 * time.Millisecond):
    response, err := timeoutResponse()
    }
    // Send Response
    server.Respond(response)
    }
    

    View Slide

25. Logic
    Handler
    Storage
    Service
    

    View Slide

26. platform-library
    Library for building services that talk Protobuf via RMQ
    Logic
    Handler
    Storage
    Service
    

    View Slide

27. service-library
    Library for building services that talk Protobuf via RMQ
    Self-configuring external service adapters
    platform-library
    Logic
    Handler
    Storage
    Service
    

    View Slide

28. Services get for free:
    • Provisioning
    • Service discovery
    • Configuration
    • Monitoring
    • Authentication/authorisation
    • AB testing
    • Self configuring connectivity 

    to third-party services
    Service
    Library for building services that talk Protobuf via RMQ
    service-library
    platform-library
    Logic
    Handler
    Storage
    Self-configuring external service adapters
    

    View Slide

29. View Slide

30. Provisioning Daemon Provisioning Daemon Provisioning Daemon
    CI Pipeline (Janky/Jenkins)
    Amazon S3
    Provisioning Scheduler - eg. Marathon on Mesos
    

    View Slide

31. DEALING WITH
    COMPLEXITY
    

    View Slide

32. LOOSE

    COUPLING
    

    View Slide

33. ASYNC
    EVENT

    DRIVEN
    

    View Slide

34. TESTING
    

    View Slide

35. LOAD
    FAILURE
    DEGRADATION
    

    View Slide

36. MONITORING
    

    View Slide

37. DISTRIBUTED

    TRACING
    

    View Slide

38. api api.customer service.customer
    api api.customer service.customer
    

    View Slide

39. api api.customer service.customer
    api api.customer service.customer
    REQ
    REP
    REQ
    REP
    IN
    OUT
    IN
    OUT
    

    View Slide

40. {
    "timestamp": 1410262798427145176,
    "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765",
    "type": "OUT",
    "messageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120",
    "parentMessageId": "",
    "from": “uk.co.starlingbank.webui”,
    "to": “uk.co.starlingbank.api.customer",
    "hostname": "ip-10-13-2-251",
    "az": "eu-west-1a",
    "handlerInstanceId": “server-18bd089e-8ef1-4ca1-75cb-8...c”,
    "duration": 11222094
    }
    {
    "timestamp": 1410262798416053450,
    "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765",
    "type": "REQ",
    "messageId": "6404dd1e-c995-48a9-73dc-9edb1380f0bf",
    "parentMessageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120",
    "from": "uk.co.starlingbank.api.customer",
    "to": "uk.co.starlingbank.service.customer",
    "hostname": "ip-10-13-2-251",
    "az": "eu-west-1a"
    }
    

    View Slide

41. Tracing: 33eda743-f124-435c-71fc-3c872bbc98e6
    2014-09-07 02:20:19.867 [/] [START] → -
    2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-3-51] [REQ] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers -
    2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-2-203] [IN] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers -
    2014-09-07 02:20:19.868 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features -
    2014-09-07 02:20:19.869 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features -
    2014-09-07 02:20:19.876 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-168] [IN] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile -
    2014-09-07 02:20:19.883 [eu-west-1a/ip-10-11-3-168] [OUT] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 5.59 ms
    2014-09-07 02:20:19.886 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 8.40 ms
    2014-09-07 02:20:19.887 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 9.72 ms
    2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 13.23 ms
    2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 20.58 ms
    2014-09-07 02:20:19.890 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 22.59 ms
    2014-09-07 02:20:19.902 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.36 ms
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 1.97 ms
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search -
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.10 ms

    ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx
    2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.06 ms 

    ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx
    2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search -
    2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search -
    2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search -
    2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 0.20 ms
    2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2.25 ms
    2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch -
    2014-09-07 02:20:19.912 [eu-west-1a/ip-10-11-3-227] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch -
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 9.46 ms
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime -
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-227] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 7.58 ms
    2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime -
    2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 0.06 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 1.77 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 14.02 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 15.48 ms
    2014-09-07 02:20:19.941 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated -
    2014-09-07 02:20:19.945 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated -
    2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 1.82 ms
    2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 6.01 ms
    2014-09-07 02:20:19.948 [eu-west-1a/ip-10-11-2-203] [OUT] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 80.46 ms
    2014-09-07 02:20:19.950 [eu-west-1a/ip-10-11-3-51] [REP] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 82.71 ms
    

    View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. WHY BUILD
    MICROSERVICES?
    

    View Slide

48. SIMPLE

    INDEPENDENT
    UNITS
    

    View Slide

49. SIMPLE

    SCALABILITY
    

    View Slide

50. EXPECT

    FAILURE
    

    View Slide

51. AUTOMATE

    EVERYTHING
    

    View Slide

52. CLOUD NATIVE

    ANTIFRAGILITY
    

    View Slide

53. GO?
    

    View Slide

54. SMALL
    SIMPLE
    EASY TO READ
    EASY TO LEARN
    

    View Slide

55. CONCURRENCY
    PRIMITIVES
    INTERFACE SUPPORT
    

    View Slide

56. EASY DEPLOYMENT &
    MANAGEMENT IN
    PRODUCTION
    AND AT SCALE
    

    View Slide

57. THANKS!
    Image Credits

    IBM System/360: IBM
    Orbital Ion Cannon: www.rom.ac
    Go Gophers: Renee French
    Duopia: Jonny Hughes
    Control Room: NASA
    Microchips: Santi
    Antennas: Alex Weimer
    

    View Slide