$30 off During Our Annual Pro Sale. View Details »

Scaling Microservices in Go - High Load Strategy 2015

mattheath
January 30, 2015

Scaling Microservices in Go - High Load Strategy 2015

Presented at High Load Strategy 2015 in Vilnius, Lithuania

Moving to a microservice architecture is a complex challenge, which often requires re-evaluating our approach to technology. Having previously helped transition Hailo to a new microservice platform, built almost entirely in Go, Matt is now working with Starling, using the latest technologies to build a new breed of digital bank. This talk will cover how to develop and migrate to a microservice architecture, the benefits of this approach, common pitfalls to avoid, and lessons learnt when developing high volume, low latency, distributed Go applications.

mattheath

January 30, 2015
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript

  1. SCALING MICRO-
    SERVICES WITH GO
    HIGH LOAD
    STRATEGY 2015
    @mattheath

    View Slide

  2. S TA R L I N G

    View Slide

  3. monoliths
    traditional dev

    View Slide

  4. Database
    Application

    View Slide

  5. Database
    App App App
    Database
    App
    Load Balancer
    App App

    View Slide

  6. SLOW

    DEVELOPMENT

    View Slide

  7. SLOW

    RELEASE CYCLES

    View Slide

  8. COMPLEX

    SOFTWARE

    View Slide

  9. HARD

    TO SCALE

    View Slide

  10. SCALE

    BREAKS

    HARDWARE

    View Slide

  11. SPEED

    BREAKS

    SOFTWARE

    View Slide

  12. Communication paths
    with two programmers
    Communication paths
    with three programmers
    Communication paths
    with four programmers
    Communication paths
    with five programmers
    Communication paths
    with ten programmers

    View Slide

  13. View Slide

  14. “construct a highly agile and
    highly available service from
    ephemeral and assumed broken
    components”- Adrian Cockcroſt

    View Slide

  15. A CLOUDY

    FUTURE

    View Slide

  16. MICROSERVICES

    View Slide

  17. transport layers
    COMMUNICATION

    View Slide

  18. Database
    Service Service Service
    Load Balancer
    API / Routing Layer
    datacentre 1
    Service Service Service
    Load Balancer
    API / Routing Layer
    RabbitMQ Message Bus RabbitMQ Message Bus
    Database
    datacentre n

    View Slide

  19. View Slide

  20. package handler
    import (
    "..."
    )
    func Foo(req *server.Request) (protobuf.Message, error) {
    }

    View Slide

  21. service.Register(
    &service.Endpoint{
    Name: "foo",
    Handler: handler.Foo,
    }
    )

    View Slide

  22. deliveries, err := server.Listen()
    for delivery := range deliveries {
    go handleDelivery(delivery)
    }

    View Slide

  23. func handleDelivery(delivery amqp.Delivery) {
    // Do something for this request
    response, err := executeRequest(delivery)
    // Send Response
    server.Respond(response)
    }

    View Slide

  24. var tokens = make(chan bool, 1000)

    func handleDelivery(delivery amqp.Delivery) {
    // Throttle requests
    // error handling omitted...
    select {
    case <-tokens:
    response, err := executeRequest(delivery)
    tokens <- true // replace token
    case <-time.After(500 * time.Millisecond):
    response, err := timeoutResponse()
    }
    // Send Response
    server.Respond(response)
    }

    View Slide

  25. Logic
    Handler
    Storage
    Service

    View Slide

  26. platform-library
    Library for building services that talk Protobuf via RMQ
    Logic
    Handler
    Storage
    Service

    View Slide

  27. service-library
    Library for building services that talk Protobuf via RMQ
    Self-configuring external service adapters
    platform-library
    Logic
    Handler
    Storage
    Service

    View Slide

  28. Services get for free:
    • Provisioning
    • Service discovery
    • Configuration
    • Monitoring
    • Authentication/authorisation
    • AB testing
    • Self configuring connectivity 

    to third-party services
    Service
    Library for building services that talk Protobuf via RMQ
    service-library
    platform-library
    Logic
    Handler
    Storage
    Self-configuring external service adapters

    View Slide

  29. View Slide

  30. Provisioning Daemon Provisioning Daemon Provisioning Daemon
    CI Pipeline (Janky/Jenkins)
    Amazon S3
    Provisioning Scheduler - eg. Marathon on Mesos

    View Slide

  31. DEALING WITH
    COMPLEXITY

    View Slide

  32. LOOSE

    COUPLING

    View Slide

  33. ASYNC
    EVENT

    DRIVEN

    View Slide

  34. TESTING

    View Slide

  35. LOAD
    FAILURE
    DEGRADATION

    View Slide

  36. MONITORING

    View Slide

  37. DISTRIBUTED

    TRACING

    View Slide

  38. api api.customer service.customer
    api api.customer service.customer

    View Slide

  39. api api.customer service.customer
    api api.customer service.customer
    REQ
    REP
    REQ
    REP
    IN
    OUT
    IN
    OUT

    View Slide

  40. {
    "timestamp": 1410262798427145176,
    "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765",
    "type": "OUT",
    "messageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120",
    "parentMessageId": "",
    "from": “uk.co.starlingbank.webui”,
    "to": “uk.co.starlingbank.api.customer",
    "hostname": "ip-10-13-2-251",
    "az": "eu-west-1a",
    "handlerInstanceId": “server-18bd089e-8ef1-4ca1-75cb-8...c”,
    "duration": 11222094
    }
    {
    "timestamp": 1410262798416053450,
    "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765",
    "type": "REQ",
    "messageId": "6404dd1e-c995-48a9-73dc-9edb1380f0bf",
    "parentMessageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120",
    "from": "uk.co.starlingbank.api.customer",
    "to": "uk.co.starlingbank.service.customer",
    "hostname": "ip-10-13-2-251",
    "az": "eu-west-1a"
    }

    View Slide

  41. Tracing: 33eda743-f124-435c-71fc-3c872bbc98e6
    2014-09-07 02:20:19.867 [/] [START] → -
    2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-3-51] [REQ] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers -
    2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-2-203] [IN] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers -
    2014-09-07 02:20:19.868 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features -
    2014-09-07 02:20:19.869 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features -
    2014-09-07 02:20:19.876 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-168] [IN] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile -
    2014-09-07 02:20:19.883 [eu-west-1a/ip-10-11-3-168] [OUT] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 5.59 ms
    2014-09-07 02:20:19.886 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 8.40 ms
    2014-09-07 02:20:19.887 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 9.72 ms
    2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 13.23 ms
    2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 20.58 ms
    2014-09-07 02:20:19.890 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 22.59 ms
    2014-09-07 02:20:19.902 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.36 ms
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 1.97 ms
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search -
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.10 ms

    ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx
    2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.06 ms 

    ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx
    2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search -
    2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search -
    2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search -
    2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 0.20 ms
    2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2.25 ms
    2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch -
    2014-09-07 02:20:19.912 [eu-west-1a/ip-10-11-3-227] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch -
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 9.46 ms
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime -
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-227] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 7.58 ms
    2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime -
    2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 0.06 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 1.77 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 14.02 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 15.48 ms
    2014-09-07 02:20:19.941 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated -
    2014-09-07 02:20:19.945 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated -
    2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 1.82 ms
    2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 6.01 ms
    2014-09-07 02:20:19.948 [eu-west-1a/ip-10-11-2-203] [OUT] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 80.46 ms
    2014-09-07 02:20:19.950 [eu-west-1a/ip-10-11-3-51] [REP] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 82.71 ms

    View Slide

  42. View Slide

  43. View Slide

  44. View Slide

  45. View Slide

  46. View Slide

  47. WHY BUILD
    MICROSERVICES?

    View Slide

  48. SIMPLE

    INDEPENDENT
    UNITS

    View Slide

  49. SIMPLE

    SCALABILITY

    View Slide

  50. EXPECT

    FAILURE

    View Slide

  51. AUTOMATE

    EVERYTHING

    View Slide

  52. CLOUD NATIVE

    ANTIFRAGILITY

    View Slide

  53. GO?

    View Slide

  54. SMALL
    SIMPLE
    EASY TO READ
    EASY TO LEARN

    View Slide

  55. CONCURRENCY
    PRIMITIVES
    INTERFACE SUPPORT

    View Slide

  56. EASY DEPLOYMENT &
    MANAGEMENT IN
    PRODUCTION
    AND AT SCALE

    View Slide

  57. THANKS!
    Image Credits

    IBM System/360: IBM
    Orbital Ion Cannon: www.rom.ac
    Go Gophers: Renee French
    Duopia: Jonny Hughes
    Control Room: NASA
    Microchips: Santi
    Antennas: Alex Weimer

    View Slide