Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling Microservices in Go - High Load Strategy 2015

mattheath
January 30, 2015

Scaling Microservices in Go - High Load Strategy 2015

Presented at High Load Strategy 2015 in Vilnius, Lithuania

Moving to a microservice architecture is a complex challenge, which often requires re-evaluating our approach to technology. Having previously helped transition Hailo to a new microservice platform, built almost entirely in Go, Matt is now working with Starling, using the latest technologies to build a new breed of digital bank. This talk will cover how to develop and migrate to a microservice architecture, the benefits of this approach, common pitfalls to avoid, and lessons learnt when developing high volume, low latency, distributed Go applications.

mattheath

January 30, 2015
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript

  1. SCALING MICRO-
    SERVICES WITH GO
    HIGH LOAD
    STRATEGY 2015
    @mattheath

    View full-size slide

  2. S TA R L I N G

    View full-size slide

  3. monoliths
    traditional dev

    View full-size slide

  4. Database
    Application

    View full-size slide

  5. Database
    App App App
    Database
    App
    Load Balancer
    App App

    View full-size slide

  6. SLOW

    DEVELOPMENT

    View full-size slide

  7. SLOW

    RELEASE CYCLES

    View full-size slide

  8. COMPLEX

    SOFTWARE

    View full-size slide

  9. HARD

    TO SCALE

    View full-size slide

  10. SCALE

    BREAKS

    HARDWARE

    View full-size slide

  11. SPEED

    BREAKS

    SOFTWARE

    View full-size slide

  12. Communication paths
    with two programmers
    Communication paths
    with three programmers
    Communication paths
    with four programmers
    Communication paths
    with five programmers
    Communication paths
    with ten programmers

    View full-size slide

  13. “construct a highly agile and
    highly available service from
    ephemeral and assumed broken
    components”- Adrian Cockcroſt

    View full-size slide

  14. A CLOUDY

    FUTURE

    View full-size slide

  15. MICROSERVICES

    View full-size slide

  16. transport layers
    COMMUNICATION

    View full-size slide

  17. Database
    Service Service Service
    Load Balancer
    API / Routing Layer
    datacentre 1
    Service Service Service
    Load Balancer
    API / Routing Layer
    RabbitMQ Message Bus RabbitMQ Message Bus
    Database
    datacentre n

    View full-size slide

  18. package handler
    import (
    "..."
    )
    func Foo(req *server.Request) (protobuf.Message, error) {
    }

    View full-size slide

  19. service.Register(
    &service.Endpoint{
    Name: "foo",
    Handler: handler.Foo,
    }
    )

    View full-size slide

  20. deliveries, err := server.Listen()
    for delivery := range deliveries {
    go handleDelivery(delivery)
    }

    View full-size slide

  21. func handleDelivery(delivery amqp.Delivery) {
    // Do something for this request
    response, err := executeRequest(delivery)
    // Send Response
    server.Respond(response)
    }

    View full-size slide

  22. var tokens = make(chan bool, 1000)

    func handleDelivery(delivery amqp.Delivery) {
    // Throttle requests
    // error handling omitted...
    select {
    case <-tokens:
    response, err := executeRequest(delivery)
    tokens <- true // replace token
    case <-time.After(500 * time.Millisecond):
    response, err := timeoutResponse()
    }
    // Send Response
    server.Respond(response)
    }

    View full-size slide

  23. Logic
    Handler
    Storage
    Service

    View full-size slide

  24. platform-library
    Library for building services that talk Protobuf via RMQ
    Logic
    Handler
    Storage
    Service

    View full-size slide

  25. service-library
    Library for building services that talk Protobuf via RMQ
    Self-configuring external service adapters
    platform-library
    Logic
    Handler
    Storage
    Service

    View full-size slide

  26. Services get for free:
    • Provisioning
    • Service discovery
    • Configuration
    • Monitoring
    • Authentication/authorisation
    • AB testing
    • Self configuring connectivity 

    to third-party services
    Service
    Library for building services that talk Protobuf via RMQ
    service-library
    platform-library
    Logic
    Handler
    Storage
    Self-configuring external service adapters

    View full-size slide

  27. Provisioning Daemon Provisioning Daemon Provisioning Daemon
    CI Pipeline (Janky/Jenkins)
    Amazon S3
    Provisioning Scheduler - eg. Marathon on Mesos

    View full-size slide

  28. DEALING WITH
    COMPLEXITY

    View full-size slide

  29. LOOSE

    COUPLING

    View full-size slide

  30. ASYNC
    EVENT

    DRIVEN

    View full-size slide

  31. LOAD
    FAILURE
    DEGRADATION

    View full-size slide

  32. DISTRIBUTED

    TRACING

    View full-size slide

  33. api api.customer service.customer
    api api.customer service.customer

    View full-size slide

  34. api api.customer service.customer
    api api.customer service.customer
    REQ
    REP
    REQ
    REP
    IN
    OUT
    IN
    OUT

    View full-size slide

  35. {
    "timestamp": 1410262798427145176,
    "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765",
    "type": "OUT",
    "messageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120",
    "parentMessageId": "",
    "from": “uk.co.starlingbank.webui”,
    "to": “uk.co.starlingbank.api.customer",
    "hostname": "ip-10-13-2-251",
    "az": "eu-west-1a",
    "handlerInstanceId": “server-18bd089e-8ef1-4ca1-75cb-8...c”,
    "duration": 11222094
    }
    {
    "timestamp": 1410262798416053450,
    "traceId": "d30479b8-1491-4390-7cf5-4cd14bc4b765",
    "type": "REQ",
    "messageId": "6404dd1e-c995-48a9-73dc-9edb1380f0bf",
    "parentMessageId": "a661f9ef-774c-49b2-6e74-cfed65f7d120",
    "from": "uk.co.starlingbank.api.customer",
    "to": "uk.co.starlingbank.service.customer",
    "hostname": "ip-10-13-2-251",
    "az": "eu-west-1a"
    }

    View full-size slide

  36. Tracing: 33eda743-f124-435c-71fc-3c872bbc98e6
    2014-09-07 02:20:19.867 [/] [START] → -
    2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-3-51] [REQ] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers -
    2014-09-07 02:20:19.867 [eu-west-1a/ip-10-11-2-203] [IN] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers -
    2014-09-07 02:20:19.868 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features -
    2014-09-07 02:20:19.869 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features -
    2014-09-07 02:20:19.876 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-168] [IN] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list -
    2014-09-07 02:20:19.877 [eu-west-1a/ip-10-11-3-111] [REQ] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile -
    2014-09-07 02:20:19.883 [eu-west-1a/ip-10-11-3-168] [OUT] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 5.59 ms
    2014-09-07 02:20:19.886 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.hob → uk.co.starlingbank.service.config.compile - 8.40 ms
    2014-09-07 02:20:19.887 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 9.72 ms
    2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [REP] uk.co.starlingbank.service.feature-flags → uk.co.starlingbank.service.hob.list - 13.23 ms
    2014-09-07 02:20:19.889 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 20.58 ms
    2014-09-07 02:20:19.890 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.feature-flags.features - 22.59 ms
    2014-09-07 02:20:19.902 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.903 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.904 [eu-west-1a/ip-10-11-3-111] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.36 ms
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 1.97 ms
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search -
    2014-09-07 02:20:19.905 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.10 ms

    ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx
    2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare -
    2014-09-07 02:20:19.906 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.fare.basefare - 0.06 ms 

    ERR - uk.co.starlingbank.service.fare.basefare: Missing config at xxx
    2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search -
    2014-09-07 02:20:19.907 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search -
    2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search -
    2014-09-07 02:20:19.908 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 0.20 ms
    2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.zoning.search - 2.25 ms
    2014-09-07 02:20:19.909 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch -
    2014-09-07 02:20:19.912 [eu-west-1a/ip-10-11-3-227] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch -
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 9.46 ms
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-58] [REQ] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime -
    2014-09-07 02:20:19.919 [eu-west-1a/ip-10-11-3-227] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.raziel.multisearch - 7.58 ms
    2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [IN] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime -
    2014-09-07 02:20:19.920 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 0.06 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [REP] uk.co.starlingbank.service.nearest-driver → uk.co.starlingbank.service.eta.multitraveltime - 1.77 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-3-58] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 14.02 ms
    2014-09-07 02:20:19.921 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.nearest-driver.search - 15.48 ms
    2014-09-07 02:20:19.941 [eu-west-1a/ip-10-11-2-203] [REQ] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated -
    2014-09-07 02:20:19.945 [eu-west-1a/ip-10-11-2-214] [IN] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated -
    2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-214] [OUT] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 1.82 ms
    2014-09-07 02:20:19.947 [eu-west-1a/ip-10-11-2-203] [REP] uk.co.starlingbank.api.customer → uk.co.starlingbank.service.experiment.readlastupdated - 6.01 ms
    2014-09-07 02:20:19.948 [eu-west-1a/ip-10-11-2-203] [OUT] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 80.46 ms
    2014-09-07 02:20:19.950 [eu-west-1a/ip-10-11-3-51] [REP] uk.co.starlingbank.api → uk.co.starlingbank.api.customer.neardrivers - 82.71 ms

    View full-size slide

  37. WHY BUILD
    MICROSERVICES?

    View full-size slide

  38. SIMPLE

    INDEPENDENT
    UNITS

    View full-size slide

  39. SIMPLE

    SCALABILITY

    View full-size slide

  40. EXPECT

    FAILURE

    View full-size slide

  41. AUTOMATE

    EVERYTHING

    View full-size slide

  42. CLOUD NATIVE

    ANTIFRAGILITY

    View full-size slide

  43. SMALL
    SIMPLE
    EASY TO READ
    EASY TO LEARN

    View full-size slide

  44. CONCURRENCY
    PRIMITIVES
    INTERFACE SUPPORT

    View full-size slide

  45. EASY DEPLOYMENT &
    MANAGEMENT IN
    PRODUCTION
    AND AT SCALE

    View full-size slide

  46. THANKS!
    Image Credits

    IBM System/360: IBM
    Orbital Ion Cannon: www.rom.ac
    Go Gophers: Renee French
    Duopia: Jonny Hughes
    Control Room: NASA
    Microchips: Santi
    Antennas: Alex Weimer

    View full-size slide