Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Microservice Architectures with Go - VoxxedDays Berlin 2016

mattheath
January 28, 2016

Building Microservice Architectures with Go - VoxxedDays Berlin 2016

Presented at VoxxedDays Berlin 2016

Traditionally applications have been built as monoliths; single applications which become larger and more complex over time, which limit our ability to react to change. An example of this is the banking industry where mergers and acquisitions between banks have lead to a patchwork of different systems & technologies that cost billions of dollars per year to maintain. As a result, the pace of innovation in the banking industry has slowed to a crawl.

At Mondo we're building a new kind of bank, a smart bank that belongs in the 21st century, and we’re building it almost entirely in Go. This talk will cover how we’re developing new core banking systems from scratch backed by a microservice platform written in Go, running across multiple data centres using open source frameworks and tools including Docker and Mesos. We'll look at why Go is perfectly suited to this, our architectural decisions, common pitfalls to avoid, and how microservice architectures can vastly increase both the velocity of development teams and the scalability and fault tolerance of our systems.

mattheath

January 28, 2016
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript

  1. @mattheath
    #VoxxedBerlin
    Platinum Sponsor
    Building Microservice
    Architectures with Go
    Matt Heath
    Mondo

    View full-size slide

  2. monoliths
    traditional dev

    View full-size slide

  3. DATABASE
    APPLICATION

    View full-size slide

  4. DATABASE
    APPLICATION

    View full-size slide

  5. DATABASE
    DATABASES
    APPLICATION

    View full-size slide

  6. DATABASE
    DATABASES
    APPLICATION
    SEARCH

    View full-size slide

  7. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH

    View full-size slide

  8. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH
    CAT GIFS

    View full-size slide

  9. ALL
    HAIL
    THE
    MONOLITH

    View full-size slide

  10. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH
    CAT GIFS

    View full-size slide

  11. Microservices
    at Mondo?

    View full-size slide

  12. Single Responsibility
    Principle

    View full-size slide

  13. Bounded Context

    View full-size slide

  14. Well defined
    Interfaces

    View full-size slide

  15. Composability

    View full-size slide

  16. Getting started

    View full-size slide

  17. LOAD BALANCER

    View full-size slide

  18. LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  19. TRANSPORT
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  20. SERVICE SERVICE SERVICE
    TRANSPORT
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  21. SERVICE SERVICE SERVICE
    TRANSPORT
    DATABASE DATABASE DATABASE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  22. SERVICE SERVICE SERVICE
    TRANSPORT
    DATABASE DATABASE DATABASE
    LOAD BALANCER
    API GATEWAY

    View full-size slide

  23. APPLICATION
    LOAD BALANCER

    View full-size slide

  24. API GATEWAY
    APPLICATION
    LOAD BALANCER

    View full-size slide

  25. API GATEWAY
    APPLICATION
    LOAD BALANCER

    View full-size slide

  26. API GATEWAY
    APPLICATION
    LOAD BALANCER
    SERVICE
    SERVICE
    SERVICES

    View full-size slide

  27. Simple
    Static typing
    Static linking

    View full-size slide

  28. Comprehensive stdlib
    eg. Networking

    View full-size slide

  29. Go Kit
    micro
    gRPC
    Kite

    View full-size slide

  30. mondough/typhon
    mondough/mercury

    View full-size slide

  31. LOAD BALANCER

    View full-size slide

  32. LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  33. API
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  34. /webhooks —-> Webhook API

    View full-size slide

  35. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  36. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  37. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    DATABASE

    View full-size slide

  38. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    DATABASE
    DATABASE

    View full-size slide

  39. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    DATABASE
    DATABASE
    EXTERNAL
    PROVIDER

    View full-size slide

  40. Logic
    Handlers
    Storage
    SERVICE

    View full-size slide

  41. mercury
    Logic
    Handlers
    Storage
    SERVICE

    View full-size slide

  42. type Handler func(request) (response, error)

    View full-size slide

  43. type Transport interface {
    ...
    }

    View full-size slide

  44. type Transport interface {
    Listen(serviceName string, inbound chan<- request) error
    StopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    ...
    }

    View full-size slide

  45. type Transport interface {
    Listen(serviceName string, inbound chan<- request) error
    StopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    ...
    }

    View full-size slide

  46. mercury
    Logic
    Handlers
    Storage
    SERVICE

    View full-size slide

  47. mercury
    Logic
    Handlers
    Storage libraries
    SERVICE

    View full-size slide

  48. mercury
    Logic
    Handlers
    Storage libraries
    SERVICE
    Deployment
    Service Discovery
    Configuration
    Monitoring
    Authentication
    Authorisation
    Storage
    Circuit Breaking

    View full-size slide

  49. Statically
    Compiled

    View full-size slide

  50. Root CA
    Certs

    View full-size slide

  51. Making our
    service reliable

    View full-size slide

  52. Event Driven
    Architecture

    View full-size slide

  53. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  54. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  55. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  56. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    SERVICE
    C
    SERVICE
    D
    E

    View full-size slide

  57. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    SERVICE
    C
    SERVICE
    D
    G
    E
    F

    View full-size slide

  58. ?
    ? ?
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    ?
    ?
    ??
    ?
    ?
    ?
    ?
    ?
    ?
    ?

    View full-size slide

  59. Topology
    Management

    View full-size slide

  60. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  61. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  62. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  63. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    SLOW /
    ERRORS

    View full-size slide

  64. Fanout &
    Cancellation

    View full-size slide

  65. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  66. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  67. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  68. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  69. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  70. Context
    Propagation

    View full-size slide

  71. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  72. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  73. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  74. Load
    Failure
    Degradation

    View full-size slide

  75. Monitor your
    Business Logic

    View full-size slide

  76. ♥ customers

    View full-size slide

  77. In-Band vs
    Out of Band

    View full-size slide

  78. type Checker func() (error, map[string]string)

    View full-size slide

  79. Distributed Tracing

    View full-size slide

  80. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  81. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  82. 8096820c-3b7b-47ec-bce6-1c239252ab40

    View full-size slide

  83. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  84. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  85. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  86. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  87. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  88. api
    api
    api.customer
    api.customer
    service.customer
    service.customer
    SEND
    RECV
    SEND
    RECV
    RECV
    SEND
    RECV
    SEND

    View full-size slide

  89. api api.customer service.customer
    SEND
    RECV
    SEND
    RECV
    RECV
    SEND
    RECV
    SEND
    phosphor

    View full-size slide

  90. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  91. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  92. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  93. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  94. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  95. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  96. Small
    Simple
    Easy to learn

    View full-size slide

  97. Concurrency
    Interfaces
    Networking

    View full-size slide

  98. Starting with
    Microservices?

    View full-size slide

  99. #voxxedberlin
    Thanks!
    @mattheath
    @getmondo

    View full-size slide

  100. ATM: Thomas Hawk

    Bank of Commerce: ABQ Museum Archives
    IBM System/360: IBM
    Absorbed: Saxbald Photography
    Orbital Ion Cannon: www.rom.ac
    Go Gopher: Renee French
    Control Room: NASA
    ATM Failure: George Redgrave
    Credits

    View full-size slide