Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Microservice Architectures with Go - Microservices Meetup Berlin

mattheath
January 28, 2016

Building Microservice Architectures with Go - Microservices Meetup Berlin

Presented at Microservices Meetup Berlin
http://www.meetup.com/Microservices-Meetup-Berlin/events/228032860/

Traditionally applications have been built as monoliths; single applications which become larger and more complex over time, which limit our ability to react to change. An example of this is the banking industry where mergers and acquisitions between banks have lead to a patchwork of different systems & technologies that cost billions of dollars per year to maintain. As a result, the pace of innovation in the banking industry has slowed to a crawl.

At Mondo we're building a new kind of bank, a smart bank that belongs in the 21st century, and we’re building it almost entirely in Go. This talk will cover how we’re developing new core banking systems from scratch backed by a microservice platform written in Go, running across multiple data centres using open source frameworks and tools including Docker and Mesos. We'll look at why Go is perfectly suited to this, our architectural decisions, common pitfalls to avoid, and how microservice architectures can vastly increase both the velocity of development teams and the scalability and fault tolerance of our systems.

mattheath

January 28, 2016
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript

  1. Building Microservice
    Architectures with Go
    Matt Heath, Mondo

    View Slide

  2. @mattheath

    View Slide

  3. View Slide

  4. View Slide

  5. 1895

    View Slide

  6. monoliths
    traditional dev

    View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. ?

    View Slide

  17. DATABASE
    APPLICATION

    View Slide

  18. DATABASE
    APPLICATION

    View Slide

  19. DATABASE
    DATABASES
    APPLICATION

    View Slide

  20. DATABASE
    DATABASES
    APPLICATION
    SEARCH

    View Slide

  21. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH

    View Slide

  22. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH
    CAT GIFS

    View Slide

  23. ALL
    HAIL
    THE
    MONOLITH

    View Slide

  24. DO NOT WANT

    View Slide

  25. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH
    CAT GIFS

    View Slide

  26. APPLICATION

    View Slide

  27. View Slide

  28. View Slide

  29. Microservices
    at Mondo?

    View Slide

  30. Single Responsibility
    Principle

    View Slide

  31. Bounded Context

    View Slide

  32. Well defined
    Interfaces

    View Slide

  33. Composability

    View Slide

  34. Getting started

    View Slide

  35. LOAD BALANCER

    View Slide

  36. LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  37. TRANSPORT
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  38. SERVICE SERVICE SERVICE
    TRANSPORT
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  39. SERVICE SERVICE SERVICE
    TRANSPORT
    DATABASE DATABASE DATABASE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  40. SERVICE SERVICE SERVICE
    TRANSPORT
    DATABASE DATABASE DATABASE
    LOAD BALANCER
    API GATEWAY

    View Slide

  41. APPLICATION
    LOAD BALANCER

    View Slide

  42. API GATEWAY
    APPLICATION
    LOAD BALANCER

    View Slide

  43. API GATEWAY
    APPLICATION
    LOAD BALANCER

    View Slide

  44. API GATEWAY
    APPLICATION
    LOAD BALANCER
    SERVICE
    SERVICE
    SERVICES

    View Slide

  45. API GATEWAY
    LOAD BALANCER
    SERVICE
    SERVICE
    SERVICES

    View Slide

  46. View Slide

  47. Simple
    Static typing
    Static linking

    View Slide

  48. Comprehensive stdlib
    eg. Networking

    View Slide

  49. Concurrency

    View Slide

  50. Interfaces

    View Slide

  51. Go Kit
    micro
    gRPC
    Kite

    View Slide

  52. mondough/typhon
    mondough/mercury

    View Slide

  53. LOAD BALANCER

    View Slide

  54. LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  55. API
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  56. View Slide

  57. /webhooks —-> Webhook API

    View Slide

  58. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  59. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  60. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    DATABASE

    View Slide

  61. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    DATABASE DATABASE

    View Slide

  62. SERVICE

    View Slide

  63. Logic
    Handlers
    Storage
    SERVICE

    View Slide

  64. mercury
    Logic
    Handlers
    Storage
    SERVICE

    View Slide

  65. mercury
    Logic
    Handlers
    Storage libraries
    SERVICE

    View Slide

  66. func main() {
    }

    View Slide

  67. func main() {
    }

    View Slide

  68. func main() {
    }

    View Slide

  69. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "Store bank accounts",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View Slide

  70. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "Store bank accounts",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View Slide

  71. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "Store bank accounts",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View Slide

  72. type Endpoint struct {
    Name string
    Handler Handler
    Request interface{}
    Response interface{}
    }

    View Slide

  73. type Endpoint struct {
    Name string
    Handler Handler
    Request interface{}
    Response interface{}
    }

    View Slide

  74. type Endpoint struct {
    Name string
    Handler Handler
    Request interface{}
    Response interface{}
    }

    View Slide

  75. type Handler func(request) (response, error)

    View Slide

  76. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "boop",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View Slide

  77. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "boop",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View Slide

  78. type Server interface {
    Name() string
    AddEndpoints(eps ...Endpoint)
    RemoveEndpoints(eps ...Endpoint)
    Endpoint(name string) (Endpoint, bool)
    Endpoints() []Endpoint
    Start(trans transport.Transport) error
    Run(trans transport.Transport)
    Stop()
    Middleware() []ServerMiddleware
    SetMiddleware([]ServerMiddleware)
    AddMiddleware(ServerMiddleware)
    }

    View Slide

  79. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() Listen(serviceName string, inbound chanStopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View Slide

  80. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() Listen(serviceName string, inbound chanStopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View Slide

  81. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() Listen(serviceName string, inbound chanStopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View Slide

  82. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() Listen(serviceName string, inbound chanStopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View Slide

  83. mercury
    Logic
    Handlers
    Storage libraries
    SERVICE
    Deployment
    Service Discovery
    Configuration
    Monitoring
    Authentication
    Authorisation
    Storage
    Circuit Breaking

    View Slide

  84. View Slide

  85. View Slide

  86. Statically Compiled
    Small Images
    Root CA Certs

    View Slide

  87. View Slide

  88. Making our
    service reliable

    View Slide

  89. Event Driven
    Architecture

    View Slide

  90. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  91. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  92. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View Slide

  93. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    SERVICE
    C
    SERVICE
    D
    E

    View Slide

  94. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    SERVICE
    C
    SERVICE
    D
    G
    E
    F

    View Slide

  95. ?
    ? ?
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    ?
    ?
    ??
    ?
    ?
    ?
    ?
    ?
    ?
    ?

    View Slide

  96. Topology
    Management

    View Slide

  97. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  98. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  99. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  100. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    SLOW /
    ERRORS

    View Slide

  101. View Slide

  102. View Slide

  103. View Slide

  104. View Slide

  105. View Slide

  106. Fanout &
    Cancellation

    View Slide

  107. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  108. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  109. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  110. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  111. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View Slide

  112. Context
    Propagation

    View Slide

  113. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() Err() error
    Value(key interface{}) interface{}
    }

    View Slide

  114. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() Err() error
    Value(key interface{}) interface{}
    }

    View Slide

  115. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() Err() error
    Value(key interface{}) interface{}
    }

    View Slide

  116. View Slide

  117. View Slide

  118. View Slide

  119. View Slide

  120. Testing

    View Slide

  121. Load
    Failure
    Degradation

    View Slide

  122. Monitoring

    View Slide

  123. Monitor your
    Business Logic

    View Slide

  124. ♥ customers

    View Slide

  125. In-Band vs
    Out of Band

    View Slide

  126. type Checker func() (error, map[string]string)

    View Slide

  127. View Slide

  128. Distributed Tracing

    View Slide

  129. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View Slide

  130. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View Slide

  131. 8096820c-3b7b-47ec-bce6-1c239252ab40

    View Slide

  132. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View Slide

  133. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View Slide

  134. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() Err() error
    Value(key interface{}) interface{}
    }

    View Slide

  135. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() Err() error
    Value(key interface{}) interface{}
    }

    View Slide

  136. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View Slide

  137. api
    api
    api.customer
    api.customer
    service.customer
    service.customer
    SEND
    RECV
    SEND
    RECV
    RECV
    SEND
    RECV
    SEND

    View Slide

  138. api api.customer service.customer
    SEND
    RECV
    SEND
    RECV
    RECV
    SEND
    RECV
    SEND
    phosphor

    View Slide

  139. View Slide

  140. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View Slide

  141. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View Slide

  142. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View Slide

  143. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View Slide

  144. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View Slide

  145. View Slide

  146. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View Slide

  147. View Slide

  148. View Slide

  149. View Slide

  150. Small
    Simple
    Easy to learn

    View Slide

  151. Concurrency
    Interfaces
    Networking

    View Slide

  152. Downsides?

    View Slide

  153. Starting with
    Microservices?

    View Slide

  154. Thanks!
    @mattheath
    @getmondo

    View Slide

  155. ATM: Thomas Hawk

    Bank of Commerce: ABQ Museum Archives
    IBM System/360: IBM
    Absorbed: Saxbald Photography
    Orbital Ion Cannon: www.rom.ac
    Go Gopher: Renee French
    Control Room: NASA
    ATM Failure: George Redgrave
    Credits

    View Slide