Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Microservice Architectures with Go - Microservices Meetup Berlin

mattheath
January 28, 2016

Building Microservice Architectures with Go - Microservices Meetup Berlin

Presented at Microservices Meetup Berlin
http://www.meetup.com/Microservices-Meetup-Berlin/events/228032860/

Traditionally applications have been built as monoliths; single applications which become larger and more complex over time, which limit our ability to react to change. An example of this is the banking industry where mergers and acquisitions between banks have lead to a patchwork of different systems & technologies that cost billions of dollars per year to maintain. As a result, the pace of innovation in the banking industry has slowed to a crawl.

At Mondo we're building a new kind of bank, a smart bank that belongs in the 21st century, and we’re building it almost entirely in Go. This talk will cover how we’re developing new core banking systems from scratch backed by a microservice platform written in Go, running across multiple data centres using open source frameworks and tools including Docker and Mesos. We'll look at why Go is perfectly suited to this, our architectural decisions, common pitfalls to avoid, and how microservice architectures can vastly increase both the velocity of development teams and the scalability and fault tolerance of our systems.

mattheath

January 28, 2016
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript

  1. Building Microservice
    Architectures with Go
    Matt Heath, Mondo

    View full-size slide

  2. monoliths
    traditional dev

    View full-size slide

  3. DATABASE
    APPLICATION

    View full-size slide

  4. DATABASE
    APPLICATION

    View full-size slide

  5. DATABASE
    DATABASES
    APPLICATION

    View full-size slide

  6. DATABASE
    DATABASES
    APPLICATION
    SEARCH

    View full-size slide

  7. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH

    View full-size slide

  8. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH
    CAT GIFS

    View full-size slide

  9. ALL
    HAIL
    THE
    MONOLITH

    View full-size slide

  10. DATABASE
    DATABASES
    APPLICATION
    CACHE
    SEARCH
    CAT GIFS

    View full-size slide

  11. Microservices
    at Mondo?

    View full-size slide

  12. Single Responsibility
    Principle

    View full-size slide

  13. Bounded Context

    View full-size slide

  14. Well defined
    Interfaces

    View full-size slide

  15. Composability

    View full-size slide

  16. Getting started

    View full-size slide

  17. LOAD BALANCER

    View full-size slide

  18. LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  19. TRANSPORT
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  20. SERVICE SERVICE SERVICE
    TRANSPORT
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  21. SERVICE SERVICE SERVICE
    TRANSPORT
    DATABASE DATABASE DATABASE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  22. SERVICE SERVICE SERVICE
    TRANSPORT
    DATABASE DATABASE DATABASE
    LOAD BALANCER
    API GATEWAY

    View full-size slide

  23. APPLICATION
    LOAD BALANCER

    View full-size slide

  24. API GATEWAY
    APPLICATION
    LOAD BALANCER

    View full-size slide

  25. API GATEWAY
    APPLICATION
    LOAD BALANCER

    View full-size slide

  26. API GATEWAY
    APPLICATION
    LOAD BALANCER
    SERVICE
    SERVICE
    SERVICES

    View full-size slide

  27. API GATEWAY
    LOAD BALANCER
    SERVICE
    SERVICE
    SERVICES

    View full-size slide

  28. Simple
    Static typing
    Static linking

    View full-size slide

  29. Comprehensive stdlib
    eg. Networking

    View full-size slide

  30. Go Kit
    micro
    gRPC
    Kite

    View full-size slide

  31. mondough/typhon
    mondough/mercury

    View full-size slide

  32. LOAD BALANCER

    View full-size slide

  33. LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  34. API
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  35. /webhooks —-> Webhook API

    View full-size slide

  36. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  37. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  38. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    DATABASE

    View full-size slide

  39. WEBHOOK
    API
    AUTH
    SERVICE
    WEBHOOK
    SERVICE
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    DATABASE DATABASE

    View full-size slide

  40. Logic
    Handlers
    Storage
    SERVICE

    View full-size slide

  41. mercury
    Logic
    Handlers
    Storage
    SERVICE

    View full-size slide

  42. mercury
    Logic
    Handlers
    Storage libraries
    SERVICE

    View full-size slide

  43. func main() {
    }

    View full-size slide

  44. func main() {
    }

    View full-size slide

  45. func main() {
    }

    View full-size slide

  46. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "Store bank accounts",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View full-size slide

  47. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "Store bank accounts",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View full-size slide

  48. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "Store bank accounts",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View full-size slide

  49. type Endpoint struct {
    Name string
    Handler Handler
    Request interface{}
    Response interface{}
    }

    View full-size slide

  50. type Endpoint struct {
    Name string
    Handler Handler
    Request interface{}
    Response interface{}
    }

    View full-size slide

  51. type Endpoint struct {
    Name string
    Handler Handler
    Request interface{}
    Response interface{}
    }

    View full-size slide

  52. type Handler func(request) (response, error)

    View full-size slide

  53. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "boop",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View full-size slide

  54. func main() {
    srv := service.Init(service.Config{
    Name: "service.account",
    Description: "boop",
    })
    srv.AddEndpoints(
    handler.Create,
    handler.Read,
    handler.List,
    )
    srv.Run()
    }

    View full-size slide

  55. type Server interface {
    Name() string
    AddEndpoints(eps ...Endpoint)
    RemoveEndpoints(eps ...Endpoint)
    Endpoint(name string) (Endpoint, bool)
    Endpoints() []Endpoint
    Start(trans transport.Transport) error
    Run(trans transport.Transport)
    Stop()
    Middleware() []ServerMiddleware
    SetMiddleware([]ServerMiddleware)
    AddMiddleware(ServerMiddleware)
    }

    View full-size slide

  56. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() <-chan struct{}
    Listen(serviceName string, inbound chan<- request) error
    StopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View full-size slide

  57. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() <-chan struct{}
    Listen(serviceName string, inbound chan<- request) error
    StopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View full-size slide

  58. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() <-chan struct{}
    Listen(serviceName string, inbound chan<- request) error
    StopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View full-size slide

  59. type Transport interface {
    Tomb() *tomb.Tomb
    Ready() <-chan struct{}
    Listen(serviceName string, inbound chan<- request) error
    StopListening(serviceName string) bool
    Respond(req request, resp response) error
    Send(req request, timeout time.Duration) (response, error)
    }

    View full-size slide

  60. mercury
    Logic
    Handlers
    Storage libraries
    SERVICE
    Deployment
    Service Discovery
    Configuration
    Monitoring
    Authentication
    Authorisation
    Storage
    Circuit Breaking

    View full-size slide

  61. Statically Compiled
    Small Images
    Root CA Certs

    View full-size slide

  62. Making our
    service reliable

    View full-size slide

  63. Event Driven
    Architecture

    View full-size slide

  64. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  65. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  66. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER

    View full-size slide

  67. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    SERVICE
    C
    SERVICE
    D
    E

    View full-size slide

  68. API
    SERVICE
    SERVICE
    A
    SERVICE
    B
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    SERVICE
    C
    SERVICE
    D
    G
    E
    F

    View full-size slide

  69. ?
    ? ?
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    ?
    ?
    ??
    ?
    ?
    ?
    ?
    ?
    ?
    ?

    View full-size slide

  70. Topology
    Management

    View full-size slide

  71. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  72. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  73. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  74. WEBHOOK
    API
    LOAD BALANCER
    HTTP API & ROUTING LAYER
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    SLOW /
    ERRORS

    View full-size slide

  75. Fanout &
    Cancellation

    View full-size slide

  76. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  77. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  78. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  79. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  80. WEBHOOK
    API
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE
    WEBHOOK
    SERVICE

    View full-size slide

  81. Context
    Propagation

    View full-size slide

  82. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  83. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  84. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  85. Load
    Failure
    Degradation

    View full-size slide

  86. Monitor your
    Business Logic

    View full-size slide

  87. ♥ customers

    View full-size slide

  88. In-Band vs
    Out of Band

    View full-size slide

  89. type Checker func() (error, map[string]string)

    View full-size slide

  90. Distributed Tracing

    View full-size slide

  91. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  92. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  93. 8096820c-3b7b-47ec-bce6-1c239252ab40

    View full-size slide

  94. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  95. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  96. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  97. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  98. api
    api
    api.customer
    api.customer
    service.customer
    service.customer

    View full-size slide

  99. api
    api
    api.customer
    api.customer
    service.customer
    service.customer
    SEND
    RECV
    SEND
    RECV
    RECV
    SEND
    RECV
    SEND

    View full-size slide

  100. api api.customer service.customer
    SEND
    RECV
    SEND
    RECV
    RECV
    SEND
    RECV
    SEND
    phosphor

    View full-size slide

  101. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  102. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  103. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  104. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  105. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  106. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
    API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns

    View full-size slide

  107. Small
    Simple
    Easy to learn

    View full-size slide

  108. Concurrency
    Interfaces
    Networking

    View full-size slide

  109. Starting with
    Microservices?

    View full-size slide

  110. Thanks!
    @mattheath
    @getmondo

    View full-size slide

  111. ATM: Thomas Hawk

    Bank of Commerce: ABQ Museum Archives
    IBM System/360: IBM
    Absorbed: Saxbald Photography
    Orbital Ion Cannon: www.rom.ac
    Go Gopher: Renee French
    Control Room: NASA
    ATM Failure: George Redgrave
    Credits

    View full-size slide