Building Microservice Architectures with Go - Microservices Meetup Berlin

Presented at Microservices Meetup Berlin
http://www.meetup.com/Microservices-Meetup-Berlin/events/228032860/

Traditionally applications have been built as monoliths; single applications which become larger and more complex over time, which limit our ability to react to change. An example of this is the banking industry where mergers and acquisitions between banks have lead to a patchwork of different systems & technologies that cost billions of dollars per year to maintain. As a result, the pace of innovation in the banking industry has slowed to a crawl.

At Mondo we're building a new kind of bank, a smart bank that belongs in the 21st century, and we’re building it almost entirely in Go. This talk will cover how we’re developing new core banking systems from scratch backed by a microservice platform written in Go, running across multiple data centres using open source frameworks and tools including Docker and Mesos. We'll look at why Go is perfectly suited to this, our architectural decisions, common pitfalls to avoid, and how microservice architectures can vastly increase both the velocity of development teams and the scalability and fault tolerance of our systems.

Transcript

1. Building Microservice Architectures with Go Matt Heath, Mondo
2. @mattheath
6. monoliths traditional dev
16. ?
17. DATABASE APPLICATION
18. DATABASE APPLICATION
19. DATABASE DATABASES APPLICATION
20. DATABASE DATABASES APPLICATION SEARCH
21. DATABASE DATABASES APPLICATION CACHE SEARCH
22. DATABASE DATABASES APPLICATION CACHE SEARCH CAT GIFS
23. ALL HAIL THE MONOLITH
24. DO NOT WANT
25. DATABASE DATABASES APPLICATION CACHE SEARCH CAT GIFS
26. APPLICATION
29. Microservices at Mondo?
30. Single Responsibility Principle
31. Bounded Context
32. Well defined Interfaces
33. Composability
34. Getting started
35. LOAD BALANCER
36. LOAD BALANCER HTTP API & ROUTING LAYER
37. TRANSPORT LOAD BALANCER HTTP API & ROUTING LAYER
38. SERVICE SERVICE SERVICE TRANSPORT LOAD BALANCER HTTP API & ROUTING LAYER
39. SERVICE SERVICE SERVICE TRANSPORT DATABASE DATABASE DATABASE LOAD BALANCER HTTP API & ROUTING LAYER
40. SERVICE SERVICE SERVICE TRANSPORT DATABASE DATABASE DATABASE LOAD BALANCER API GATEWAY
41. APPLICATION LOAD BALANCER
42. API GATEWAY APPLICATION LOAD BALANCER
43. API GATEWAY APPLICATION LOAD BALANCER
44. API GATEWAY APPLICATION LOAD BALANCER SERVICE SERVICE SERVICES
45. API GATEWAY LOAD BALANCER SERVICE SERVICE SERVICES
47. Simple Static typing Static linking
48. Comprehensive stdlib eg. Networking
49. Concurrency
50. Interfaces
51. Go Kit micro gRPC Kite
52. mondough/typhon mondough/mercury
53. LOAD BALANCER
54. LOAD BALANCER HTTP API & ROUTING LAYER
55. API SERVICE LOAD BALANCER HTTP API & ROUTING LAYER
57. /webhooks —-> Webhook API
58. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER
59. WEBHOOK API AUTH SERVICE WEBHOOK SERVICE LOAD BALANCER HTTP API & ROUTING LAYER
60. WEBHOOK API AUTH SERVICE WEBHOOK SERVICE LOAD BALANCER HTTP API & ROUTING LAYER DATABASE
61. WEBHOOK API AUTH SERVICE WEBHOOK SERVICE LOAD BALANCER HTTP API & ROUTING LAYER DATABASE DATABASE
62. SERVICE
63. Logic Handlers Storage SERVICE
64. mercury Logic Handlers Storage SERVICE
65. mercury Logic Handlers Storage libraries SERVICE
66. func main() { }
67. func main() { }
68. func main() { }
69. func main() { srv := service.Init(service.Config{ Name: "service.account", Description: "Store bank accounts", }) srv.AddEndpoints( handler.Create, handler.Read, handler.List, ) srv.Run() }
70. func main() { srv := service.Init(service.Config{ Name: "service.account", Description: "Store bank accounts", }) srv.AddEndpoints( handler.Create, handler.Read, handler.List, ) srv.Run() }
71. func main() { srv := service.Init(service.Config{ Name: "service.account", Description: "Store bank accounts", }) srv.AddEndpoints( handler.Create, handler.Read, handler.List, ) srv.Run() }
72. type Endpoint struct { Name string Handler Handler Request interface{} Response interface{} }
73. type Endpoint struct { Name string Handler Handler Request interface{} Response interface{} }
74. type Endpoint struct { Name string Handler Handler Request interface{} Response interface{} }
75. type Handler func(request) (response, error)
76. func main() { srv := service.Init(service.Config{ Name: "service.account", Description: "boop", }) srv.AddEndpoints( handler.Create, handler.Read, handler.List, ) srv.Run() }
77. func main() { srv := service.Init(service.Config{ Name: "service.account", Description: "boop", }) srv.AddEndpoints( handler.Create, handler.Read, handler.List, ) srv.Run() }
78. type Server interface { Name() string AddEndpoints(eps ...Endpoint) RemoveEndpoints(eps ...Endpoint) Endpoint(name string) (Endpoint, bool) Endpoints() []Endpoint Start(trans transport.Transport) error Run(trans transport.Transport) Stop() Middleware() []ServerMiddleware SetMiddleware([]ServerMiddleware) AddMiddleware(ServerMiddleware) }
79. type Transport interface { Tomb() *tomb.Tomb Ready() <-chan struct{} Listen(serviceName string, inbound chan<- request) error StopListening(serviceName string) bool Respond(req request, resp response) error Send(req request, timeout time.Duration) (response, error) }
80. type Transport interface { Tomb() *tomb.Tomb Ready() <-chan struct{} Listen(serviceName string, inbound chan<- request) error StopListening(serviceName string) bool Respond(req request, resp response) error Send(req request, timeout time.Duration) (response, error) }
81. type Transport interface { Tomb() *tomb.Tomb Ready() <-chan struct{} Listen(serviceName string, inbound chan<- request) error StopListening(serviceName string) bool Respond(req request, resp response) error Send(req request, timeout time.Duration) (response, error) }
82. type Transport interface { Tomb() *tomb.Tomb Ready() <-chan struct{} Listen(serviceName string, inbound chan<- request) error StopListening(serviceName string) bool Respond(req request, resp response) error Send(req request, timeout time.Duration) (response, error) }
83. mercury Logic Handlers Storage libraries SERVICE Deployment Service Discovery Configuration Monitoring Authentication Authorisation Storage Circuit Breaking
86. Statically Compiled Small Images Root CA Certs
88. Making our service reliable
89. Event Driven Architecture
90. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER
91. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER
92. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER
93. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER SERVICE C SERVICE D E
94. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER SERVICE C SERVICE D G E F
95. ? ? ? LOAD BALANCER HTTP API & ROUTING LAYER ? ? ?? ? ? ? ? ? ? ?
96. Topology Management
97. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
98. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
99. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
100. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE SLOW / ERRORS
106. Fanout & Cancellation
107. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
108. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
109. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
110. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
111. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
112. Context Propagation
113. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
114. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
115. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
120. Testing
121. Load Failure Degradation
122. Monitoring
123. Monitor your Business Logic
124. ♥ customers
125. In-Band vs Out of Band
126. type Checker func() (error, map[string]string)
128. Distributed Tracing
129. api api api.customer api.customer service.customer service.customer
130. api api api.customer api.customer service.customer service.customer
131. 8096820c-3b7b-47ec-bce6-1c239252ab40
132. api api api.customer api.customer service.customer service.customer
133. api api api.customer api.customer service.customer service.customer
134. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
135. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
136. api api api.customer api.customer service.customer service.customer
137. api api api.customer api.customer service.customer service.customer SEND RECV SEND RECV RECV SEND RECV SEND
138. api api.customer service.customer SEND RECV SEND RECV RECV SEND RECV SEND phosphor
140. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
141. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
142. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
143. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
144. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
146. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
150. Small Simple Easy to learn
151. Concurrency Interfaces Networking
152. Downsides?
153. Starting with Microservices?
154. Thanks! @mattheath @getmondo
155. ATM: Thomas Hawk
 Bank of Commerce: ABQ Museum Archives IBM System/360: IBM Absorbed: Saxbald Photography Orbital Ion Cannon: www.rom.ac Go Gopher: Renee French Control Room: NASA ATM Failure: George Redgrave Credits