Architecting a Bank from scratch

Matt will talk about how Monzo developed their banking system from scratch and some of the challenges they faced as they grew from a small team with a prototype and a handful of test cards, to a scalable platform with nearly half a million customers.

In the talk we’ll cover how we built a basic prototype to test our assumptions before gaining a banking licence, how we iterated on this to add the functionality in the current Monzo app, and how we used modern technologies including Kubernetes, AWS, Docker, Cassandra, etcd, and Kafka (amongst others), to build a scalable, cloud-native, banking platform. We’ll also look at some of the challenges we had as we used and introduced these technologies, what went well, what didn’t, and some of the lessons we’ve learnt along the way. They’ll also be plenty of time for questions!

Transcript

2. 
 Architecting a Bank
 from scratch Matt Heath, Monzo
3. ! ➡
4. Hi, I’m Matt @mattheath
6. LICENCE WITH RESTRICTIONS WE ARE HERE A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS 50K MAX DEPOSIT WE ARE HERE A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC E APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS WE ARE HERE AUG
 2017 JAN
 2016 Feb
 2015 APR
 2017 A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LICENCE WITH RESTRICTIONS WE ARE HERE
7. A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS 50K MAX DEPOSIT WE ARE HERE A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC E APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS WE ARE HERE AUG
 2017 JAN
 2016 Feb
 2015 APR
 2017 PREPAID
 LAUNCH LICENCE WITH RESTRICTIONS WE ARE HERE A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LICENCE WITH RESTRICTIONS WE ARE HERE
8. Nov 2015 Nov 2017 CUSTOMER
 GROWTH
13. BUILD A PROTOTYPE
14. Henrik Kniberg
15. DEBT EXCHANGE
16. James Leah Matt
17. James Leah Matt
18. James Leah Matt
23. X X X
27. ☕☕
28. ☕☕
29. ?
30. monoliths traditional dev
31. Application
32. Application Database
33. Application Database
34. Application Databases
35. Application Databases Search
36. Application Databases Search Caching
37. Application Databases Search Caching CAT GIFS
38. ALL HAIL THE MONOLITH
40. Application Databases Search Caching CAT GIFS
41. APPLICATION Application
44. LEDGER
 PAYMENT API MOBILE APP
45. Payment API Ledger
46. Payment API Ledger Postgres
47. Transactions Payment API Ledger
48. Transactions Payment API Ledger Accounts
49. Payment API Transactions Ledger Accounts Cassandra Postgres
50. COMMUNICATION BUILD DEPLOYMENT
51. Payment API Transactions Ledger Accounts Cassandra Postgres ? ? ? ? ? ? ? ? ?
52. Henrik Kniberg
58. Payment API Transactions Ledger Accounts Cassandra Postgres
59. Card API Payment API Transactions Ledger Accounts Cassandra Postgres
68. Feb 2015 services Nov 2017 350
70. ?
71. server
72. server server
73. server server server server server server
74. server server server server server server ?
76. pod
77. pod container container container
78. pod transaction service
79. pod web server config volume
80. pod volume
81. pod
82. pod
83. pods replica set
84. replica set pods deployment
85. pods replica set deployment
86. pods replica set deployment change
87. deployment change old pods old replica set
88. deployment change old pods old replica set new replica set
89. deployment change old pods old replica set new replica set new pod
90. deployment change old pods old replica set new replica set new pod
91. deployment change old replica set new pods new replica set
92. deployment change new replica set new pods
93. deployment replica set pods
94. pods
95. pods service
96. pods service kubedns
97. pods service routing - static cluster IP managed replicas resource allocations rollout strategies liveness checks anti-affinity, etc kubedns
98. server server server server server server
99. server server server server server server Kubernetes
100. server server server server server kubelet
101. kubelet kubelet kubelet kubelet kubelet kubelet
102. kubelet kubelet kubelet kubelet kubelet kubelet Kubernetes Master
103. kubelet kubelet kubelet kubelet kubelet kubelet Kubernetes Master etcd
104. kubernetes
106. + = ❤
107. kubernetes service service service service service service service service service service
108. Service Service
109. Service Service RabbitMQ
110. Service Service RabbitMQ
111. Service Service Transport
112. Service Service Transport Service Service
113. Service Service Transport Service Service Client library Server library
114. Service Discovery Load Balancing Timeouts Failure Detection Retries Rate Limiting Circuit Breaking Connection Pooling Context Propagation Metrics and Tracing Request Cancellation
115. Service Service Service Service
116. Service Service Service Service HTTP HTTP
117. Service Service Service Service Kubernetes
 Service
118. Service Discovery Load Balancing Timeouts Failure Detection Retries Rate Limiting Circuit Breaking Connection Pooling Context Propagation Metrics and Tracing Request Cancellation
119. Service Discovery Load Balancing Timeouts Failure Detection Retries Rate Limiting Circuit Breaking Connection Pooling Context Propagation Metrics and Tracing Request Cancellation
120. Service Service Service Service ?
121. Service Service Service Service linkerd
122. Service linkerd
123. Service linkerd service
 discovery
124. Service Service Service Service linkerd service
 discovery
125. Service Service Service Service linkerd
126. Service Service Service Service linkerd
127. Service Service Service Service linkerd
128. Service Service Service Service linkerd
129. THE BIG SWITCH
130. Services Cassandra Load Balancer API
131. Mesos
 RabbitMQ API Cassandra Load Balancer
132. Mesos
 RabbitMQ API Cassandra Load Balancer Proxy
133. Cassandra Load Balancer Mesos
 RabbitMQ API Proxy
134. Cassandra Load Balancer Mesos
 RabbitMQ API Proxy Kubernetes
 HTTP & LinkerD
135. Cassandra Load Balancer Mesos
 RabbitMQ API Proxy Kubernetes
 HTTP & LinkerD
136. Cassandra Load Balancer Proxy Kubernetes
 HTTP & LinkerD
137. Cassandra Load Balancer Kubernetes
 HTTP & LinkerD
139. STRUCTURE
 COMMUNICATION BUILD DEPLOYMENT
141. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
142. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
143. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
144. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
145. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
147. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
150. OTHER CHALLENGES
151. NETWORKS ARE HARD
152. Amazon 
 eu-west-1
153. Amazon 
 eu-west-1
155. Amazon 
 eu-west-1 colocation
 dc 1 colocation
 dc 2
156. Amazon 
 eu-west-1 colocation
 dc 1 colocation
 dc 2
157. Amazon 
 eu-west-1 colocation
 dc 1 colocation
 dc 2
158. Amazon 
 eu-west-1 colocation
 dc 1 colocation
 dc 2 BGP BGP BGP BGP
159. Amazon 
 eu-west-1 colocation
 dc 1 colocation
 dc 2 BGP BGP
160. Amazon 
 eu-west-1 colocation
 dc 1 colocation
 dc 2
161. pods service routing - static cluster IP non-static IP
 /24 range assigned per node
162. third parties colocation “connectivity” pod services HA VPN
 Appliance VPN client Kubernetes
163. third parties colocation “connectivity” pod service HA VPN
 Appliance Kubernetes Wireguard
 network
 interface service pod GUE 
 forwarder routing table
 routes specific
 IP range to local
 gue0 interface GUE 
 listener routing table
 routes packets
 to local wg0
 interface *lots of stuff omitted for brevity, this is a lot more complicated than the diagram implies
164. Amazon 
 eu-west-1 Yay abstraction!
165. NETWORK ISOLATION
168. SECRET STORAGE
169. STATEFUL SERVICES
170. TESTING MICROSERVICES
171. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns