Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Cloud Native Bank

mattheath
October 03, 2018

Building a Cloud Native Bank

A brief look at how Monzo, a UK based digital bank, have leveraged the features of Amazon's public cloud to build a secure, cloud native, banking platform — using modern open source technologies and common software development practices.

Presented at the AWS User Group UK meetup

mattheath

October 03, 2018
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript

  1. Building a
    Cloud-Native Bank
    Matt Heath, Monzo

    View Slide

  2. ! ⛅ #

    View Slide

  3. Hi, I’m Matt

    View Slide

  4. @mattheath

    View Slide

  5. View Slide

  6. View Slide

  7. Nov
    2015
    Oct
    2018
    CUSTOMER

    GROWTH

    View Slide

  8. $
    1,031,133

    View Slide

  9. £4 billion spent
    %

    View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. #

    View Slide

  19. &

    View Slide

  20. View Slide

  21. '

    View Slide

  22. LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    50K MAX
    DEPOSIT
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    E APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    AUG

    2017
    JAN

    2016
    Feb

    2015
    APR

    2017
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE

    View Slide

  23. Henrik Kniberg

    View Slide

  24. View Slide

  25. ?

    View Slide


  26. View Slide


  27. View Slide


  28. View Slide

  29. Henrik Kniberg

    View Slide

  30. James
    Leah
    Matt

    View Slide

  31. James
    Leah
    Matt
    *

    View Slide

  32. James
    Leah
    Matt
    +

    View Slide

  33. ,

    View Slide

  34. View Slide

  35. View Slide

  36. -
    -
    -

    View Slide

  37. -
    -
    -
    X
    X
    X

    View Slide

  38. -

    View Slide

  39. %
    -
    %

    View Slide

  40. .

    View Slide

  41. -
    ☕☕

    View Slide

  42. ☕☕ -
    %

    View Slide


  43. View Slide

  44. EXTENSIBLE
    SCALABLE
    RELIABLE
    SECURE

    CHEAP

    View Slide

  45. ?

    View Slide

  46. Application

    View Slide

  47. Application

    View Slide

  48. APPLICATION
    Application

    View Slide

  49. View Slide

  50. View Slide

  51. View Slide

  52. LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    50K MAX
    DEPOSIT
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    E APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    AUG

    2017
    JAN

    2016
    Feb

    2015
    APR

    2017
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE

    View Slide

  53. A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    50K MAX
    DEPOSIT
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    E APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    AUG

    2017
    JAN

    2016
    Feb

    2015
    APR

    2017
    PREPAID

    CARD

    LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE

    View Slide

  54. View Slide

  55. View Slide

  56. View Slide

  57. View Slide

  58. ?

    View Slide

  59. EC2

    Instance

    View Slide

  60. instance instance

    View Slide

  61. instance instance instance instance instance instance

    View Slide

  62. instance instance instance instance instance instance
    {
    {
    {
    eu-west-1a eu-west-1b eu-west-1c

    View Slide

  63. instance instance instance instance instance instance
    Kubernetes

    View Slide

  64. instance instance instance instance instance
    kubelet

    View Slide

  65. kubelet kubelet kubelet kubelet kubelet kubelet

    View Slide

  66. kubelet kubelet kubelet kubelet kubelet kubelet
    Kubernetes Master

    View Slide

  67. kubelet kubelet kubelet kubelet kubelet kubelet
    Kubernetes Master
    etcd

    View Slide

  68. kubelet kubelet kubelet kubelet kubelet kubelet
    Kubernetes Master
    etcd

    View Slide

  69. etcd

    View Slide

  70. etcd etcd
    etcd

    View Slide

  71. etcd etcd
    etcd
    etcd etcd
    etcd
    etcd etcd
    etcd

    View Slide

  72. etcd etcd
    etcd
    etcd etcd
    etcd
    etcd etcd
    etcd
    quorum = 5 or more nodes alive (n/2+1)

    View Slide

  73. etcd etcd
    etcd
    etcd etcd
    etcd
    etcd etcd
    etcd
    quorum maintained

    View Slide

  74. etcd etcd
    etcd
    etcd etcd
    etcd
    etcd etcd
    etcd
    quorum maintained '

    View Slide

  75. etcd
    etcd
    etcd
    etcd
    etcd
    unhealthy machines terminated 0

    View Slide

  76. etcd etcd
    etcd
    etcd etcd
    etcd
    etcd etcd
    etcd
    new machines automatically reattach EBS, cluster recovers 1

    View Slide

  77. etcd ASG 4 etcd ASG 7
    etcd ASG 1
    etcd ASG 5 etcd ASG 8
    etcd ASG 2
    etcd ASG 3 etcd ASG 6
    etcd ASG 0
    each etcd node is an autoscaling group of 1, with a fixed identity

    View Slide

  78. etcd etcd
    etcd
    etcd etcd
    etcd
    etcd etcd
    etcd
    quorum lost 2

    View Slide

  79. etcd
    etcd
    etcd
    etcd
    unhealthy machines terminated 0

    View Slide

  80. etcd
    etcd
    etcd
    etcd
    etcd ASG 7
    etcd ASG 1
    etcd ASG 8
    etcd ASG 2
    etcd ASG 6
    autoscaling groups start new instances

    View Slide

  81. etcd etcd
    etcd
    etcd etcd
    etcd
    etcd etcd
    etcd
    unhealthy machines replaced automatically, EBS reattached, cluster recovers 3

    View Slide

  82. Blog

    View Slide

  83. kubernetes

    View Slide

  84. kubernetes
    pod pod pod pod pod pod
    pod
    pod
    pod pod

    View Slide

  85. pod
    container
    service

    View Slide

  86. pod
    scratch container
    go service

    View Slide

  87. kubernetes
    service service service service service service
    service
    service
    service service

    View Slide

  88. kubernetes
    service service service service service service
    service
    service
    service service

    View Slide

  89. kubernetes
    service service service service service service
    service
    service
    service service

    View Slide

  90. kubernetes
    service service service service service service
    service
    service
    service service

    View Slide

  91. kubernetes
    service service service service service service
    service
    service
    service service service
    service
    service

    View Slide

  92. kubernetes
    service service service service service
    service service service
    service
    service

    View Slide

  93. kubernetes
    service Prometheus service service Kafka linkerd
    Kafka
    service
    service Elasticsearch calico
    API
    service

    View Slide

  94. 4

    View Slide

  95. 1

    View Slide

  96. Request
    Limit

    View Slide

  97. CPU Throttling

    View Slide

  98. View Slide

  99. API

    View Slide

  100. API

    View Slide

  101. API Gateway
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service

    View Slide

  102. View Slide

  103. View Slide

  104. View Slide

  105. API Gateway
    Accounts Cards Pots Feed Payments …

    View Slide

  106. API Gateway
    Accounts Cards Pots Feed Payments …
    New

    Exciting

    API!!
    5

    View Slide

  107. Cards Pots Payments …
    API Gateway
    Accounts
    New

    Exciting

    API!!
    0
    0
    0
    0
    0
    Feed

    View Slide

  108. Cards Pots Payments …
    API Gateway
    Accounts
    New

    Exciting

    API!!
    0
    0
    0
    0
    0
    6
    Feed

    View Slide

  109. API Gateway
    Accounts Cards Pots Payments …
    New

    Exciting

    API!!
    5
    6
    Feed

    View Slide

  110. Service
    Service

    View Slide

  111. Service
    Service Service
    Service

    View Slide

  112. Service
    Service Service
    Service
    Kubernetes

    Service

    View Slide

  113. Service Discovery
    Load Balancing
    Timeouts and Expirations
    Retries
    Rate Limiting
    Connection Pooling
    Circuit Breaking
    Failure Detection
    Metrics and Tracing
    Interrupts
    Context Propagation

    View Slide

  114. Service Discovery
    Load Balancing
    Timeouts and Expirations
    Retries
    Rate Limiting
    Connection Pooling
    Circuit Breaking
    Failure Detection
    Metrics and Tracing
    Interrupts
    Context Propagation

    View Slide

  115. Service
    Service Service
    Service
    ?

    View Slide

  116. Service
    Service Service
    Service
    linkerd

    View Slide

  117. Service
    linkerd

    View Slide

  118. Service
    linkerd
    service

    discovery

    View Slide

  119. Service
    Service Service
    Service
    linkerd
    service

    discovery

    View Slide

  120. Service
    Service Service
    Service
    linkerd

    View Slide

  121. Service
    Service Service
    Service
    linkerd

    View Slide

  122. Service
    Service Service
    Service
    linkerd

    View Slide

  123. Service
    Service Service
    Service
    linkerd

    View Slide

  124. kind: DaemonSet
    metadata:
    name: linkerd
    spec:
    template:
    spec:
    containers:
    - name: linkerd
    image: ecr:linkerd_vXXX
    - "/etc/linkerd/linkerd.yaml"
    volumeMounts:
    - name: linkerd-config
    mountPath: /etc/linkerd
    readOnly: true
    ports:
    - name: http
    containerPort: 443
    hostPort: 4140
    env:
    - name: POD_IP
    valueFrom:
    fieldRef:
    fieldPath: status.podIP
    resources:
    limits:
    cpu: 4
    memory: 2Gi
    monzo.com/cpu-period: 2500
    requests:
    cpu: 1
    memory: 1Gi
    services send
    requests to local

    linkerd daemonset
    linkerd
    daemonset
    pods & services

    on other machines

    View Slide

  125. View Slide

  126. View Slide

  127. View Slide

  128. Event Driven

    Architecture

    View Slide

  129. Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    API
    Service

    View Slide

  130. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway

    View Slide

  131. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    Service
    D
    Service
    E

    View Slide

  132. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    Service
    C
    Service
    D
    Service
    E

    View Slide

  133. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    Service
    C
    Service
    D
    Service
    E
    6

    View Slide

  134. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    Service
    C
    Service
    D
    Service
    E
    6
    7

    View Slide

  135. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    Service
    C
    Service
    D
    Service
    E
    6
    7
    7
    7
    7

    View Slide

  136. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    Service
    C
    Service
    D
    Service
    E
    6
    7
    7
    7
    7
    Service
    D

    View Slide

  137. API
    Service
    Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    Service
    C
    Service
    D
    Service
    E
    6
    Service
    D

    View Slide

  138. 8

    View Slide

  139. API
    Service
    API
    Service
    Load Balancer
    Monzo API Gateway
    API
    Service

    View Slide

  140. API
    Service
    API
    Service
    Load Balancer
    Monzo API Gateway
    API
    Service

    View Slide

  141. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service

    View Slide

  142. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    AWS Shield
    AWS WAF

    View Slide

  143. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    AWS Shield
    AWS WAF
    AWS API

    View Slide

  144. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    AWS Shield
    AWS WAF
    AWS API

    View Slide

  145. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    AWS Shield
    AWS WAF
    AWS API

    View Slide

  146. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    AWS Shield
    AWS WAF
    AWS API
    Reject traffic
    at ALB
    Update rules

    via API

    View Slide

  147. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    AWS Shield
    AWS WAF
    AWS API
    Reject traffic
    at ALB
    Update rules

    via API

    View Slide

  148. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    AWS Shield
    AWS WAF
    8

    View Slide

  149. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    8

    View Slide

  150. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    8
    service

    View Slide

  151. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    8
    service
    &

    View Slide

  152. View Slide

  153. View Slide

  154. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    8
    service
    VPC
    Segregation

    Network ACLs
    Sec Groups

    View Slide

  155. API
    Service
    API
    Service
    Application Load Balancer (ALB)
    Monzo API Gateway
    API
    Service
    8
    service
    8

    View Slide

  156. instance
    instance instance
    instance instance instance
    instance instance instance
    9

    View Slide

  157. instance
    instance instance
    instance instance instance
    instance instance instance
    9
    IAM
    Auditing
    CloudTrail
    8

    View Slide

  158. View Slide

  159. :

    View Slide

  160. View Slide

  161. Ship it and iterate

    Make changes small

    Make changes often
    Technical debt as a tool

    View Slide

  162. API Gateway
    Accounts Cards Pots Feed Payments …
    New

    Exciting

    API!!
    5

    View Slide

  163. View Slide

  164. Production load tests & Shadow Traffic

    View Slide

  165. EXTENSIBLE
    SCALABLE
    RELIABLE
    SECURE

    CHEAP

    View Slide

  166. Empower teams

    View Slide

  167. monzo.com/careers

    View Slide