Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Bank with Kubernetes – Kubernetes London Meetup, Autumn 2016

Building a Bank with Kubernetes – Kubernetes London Meetup, Autumn 2016

Presented at the Kubernetes London Meetup, Autumn 2016

In this talk, Oliver Beattie, Head of Engineering at Monzo, talks about how Monzo is building a modern bank.

* How Monzo has architected its backend as a collection of microservices
* How the communication between microservices is one of the most important parts of the system
* How Monzo interconnects cloud and overlay networks with physical hardware and third parties

Oliver Beattie

October 19, 2016
Tweet

More Decks by Oliver Beattie

Other Decks in Technology

Transcript

  1. Building a Bank with Kubernetes

    View Slide

  2. Oliver Beattie
    @obeattie
    Head of Engineering, Monzo

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. Application
    Database

    View Slide

  11. Application
    Database Cache

    View Slide

  12. Application
    Database Cache
    Load balancer
    Application
    Application

    View Slide

  13. Application
    Database Cache
    Load balancer
    Cache
    Cache
    Application
    Application

    View Slide

  14. Application
    Database Cache
    Load balancer
    Database
    Database
    Cache
    Cache
    Application
    Application

    View Slide

  15. Application
    Database Cache
    Load balancer
    Application
    Application
    Database
    Database
    Cache
    Cache

    View Slide

  16. Application
    Database Cache
    Load balancer
    Application
    Application
    Database
    Database
    Cache
    Cache

    View Slide

  17. View Slide

  18. View Slide

  19. Efficient
    Resilient
    Extensible
    Secure

    View Slide

  20. View Slide

  21. Efficient
    Resilient
    Extensible
    Secure

    View Slide

  22. View Slide

  23. k8 -worker hosts
    POD #2
    CONTAINER
    POD #1
    CONTAINER
    POD #3
    CONTAINER
    POD #4
    CONTAINER
    1000MB memory; 100% utilised
    250MB 250MB
    250MB
    250MB

    View Slide

  24. Efficient
    Resilient
    Extensible
    Secure

    View Slide

  25. View Slide

  26. View Slide

  27. Load balancing
    Tracing
    Circuit breakers
    Retries
    Canarying
    Service discovery
    Logging
    Timeouts
    Expirations
    Security policies

    View Slide

  28. linkerd

    View Slide

  29. HOST A
    service.
    CONTAINER
    10.224.15.2
    LINKERD
    10.102.32.198
    10.224.15.3
    GET / HTTP/1.1
    Host: service.
    HOST A
    service.cruft
    CONTAINER
    10.224.16.2
    service.cruft
    CONTAINER
    10.224.16.1
    LINKERD
    10.102.34.192
    10.224.16.3
    HOST C
    service.
    CONTAINER
    10.224.17.2
    LINKERD
    10.102.36.187
    10.224.17.3
    service.cruft
    CONTAINER
    10.224.17.1
    service.cruft
    CONTAINER
    10.224.15.1

    View Slide

  30. HOST A
    service.
    CONTAINER
    10.224.15.2
    service.cruft
    CONTAINER
    10.224.15.1
    LINKERD
    10.102.32.198
    10.224.15.3
    Host: service.
    → 10.224.17.3
    GET / HTTP/1.1
    Host: service.
    HOST A
    service.cruft
    CONTAINER
    10.224.16.2
    service.cruft
    CONTAINER
    10.224.16.1
    LINKERD
    10.102.34.192
    10.224.16.3
    HOST C
    service.
    CONTAINER
    10.224.17.2
    service.cruft
    CONTAINER
    10.224.17.1
    LINKERD
    10.102.36.187
    10.224.17.3

    View Slide

  31. HOST A
    service.
    CONTAINER
    10.224.15.2
    service.cruft
    CONTAINER
    10.224.15.1
    LINKERD
    10.102.32.198
    10.224.15.3
    Host: service.
    → 10.224.17.3
    GET / HTTP/1.1
    Host: service.
    HOST A
    service.cruft
    CONTAINER
    10.224.16.2
    service.cruft
    CONTAINER
    10.224.16.1
    LINKERD
    10.102.34.192
    10.224.16.3
    HOST C
    service.
    CONTAINER
    10.224.17.2
    service.cruft
    CONTAINER
    10.224.17.1
    LINKERD
    10.102.36.187
    10.224.17.3

    View Slide

  32. HOST A
    service.
    CONTAINER
    10.224.15.2
    service.cruft
    CONTAINER
    10.224.15.1
    LINKERD
    10.102.32.198
    10.224.15.3
    Host: service.
    → 10.224.17.3
    GET / HTTP/1.1
    Host: service.
    HOST A
    service.cruft
    CONTAINER
    10.224.16.2
    service.cruft
    CONTAINER
    10.224.16.1
    LINKERD
    10.102.34.192
    10.224.16.3
    HOST C
    service.
    CONTAINER
    10.224.17.2
    service.cruft
    CONTAINER
    10.224.17.1
    LINKERD
    10.102.36.187
    10.224.17.3
    Host: service.
    → 10.224.17.2

    View Slide

  33. View Slide

  34. AWS region
    eu-west-1

    View Slide

  35. View Slide

  36. Co-location
    uk-1
    Co-location
    uk-2
    AWS region
    eu-west-1

    View Slide

  37. Co-location
    uk-1
    Co-location
    uk-2
    AWS region
    eu-west-1

    View Slide

  38. Co-location
    uk-1
    Co-location
    uk-2
    AWS region
    eu-west-1


    View Slide

  39. Co-location
    uk-1
    Co-location
    uk-2
    AWS region
    eu-west-1
    BGP
    BGP
    BGP
    BGP

    View Slide

  40. Co-location
    uk-1
    Co-location
    uk-2
    AWS region
    eu-west-1
    BGP
    BGP
    BGP
    BGP
    ⚡ ⚡

    View Slide

  41. Co-location
    uk-1
    Co-location
    uk-2
    AWS region
    eu-west-1
    BGP
    BGP
    ⚡ ⚡

    View Slide

  42. Connectivity Pod
    BGP
    IPSec
    BGP BGP
    Openswan
    (IPSec VPN)
    GNU Zebra

    (BGP)
    Hardware VPN
    device
    Services
    Services
    Services
    AWS Co-location Third parties

    View Slide

  43. Efficient
    Resilient
    Extensible
    Secure

    View Slide

  44. Device isolation
    Process isolation
    Data encryption
    Filesystem isolation
    Privilege isolation
    Network isolation
    Resource isolation
    Principle of least privilege
    Log monitoring
    Secret management

    View Slide

  45. Device isolation
    Process isolation
    Data encryption
    Filesystem isolation
    Privilege isolation
    Network isolation
    Resource isolation
    Principle of least privilege
    Log monitoring
    Secret management

    View Slide

  46. k8s-master
    Availability Zone A Availability Zone B Availability Zone C

    admin
    user
    data
    k8s-worker
    dmz

    View Slide

  47. k8s-master
    Availability Zone A Availability Zone B Availability Zone C

    k8s-worker
    dmz

    View Slide

  48. zone: super-secure

    View Slide

  49. apiVersion: v1
    kind: Namespace
    metadata:
    name: bees

    View Slide

  50. apiVersion: v1
    kind: Namespace
    metadata:
    name: bees
    annotations:
    net.beta.kubernetes.io/network-policy: |
    {
    "ingress": {
    "isolation": "DefaultDeny"
    }
    }

    View Slide

  51. apiVersion: extensions/v1beta1
    kind: NetworkPolicy
    metadata:
    name: super-secure-zone
    spec:
    podSelector:
    matchLabels:
    zone: super-secure
    ingress:
    - from:
    - podSelector:
    matchLabels:
    zone: super-secure
    ports:
    - protocol: tcp

    View Slide

  52. @obeattie

    View Slide