Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Go and Microservices - NDC London 2018

mattheath
January 27, 2018

Go and Microservices - NDC London 2018

Presented at NDC London 2018

mattheath

January 27, 2018
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript


  1. Go and
    Microservices
    Matt Heath, Monzo

    View Slide

  2. Hi, I’m Matt
    @mattheath

    View Slide

  3. View Slide

  4. View Slide

  5. A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    50K MAX
    DEPOSIT
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    E APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    AUG

    2017
    JAN

    2016
    Feb

    2015
    APR

    2017
    PREPAID

    LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE

    View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. Simple
    Effective
    Scalable

    View Slide

  18. Simplicity

    View Slide

  19. Lightweight
    Concurrency

    View Slide

  20. Goroutines

    View Slide

  21. // Blocking function call
    handleRequest()

    View Slide

  22. // Function runs concurrently
    go handleRequest()

    View Slide

  23. package main
    func main() {
    go handleRequest()
    // …
    }

    View Slide

  24. package main
    func main() {
    go handleRequest()
    // …
    }
    main

    View Slide

  25. package main
    func main() {
    go handleRequest()
    // …
    }
    handleRequest
    main

    View Slide

  26. "Do not communicate by
    sharing memory; instead, share
    memory by communicating."
    - Effective Go

    View Slide

  27. Channels

    View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. View Slide

  32. Digging

    Deeper

    View Slide

  33. Scheduling
    Goroutines

    View Slide

  34. Managing
    Memory

    View Slide

  35. Language
    Stability

    View Slide

  36. Putting it all
    together

    View Slide

  37. But, “microservices”?

    View Slide

  38. Low overheads

    View Slide

  39. Simple deployment

    View Slide

  40. View Slide

  41. View Slide

  42. monzo/typhon

    View Slide

  43. type Service func(req Request) Response

    View Slide

  44. router.GET("/", List)
    router.POST("/", Register)
    router.DELETE("/:id", Deregister)

    View Slide

  45. View Slide

  46. // Handler is your Lambda function handler
    func Handler(request events.APIGatewayProxyRequest)
    (events.APIGatewayProxyResponse, error) {
    ...
    }
    func main() {
    lambda.Start(Handler)
    }

    View Slide

  47. View Slide

  48. ?

    View Slide

  49. Application

    View Slide

  50. Application
    Database

    View Slide

  51. Application
    Database

    View Slide

  52. Application
    Databases

    View Slide

  53. Application
    Databases
    Search

    View Slide

  54. Application
    Databases
    Search
    Caching

    View Slide

  55. Application
    Databases
    Search
    Caching
    CAT GIFS

    View Slide

  56. APPLICATION
    Application

    View Slide

  57. View Slide

  58. View Slide

  59. Feb
    2015
    services Jan
    2017
    400

    View Slide

  60. View Slide

  61. ?

    View Slide

  62. server

    View Slide

  63. server server

    View Slide

  64. server server server server server server

    View Slide

  65. server server server server server server
    ?

    View Slide

  66. View Slide

  67. server server server server server server

    View Slide

  68. server server server server server server
    Kubernetes

    View Slide

  69. server server server server server
    kubelet

    View Slide

  70. kubelet kubelet kubelet kubelet kubelet kubelet

    View Slide

  71. kubelet kubelet kubelet kubelet kubelet kubelet
    Kubernetes Master

    View Slide

  72. kubelet kubelet kubelet kubelet kubelet kubelet
    Kubernetes Master
    etcd

    View Slide

  73. kubernetes

    View Slide

  74. View Slide

  75. + =

    +

    View Slide

  76. kubernetes
    service service service service service service
    service
    service
    service service

    View Slide

  77. Service

    View Slide

  78. Service
    Service

    View Slide

  79. Service
    Service
    Transport

    View Slide

  80. Service
    Service
    Transport
    Service
    Service

    View Slide

  81. Service
    Service
    Transport
    Service
    Service
    Client library
    Server library

    View Slide

  82. Service Discovery
    Load Balancing
    Timeouts
    Failure Detection
    Retries
    Rate Limiting
    Circuit Breaking
    Connection Pooling
    Context Propagation
    Metrics and Tracing
    Request Cancellation

    View Slide

  83. Service
    Service Service
    Service

    View Slide

  84. Service
    Service Service
    Service
    HTTP
    HTTP

    View Slide

  85. Service
    Service Service
    Service
    Kubernetes

    Service

    View Slide

  86. Service Discovery
    Load Balancing
    Timeouts
    Failure Detection
    Retries
    Rate Limiting
    Circuit Breaking
    Connection Pooling
    Context Propagation
    Metrics and Tracing
    Request Cancellation

    View Slide

  87. Service Discovery
    Load Balancing
    Timeouts
    Failure Detection
    Retries
    Rate Limiting
    Circuit Breaking
    Connection Pooling
    Context Propagation
    Metrics and Tracing
    Request Cancellation

    View Slide

  88. Service
    Service Service
    Service
    ?

    View Slide

  89. Service
    Service Service
    Service
    linkerd

    View Slide

  90. Service
    linkerd

    View Slide

  91. Service
    linkerd
    service

    discovery

    View Slide

  92. Service
    Service Service
    Service
    linkerd
    service

    discovery

    View Slide

  93. Service
    Service Service
    Service
    linkerd

    View Slide

  94. Service
    Service Service
    Service
    linkerd

    View Slide

  95. Service
    Service Service
    Service
    linkerd

    View Slide

  96. Service
    Service Service
    Service
    linkerd

    View Slide

  97. Load Balancer

    View Slide

  98. Load Balancer
    Edge Proxy

    View Slide

  99. Load Balancer
    API

    Service
    Edge Proxy

    View Slide

  100. View Slide

  101. /webhooks —-> Webhook API

    View Slide

  102. Load Balancer
    Webhook

    API
    Edge Proxy

    View Slide

  103. Auth

    Service
    Webhook

    Service
    Load Balancer
    Webhook

    API
    Edge Proxy

    View Slide

  104. Auth

    Service
    Webhook

    Service
    Load Balancer
    Webhook

    API
    Database
    Edge Proxy

    View Slide

  105. Database
    Auth

    Service
    Webhook

    Service
    Load Balancer
    Webhook

    API
    Database
    Edge Proxy

    View Slide

  106. External
    Provider
    Database
    Auth

    Service
    Webhook

    Service
    Load Balancer
    Webhook

    API
    Database
    Edge Proxy

    View Slide

  107. View Slide

  108. CONSTRAIN

    COMPLEXITY

    View Slide

  109. SECURITY

    View Slide

  110. OPERATIONAL

    EASE

    View Slide

  111. WHAT HAVE

    WE LEARNT?

    View Slide

  112. Make services
    as small as possible

    View Slide

  113. Prefer services
    over libraries
    and shared code

    View Slide

  114. Prefer consistency
    in most cases

    View Slide

  115. Isolate shared
    infrastructure

    View Slide

  116. Onboarding people
    is hard

    View Slide

  117. Reduce complexity

    View Slide

  118. Monitor EVERYTHING

    View Slide

  119. WHAT WOULD WE

    DO DIFFERENTLY?

    View Slide

  120. View Slide