Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Contexts in Context

Contexts in Context

The context type allows deadlines, cancelation signals, and other request-scoped values, to propagate through request chains and across API boundaries, but in practice how can these features be used? In this talk Matt will talk about how Monzo use the humble context to power a variety of features in their Go based microservice platform.

mattheath

April 17, 2019
Tweet

More Decks by mattheath

Other Decks in Programming

Transcript

  1. Contexts in Context
    Matt Heath

    View full-size slide

  2. ➡ ➡ ❓ ➡

    View full-size slide

  3. Hi, I’m Matt

    View full-size slide

  4. Nov
    2015
    Apr
    2019

    View full-size slide

  5. Nov
    2015
    Apr
    2019

    View full-size slide

  6. Nov
    2015
    Apr
    2019

    View full-size slide

  7. Nov
    2015
    Apr
    2019

    View full-size slide

  8. Nov
    2015
    Fastest growing UK Bank
    1.8M Customers
    Apr
    2019

    View full-size slide

  9. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  10. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  11. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  12. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  13. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  14. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  15. ctx := context.Background()

    View full-size slide

  16. ctx := context.Background()
    ctx = context.WithValue(ctx, "TraceID", "boop")

    View full-size slide

  17. ctx := context.Background()
    ctx = context.WithValue(ctx, "TraceID", "boop")
    ctx.Value("TraceID") => "boop"

    View full-size slide

  18. ctx := context.Background()
    ctx = context.WithValue(ctx, "TraceID", "boop")
    ctx.Value("TraceID") => "boop"
    Contexts are immutable

    View full-size slide

  19. func(ctx, ...)

    View full-size slide

  20. func(ctx, ...)
    func(ctx, ...)

    View full-size slide

  21. func(ctx, ...)
    func(ctx, ...)

    View full-size slide

  22. func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)

    View full-size slide

  23. func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)

    View full-size slide

  24. func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)

    View full-size slide

  25. func(ctx, ...)
    req.WithContext(ctx)

    View full-size slide

  26. func(ctx, ...)
    req.WithContext(ctx) Remote API

    View full-size slide

  27. func(ctx, ...)
    req.WithContext(ctx) Remote API

    View full-size slide

  28. func(ctx, ...)
    req.WithContext(ctx) Remote API

    Deadline Exceeded
    err

    View full-size slide

  29. package context
    type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
    }

    View full-size slide

  30. LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    50K MAX
    DEPOSIT
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    E APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    AUG

    2016
    JAN

    2016
    Feb

    2015
    APR

    2017
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE

    View full-size slide

  31. LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    50K MAX
    DEPOSIT
    WE ARE
    HERE
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR MAY JUN JUL
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    E APPLICATION APPLICATION MOBILISATION LAUNCH
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    AUG

    2016
    JAN

    2016
    Feb

    2015
    APR

    2017
    A UK banking licence is authorised by the PRA and
    regulated by the PRA and FCA1, allowing deposit-
    taking and balance sheet lending. Once granted,
    it allows firms to passport across Europe, accessing
    This is followed by a “mobilisation” phase during
    which final capital is raised and IT systems are
    completed, before launching to the public.
    We received a UK banking licence in August 2016!
    FEB
    2015
    JAN
    2016
    JAN
    2017
    MAR FEB FEB
    APR MAR MAR APR
    MAY APR
    JUN MAY
    JUL JUN
    AUG JUL
    SEP AUG
    NOV SEP
    DEC NOV DEC
    PRE APPLICATION APPLICATION MOBILISATION
    LICENCE WITH
    RESTRICTIONS
    WE ARE
    HERE
    2 YEARS AGO!!

    View full-size slide

  32. APPLICATION
    Application

    View full-size slide

  33. Feb
    2015
    Apr
    2019
    1000
    services

    View full-size slide

  34. package slog


    func Error(ctx context.Context, msg string, params ...interface{}) {
    if l := DefaultLogger(); l != nil {
    l.Log(Eventf(DebugSeverity, ctx, msg, params...))
    }
    }

    View full-size slide

  35. package slog


    func Error(ctx context.Context, msg string, params ...interface{}) {
    if l := DefaultLogger(); l != nil {
    l.Log(Eventf(DebugSeverity, ctx, msg, params...))
    }
    }

    View full-size slide

  36. func thing(ctx context.Context, ...) {

    // do things...

    slog.Error(ctx, “Oh no!”, err)


    }

    View full-size slide

  37. func thing(ctx context.Context, ...) {

    // do things...

    slog.Error(ctx, “Oh no!”, err)


    }

    View full-size slide

  38. 2019-04-17 15:38:09.134539276 [DBG] Slog -> Auth request:

    {"auth_type": "Auth", "merchant": {"merchant_description": "Deliveroo! LONDON UK",
    "merchant_id": "XXXX"}, "network": “Mastercard" ...}

    {opentracing_id=2855967a9de91afb, trace_id=19ce030f-f652-4b4d-5f35-5d75f472f482,
    service=service.blah, service_version=d7b742fe5c03998748348fe9d7d81a6b8c7358d, hostname=s-
    blah-79f9b54887-j62d2}


    (handle_auth_message.go:25 handleAuthMessage)

    View full-size slide

  39. 2019-04-17 15:38:09.134539276 [DBG] Slog -> Auth request:

    {"auth_type": "Auth", "merchant": {"merchant_description": “Deliveroo! LONDON UK",
    "merchant_id": "XXXX"}, "network": “Mastercard" ...}

    {opentracing_id=2855967a9de91afb, trace_id=19ce030f-f652-4b4d-5f35-5d75f472f482,
    service=service.blah, service_version=d7b742fe5c03998748348fe9d7d81a6b8c7358d, hostname=s-
    blah-79f9b54887-j62d2}


    (handle_auth_message.go:25 handleAuthMessage)

    View full-size slide

  40. API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    API
    Service
    Edge Gateway

    View full-size slide

  41. Accounts Cards Pots Feed Payments …
    Edge Gateway

    View full-size slide

  42. Cards Pots Payments …
    Accounts
    New

    Exciting

    API!!





    Feed
    Edge Gateway

    View full-size slide

  43. Cards Pots Payments …
    Accounts
    New

    Exciting

    API!!






    Feed
    Edge Gateway

    View full-size slide

  44. Edge Gateway
    Accounts Cards Pots Payments …
    New

    Exciting

    API!!


    Feed

    View full-size slide

  45. Load Balancer
    Edge Gateway

    View full-size slide

  46. Load Balancer
    Edge Gateway Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482

    View full-size slide

  47. // Generate new ID
    // Store value in context
    tid, err := NewTraceID() // v4 UUID
    if err == nil {
    req.Context = context.WithValue(req.Context, "Trace-Id", tid)
    }

    View full-size slide

  48. // Generate new ID
    // Store value in context
    tid, err := NewTraceID() // v4 UUID
    if err == nil {
    req.Context = context.WithValue(req.Context, "Trace-Id", tid)
    }

    View full-size slide

  49. // Generate new ID
    // Store value in context
    tid, err := NewTraceID() // v4 UUID
    if err == nil {
    req.Context = context.WithValue(req.Context, "Trace-Id", tid)
    }

    View full-size slide

  50. Load Balancer
    Edge Gateway Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482
    Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482
    API
    Service

    View full-size slide

  51. Service
    A
    Service
    B
    Load Balancer
    Edge Gateway Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482
    Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482
    Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482
    API
    Service

    View full-size slide

  52. Service
    Service

    View full-size slide

  53. Service
    Service
    Context to Headers
    Headers to Context
    HTTP

    View full-size slide

  54. // Add headers previously stashed in the context
    if stash, ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil {
    for k, v := range stash {
    if req.Header[k] == nil {
    req.Header[k] = v
    }
    }
    }
    // Stash headers in the context
    stash := http.Header{}
    for k, v := range req.Header {
    for _, r := range passedHeaders {
    if r.MatchString(k) {
    stash[k] = v
    }
    }
    }
    if len(stash) > 0 {
    req.Context = context.WithValue(req.Context, ctxKey, stash)
    }

    View full-size slide

  55. // Add headers previously stashed in the context
    if stash, ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil {
    for k, v := range stash {
    if req.Header[k] == nil {
    req.Header[k] = v
    }
    }
    }
    // Stash headers in the context
    stash := http.Header{}
    for k, v := range req.Header {
    for _, r := range passedHeaders {
    if r.MatchString(k) {
    stash[k] = v
    }
    }
    }
    if len(stash) > 0 {
    req.Context = context.WithValue(req.Context, ctxKey, stash)
    }

    View full-size slide

  56. // Add headers previously stashed in the context
    if stash, ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil {
    for k, v := range stash {
    if req.Header[k] == nil {
    req.Header[k] = v
    }
    }
    }
    // Stash headers in the context
    stash := http.Header{}
    for k, v := range req.Header {
    for _, r := range passedHeaders {
    if r.MatchString(k) {
    stash[k] = v
    }
    }
    }
    if len(stash) > 0 {
    req.Context = context.WithValue(req.Context, ctxKey, stash)
    }

    View full-size slide

  57. // Add headers previously stashed in the context
    if stash, ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil {
    for k, v := range stash {
    if req.Header[k] == nil {
    req.Header[k] = v
    }
    }
    }
    // Stash headers in the context
    stash := http.Header{}
    for k, v := range req.Header {
    for _, r := range passedHeaders {
    if r.MatchString(k) {
    stash[k] = v
    }
    }
    }
    if len(stash) > 0 {
    req.Context = context.WithValue(req.Context, ctxKey, stash)
    }

    View full-size slide

  58. Service
    Service
    Context to Headers
    Headers to Context
    HTTP

    View full-size slide

  59. Service
    Service
    Context to Headers
    Headers to Context

    View full-size slide

  60. Service
    Service



    HTTP
    HTTP
    HTTP

    View full-size slide

  61. Correlate logs
    Observe database queries
    Queue publishing
    Distributed locks

    View full-size slide

  62. Correlate logs
    Observe database queries
    Queue publishing
    Distributed locks
    Understand request flow
    What’s slow?!
    What’s broken?!

    View full-size slide

  63. Service
    Service

    View full-size slide

  64. Service
    Service
    cancel() or ctx.Done()

    View full-size slide

  65. Service
    Service
    x
    cancel() or ctx.Done()
    Terminate connection (http1)

    Cancel request (http2)

    View full-size slide

  66. Service
    Service
    cancel() or ctx.Done()
    context cancelled
    Terminate connection (http1)

    Cancel request (http2)
    x

    View full-size slide

  67. Cancel downstream requests

    View full-size slide

  68. Cancel outstanding work!
    Decrease tail latencies!
    Reduce load on your database!

    View full-size slide

  69. Cancel outstanding work!
    Decrease tail latencies!
    Reduce load on your database!

    View full-size slide

  70. rspCh := make(chan Response, 1)
    go dispatchRequest(ctx, rspCh)

    View full-size slide

  71. rspCh := make(chan Response, 1)
    go dispatchRequest(ctx, rspCh)
    select {
    case rsp := <-rspCh:
    return rsp
    case <-ctx.Done():
    return ctx.Error()
    }

    View full-size slide

  72. rspCh := make(chan Response, 1)
    go dispatchRequest(ctx, rspCh)
    select {
    case rsp := <-rspCh:
    return rsp
    case <-ctx.Done():
    return ctx.Error()
    }

    View full-size slide

  73. rspCh := make(chan Response, 1)
    go dispatchRequest(ctx, rspCh)
    select {
    case rsp := <-rspCh:
    return rsp
    case <-ctx.Done():
    return ctx.Error()
    }

    View full-size slide

  74. // check if the context is cancelled!

    select {
    case <-ctx.Done():
    return ctx.Error()
    default:
    }
    wooooooo… let’s go!

    View full-size slide

  75. Cancel outstanding work!
    Decrease tail latencies!
    Reduce load on your database!

    View full-size slide

  76. Cancel outstanding work!
    Decrease tail latencies!
    Reduce load on your database!


    Unless operations should complete…

    View full-size slide

  77. Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    API
    Service

    View full-size slide

  78. Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    API
    Service
    Client disconnects

    View full-size slide

  79. Service
    A
    Service
    B
    Load Balancer
    Edge Gateway
    API
    Service
    Client disconnects
    Cascading failure!!
    Errors everywhere!!
    Intense panic!

    View full-size slide

  80. func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)
    func(ctx, ...)

    View full-size slide

  81. func(ctx, ...)
    func(ctx, ...)
    func(context.Background(), ...)
    func(ctx, ...)
    func(ctx, ...)

    View full-size slide

  82. func(ctx, ...)
    func(ctx, ...)
    func(context.Background(), ...)
    func(ctx, ...)
    func(ctx, ...)
    DANGER ZONE

    View full-size slide

  83. Request scoped variables
    Dependency Injection
    Understand your systems.

    View full-size slide

  84. monzo.com/careers

    View full-size slide