Save 37% off PRO during our Black Friday Sale! »

Contexts in Context

Contexts in Context

The context type allows deadlines, cancelation signals, and other request-scoped values, to propagate through request chains and across API boundaries, but in practice how can these features be used? In this talk Matt will talk about how Monzo use the humble context to power a variety of features in their Go based microservice platform.

67f4a8f2a209a38d7242829947b26ba3?s=128

mattheath

April 17, 2019
Tweet

Transcript

  1. Contexts in Context Matt Heath

  2. ➡ ➡ ❓ ➡

  3. Hi, I’m Matt

  4. @mattheath

  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. Nov 2015 Apr 2019

  12. Nov 2015 Apr 2019

  13. Nov 2015 Apr 2019

  14. Nov 2015 Apr 2019

  15. Nov 2015 Fastest growing UK Bank 1.8M Customers Apr 2019

  16. None
  17. None
  18. None
  19. None
  20. None
  21. package context type Context interface { Deadline() (deadline time.Time, ok

    bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
  22. package context type Context interface { Deadline() (deadline time.Time, ok

    bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
  23. package context type Context interface { Deadline() (deadline time.Time, ok

    bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
  24. package context type Context interface { Deadline() (deadline time.Time, ok

    bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
  25. package context type Context interface { Deadline() (deadline time.Time, ok

    bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
  26. package context type Context interface { Deadline() (deadline time.Time, ok

    bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
  27. ctx := context.Background()

  28. ctx := context.Background() ctx = context.WithValue(ctx, "TraceID", "boop")

  29. ctx := context.Background() ctx = context.WithValue(ctx, "TraceID", "boop") ctx.Value("TraceID") =>

    "boop"
  30. ctx := context.Background() ctx = context.WithValue(ctx, "TraceID", "boop") ctx.Value("TraceID") =>

    "boop" Contexts are immutable
  31. func(ctx, ...)

  32. func(ctx, ...) func(ctx, ...)

  33. func(ctx, ...) func(ctx, ...)

  34. func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...)

  35. func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...)

  36. func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...)

  37. None
  38. func(ctx, ...) req.WithContext(ctx)

  39. func(ctx, ...) req.WithContext(ctx) Remote API

  40. func(ctx, ...) req.WithContext(ctx) Remote API ⏱

  41. func(ctx, ...) req.WithContext(ctx) Remote API ⏱ Deadline Exceeded err

  42. package context type Context interface { Deadline() (deadline time.Time, ok

    bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
  43. None
  44. None
  45. None
  46. None
  47. LICENCE WITH RESTRICTIONS WE ARE HERE A UK banking licence

    is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS 50K MAX DEPOSIT WE ARE HERE A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC E APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS WE ARE HERE AUG
 2016 JAN
 2016 Feb
 2015 APR
 2017 A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LICENCE WITH RESTRICTIONS WE ARE HERE
  48. LICENCE WITH RESTRICTIONS WE ARE HERE A UK banking licence

    is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS 50K MAX DEPOSIT WE ARE HERE A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY JUN JUL MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC E APPLICATION APPLICATION MOBILISATION LAUNCH LICENCE WITH RESTRICTIONS WE ARE HERE AUG
 2016 JAN
 2016 Feb
 2015 APR
 2017 A UK banking licence is authorised by the PRA and regulated by the PRA and FCA1, allowing deposit- taking and balance sheet lending. Once granted, it allows firms to passport across Europe, accessing This is followed by a “mobilisation” phase during which final capital is raised and IT systems are completed, before launching to the public. We received a UK banking licence in August 2016! FEB 2015 JAN 2016 JAN 2017 MAR FEB FEB APR MAR MAR APR MAY APR JUN MAY JUL JUN AUG JUL SEP AUG NOV SEP DEC NOV DEC PRE APPLICATION APPLICATION MOBILISATION LICENCE WITH RESTRICTIONS WE ARE HERE 2 YEARS AGO!!
  49. None
  50. ?

  51. Application

  52. Application

  53. APPLICATION Application

  54. None
  55. None
  56. Feb 2015 Apr 2019 1000 services

  57. None
  58. None
  59. None
  60. None
  61. None
  62. None
  63. None
  64. None
  65. package slog
 
 func Error(ctx context.Context, msg string, params ...interface{})

    { if l := DefaultLogger(); l != nil { l.Log(Eventf(DebugSeverity, ctx, msg, params...)) } }
  66. package slog
 
 func Error(ctx context.Context, msg string, params ...interface{})

    { if l := DefaultLogger(); l != nil { l.Log(Eventf(DebugSeverity, ctx, msg, params...)) } }
  67. func thing(ctx context.Context, ...) {
 // do things... 
 slog.Error(ctx,

    “Oh no!”, err)
 
 }
  68. func thing(ctx context.Context, ...) {
 // do things... 
 slog.Error(ctx,

    “Oh no!”, err)
 
 }
  69. 2019-04-17 15:38:09.134539276 [DBG] Slog -> Auth request: 
 {"auth_type": "Auth",

    "merchant": {"merchant_description": "Deliveroo! LONDON UK", "merchant_id": "XXXX"}, "network": “Mastercard" ...}
 {opentracing_id=2855967a9de91afb, trace_id=19ce030f-f652-4b4d-5f35-5d75f472f482, service=service.blah, service_version=d7b742fe5c03998748348fe9d7d81a6b8c7358d, hostname=s- blah-79f9b54887-j62d2}
 
 (handle_auth_message.go:25 handleAuthMessage)
  70. 2019-04-17 15:38:09.134539276 [DBG] Slog -> Auth request: 
 {"auth_type": "Auth",

    "merchant": {"merchant_description": “Deliveroo! LONDON UK", "merchant_id": "XXXX"}, "network": “Mastercard" ...}
 {opentracing_id=2855967a9de91afb, trace_id=19ce030f-f652-4b4d-5f35-5d75f472f482, service=service.blah, service_version=d7b742fe5c03998748348fe9d7d81a6b8c7358d, hostname=s- blah-79f9b54887-j62d2}
 
 (handle_auth_message.go:25 handleAuthMessage)
  71. API

  72. API Service API Service API Service API Service API Service

    API Service API Service API Service Edge Gateway
  73. Accounts Cards Pots Feed Payments … Edge Gateway

  74. Cards Pots Payments … Accounts New
 Exciting
 API!! Feed Edge

    Gateway
  75. Cards Pots Payments … Accounts New
 Exciting
 API!! Feed Edge

    Gateway
  76. Edge Gateway Accounts Cards Pots Payments … New
 Exciting
 API!!

    Feed
  77. Load Balancer Edge Gateway

  78. Load Balancer Edge Gateway Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482

  79. // Generate new ID // Store value in context tid,

    err := NewTraceID() // v4 UUID if err == nil { req.Context = context.WithValue(req.Context, "Trace-Id", tid) }
  80. // Generate new ID // Store value in context tid,

    err := NewTraceID() // v4 UUID if err == nil { req.Context = context.WithValue(req.Context, "Trace-Id", tid) }
  81. // Generate new ID // Store value in context tid,

    err := NewTraceID() // v4 UUID if err == nil { req.Context = context.WithValue(req.Context, "Trace-Id", tid) }
  82. Load Balancer Edge Gateway Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482 Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482

    API Service
  83. Service A Service B Load Balancer Edge Gateway Request ID:

    19ce030f-f652-4b4d-5f35-5d75f472f482 Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482 Request ID: 19ce030f-f652-4b4d-5f35-5d75f472f482 API Service
  84. Service Service

  85. Service Service Context to Headers Headers to Context HTTP

  86. // Add headers previously stashed in the context if stash,

    ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil { for k, v := range stash { if req.Header[k] == nil { req.Header[k] = v } } } // Stash headers in the context stash := http.Header{} for k, v := range req.Header { for _, r := range passedHeaders { if r.MatchString(k) { stash[k] = v } } } if len(stash) > 0 { req.Context = context.WithValue(req.Context, ctxKey, stash) }
  87. // Add headers previously stashed in the context if stash,

    ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil { for k, v := range stash { if req.Header[k] == nil { req.Header[k] = v } } } // Stash headers in the context stash := http.Header{} for k, v := range req.Header { for _, r := range passedHeaders { if r.MatchString(k) { stash[k] = v } } } if len(stash) > 0 { req.Context = context.WithValue(req.Context, ctxKey, stash) }
  88. // Add headers previously stashed in the context if stash,

    ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil { for k, v := range stash { if req.Header[k] == nil { req.Header[k] = v } } } // Stash headers in the context stash := http.Header{} for k, v := range req.Header { for _, r := range passedHeaders { if r.MatchString(k) { stash[k] = v } } } if len(stash) > 0 { req.Context = context.WithValue(req.Context, ctxKey, stash) }
  89. // Add headers previously stashed in the context if stash,

    ok := req.Context.Value(ctxKey).(http.Header); ok && stash != nil { for k, v := range stash { if req.Header[k] == nil { req.Header[k] = v } } } // Stash headers in the context stash := http.Header{} for k, v := range req.Header { for _, r := range passedHeaders { if r.MatchString(k) { stash[k] = v } } } if len(stash) > 0 { req.Context = context.WithValue(req.Context, ctxKey, stash) }
  90. Service Service Context to Headers Headers to Context HTTP

  91. Service Service Context to Headers Headers to Context

  92. Service Service ✨ ✨ ✨ HTTP HTTP HTTP

  93. None
  94. None
  95. None
  96. Correlate logs Observe database queries Queue publishing Distributed locks

  97. Correlate logs Observe database queries Queue publishing Distributed locks Understand

    request flow What’s slow?! What’s broken?!
  98. None
  99. None
  100. Service Service

  101. Service Service cancel() or ctx.Done()

  102. Service Service x cancel() or ctx.Done() Terminate connection (http1)
 Cancel

    request (http2)
  103. Service Service cancel() or ctx.Done() context cancelled Terminate connection (http1)


    Cancel request (http2) x
  104. Cancel downstream requests

  105. Cancel outstanding work! Decrease tail latencies! Reduce load on your

    database!
  106. Cancel outstanding work! Decrease tail latencies! Reduce load on your

    database!
  107. rspCh := make(chan Response, 1) go dispatchRequest(ctx, rspCh)

  108. rspCh := make(chan Response, 1) go dispatchRequest(ctx, rspCh) select {

    case rsp := <-rspCh: return rsp case <-ctx.Done(): return ctx.Error() }
  109. rspCh := make(chan Response, 1) go dispatchRequest(ctx, rspCh) select {

    case rsp := <-rspCh: return rsp case <-ctx.Done(): return ctx.Error() }
  110. rspCh := make(chan Response, 1) go dispatchRequest(ctx, rspCh) select {

    case rsp := <-rspCh: return rsp case <-ctx.Done(): return ctx.Error() }
  111. // check if the context is cancelled! 
 select {

    case <-ctx.Done(): return ctx.Error() default: } wooooooo… let’s go!
  112. Cancel outstanding work! Decrease tail latencies! Reduce load on your

    database!
  113. Cancel outstanding work! Decrease tail latencies! Reduce load on your

    database!
 
 Unless operations should complete…
  114. Service A Service B Load Balancer Edge Gateway API Service

  115. Service A Service B Load Balancer Edge Gateway API Service

    Client disconnects
  116. Service A Service B Load Balancer Edge Gateway API Service

    Client disconnects Cascading failure!! Errors everywhere!! Intense panic!
  117. func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...) func(ctx, ...)

  118. func(ctx, ...) func(ctx, ...) func(context.Background(), ...) func(ctx, ...) func(ctx, ...)

  119. func(ctx, ...) func(ctx, ...) func(context.Background(), ...) func(ctx, ...) func(ctx, ...)

    DANGER ZONE
  120. Request scoped variables Dependency Injection Understand your systems.

  121. None
  122. monzo.com/careers