Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Understand Your System Behavior By Implement Di...

Understand Your System Behavior By Implement Distributed Tracing Using Jaeger

Avatar for Yusuf Syaifudin

Yusuf Syaifudin

December 12, 2018
Tweet

More Decks by Yusuf Syaifudin

Other Decks in Programming

Transcript

  1. Understand Your System Behavior By Implement Distributed Tracing Using Jaeger

    Yusuf Syaifudin - Yogyakarta, December 12th, 2018
  2. Why We Need Tracing? • Even in upfront the API/endpoint

    just looks like request-response model, but it may contains some complicated process behind. Let say about: validation, check data in cache, query in database, or even calling 3rd party service or another internal service. • We may can see log, but reading log generally is hard to see what system’s do in behind.
  3. Log may just look like this: { "level": "info", "client_ip":

    "182.0.201.240", "status": 200, "method": "POST", "endpoint": "/api/v2/sdk/login_or_register", "app_code": "helo-mhbiymsjdqhzwymd", "user_id": 0, "time": "2018-12-12T08:05:46Z", "latency": 191.69, "user_agent": "Chrome/63.0.3239.111", "sdk_version": "WEB_2.6.5-beta3", "sdk_platform": "WEB", "sdk_device_brand": "", "sdk_device_model": "", "sdk_device_os_version": "", "message": "API REQUEST" } Log like this is not define anything, but just a plain log.
  4. “Distributed tracing helps pinpoint where failures occur and what causes

    suboptimal performance.” - From https://searchitoperations.techtarget.com/definition/distributed-tracing
  5. Meet Dora the Explorer If you don’t know who is

    she, watch here: https://www.youtube.com/watc h?v=N33ldfKwdLs
  6. Plot • The series centers around Dora with a love

    of embarking on quests related to an activity that she wants to partake of or a place that she wants to go to. • Each episode is based around a series of cyclical events that occur along the way during Dora's travels, along with obstacles that she and Boots are forced to overcome or puzzles that they have to solve relating to riddles, the Spanish language, or counting. To see which route she should take, she always ask to Map:
  7. What is the relation to tracing in Software Development? Suppose

    that: • Dora’s mission is the one API request in server. • Map is the route from user’s request to response. • Obstacle is like a function. So, to finish one mission/travel, Dora may face several obstacles. Dora the Explorer map template. Source: https://commixus.com/dora-the-explorer-map-template-58475/ (accessed at 2 Dec 2018.)
  8. Do the tracing on Dora’s mission • Suppose that we

    want Dora to report to us what causing her to take the long time and every detail that she may find it good to report. • So, Dora can records the start time and end time of each obstacle, note some additional information when you find it useful to report, etc. Dora will leave a footprint in each of obstacle. Here's the visual:
  9. Let’s code it! • Define each obstacle as a function.

    • Then each function must be delayed in n millisecond so we can visualize it better.
  10. We can call all of defined function like this: serverSpan

    := eCtx.Get("serverSpan").(opentracing.Span) ctx := opentracing.ContextWithSpan( context.Background(), serverSpan, ) defer ctx.Done() … out, _ = startTheJourneyFromTreeHouse(ctx, in) out, _ = passTheForest(ctx, out) out, _ = crossTheLake(ctx, out, isRainyDay) out, _ = enterThePyramid(ctx, out)
  11. When we access Jaeger, it will look something like this:

    We can know easily that the most consumed task is in function crossTheLake when the parameter is_rainy_day is true.