Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Velocity 2017 SJ: Application Tracing
Search
Bryan Liles
June 21, 2017
Programming
2
360
Velocity 2017 SJ: Application Tracing
Workshops slides
Bryan Liles
June 21, 2017
Tweet
Share
More Decks by Bryan Liles
See All by Bryan Liles
DevOps Days MSP 2017
bryanl
2
1.1k
Application Ops 1.0
bryanl
1
200
DevOoops Mastery
bryanl
0
72
Go At Work
bryanl
0
230
The (Ruby) Sims
bryanl
1
130
How Vs Why
bryanl
0
130
ruby -pi -e.bak - Windy City Rails
bryanl
1
270
Arrrr Camp 2012: Simulating the World with Ruby
bryanl
2
510
Other Decks in Programming
See All in Programming
Node-RED を(HTTP で)つなげる MCP サーバーを作ってみた
highu
0
110
GoのGenericsによるslice操作との付き合い方
syumai
3
690
PicoRuby on Rails
makicamel
2
110
地方に住むエンジニアの残酷な現実とキャリア論
ichimichi
5
1.4k
イベントストーミング図からコードへの変換手順 / Procedure for Converting Event Storming Diagrams to Code
nrslib
1
500
A2A プロトコルを試してみる
azukiazusa1
2
1.2k
XP, Testing and ninja testing
m_seki
3
210
システム成長を止めない!本番無停止テーブル移行の全貌
sakawe_ee
1
150
アンドパッドの Go 勉強会「 gopher 会」とその内容の紹介
andpad
0
270
PHPで始める振る舞い駆動開発(Behaviour-Driven Development)
ohmori_yusuke
2
230
Benchmark
sysong
0
270
Create a website using Spatial Web
akkeylab
0
310
Featured
See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Agile that works and the tools we love
rasmusluckow
329
21k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Done Done
chrislema
184
16k
Balancing Empowerment & Direction
lara
1
380
The Pragmatic Product Professional
lauravandoore
35
6.7k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.8k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
60k
Statistics for Hackers
jakevdp
799
220k
Raft: Consensus for Rubyists
vanstee
140
7k
4 Signs Your Business is Dying
shpigford
184
22k
Art, The Web, and Tiny UX
lynnandtonic
299
21k
Transcript
Application Tracing Velocity San Jose 2017
Your instructor •Bryan Liles •
[email protected]
•@bryanl
WIFI: OReilly Conf
Agenda • Workstation Setup • Goals • Tracing Introduction •
Exercise 1 • Break • Exercise 2 • Exercise 3 • Beyond the basics • Discussion • Closing
Workshop Setup • https://github.com/bryanl/velocity-sj-2017 • http://bit.ly/vsj2017-apptracing
• Jaeger • Language Specific Exercises • Jaeger Demo App
• Postgres
Today’s applications are complex
Shop
Shop Inventory Management Shipping
Shop Inventory Management Shipping Redis Postgres
How do we know what’s going on?
Shop
Shop Metrics For every resource, check utilization, saturation, and errors.
• Average busy time • Amount of capacity available • Error count
Shop Logging Capture events to help identify incidents and application
specific data • Auth success/failure • Validation failures • So many more
Shop Tracing Capture timing metrics from one or multiple resources
participating in a transaction. • Find latency issues and errors across resources
Shop Tracing
Tracing Request scoped Metrics Aggregates Logging Events Request scoped events
Aggregate events Request scoped metrics
Tracing Request scoped Metrics Aggregates Logging Events Prometheus ELK OpenTracing
Tracing
None
Terminology
Span: A period of time A span contains the following:
* Operation name * Start/Finish timestamps * Tags * Logs * References to other spans
Trace: a directed acyclic graph (DAG) of spans Span A
Span B Span C Span D Span E Span F Span G Span H
Trace: a DAG of spans Time ➡ Span A Span
B Span D Span C Span E Span F Span G Span H
OpenTracing
Google Dapper "We built dapper to provide Google’s developers with
more information about the behavior of complex distributed systems"
Google Dapper Design Goals • Low Overhead • Application-level transparency
• Scalability
Zipkin • Created by Twitter • Zipkin and Dapper are
based on the same principals
OpenTracing isn’t a framework
OpenTracing describes an abstraction for tracing distributed systems
Jaeger: OpenTracing Compatible Implementation
Jaeger Driver OpenTracing API App Jaeger
Standardization Why OpenTracing?
• Standardize span management • Standardize inter-process propagation • Standardize
active span management • Standardize in-band context encoding • Standardize out-of-band trace data encoding
Why Standardize? • Tracing should be easy and unobtrusive •
Competing standards dilute progress • The ecosystem is complex and diverse. It’s easier to drive to a single standard rather than having multiple.
Open Tracing Demo
pip install -U docker-compose
Workshop Setup
Activity: Tracing Concepts
Talking to Jaeger
Reporter Sampler Jaeger
Activity: More Tracing Concepts
Discussion / Lessons Learned
What should I log or tag? https://github.com/opentracing/specification/blob/master/semantic_conventions.md
Instrumenting
Searching for tags or events
Baggage Sometimes, you want to make data available for child
spans
Closing