Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Getting Superman X-Ray Vision: Bringing Observa...
Search
Ricardo Ferreira
March 26, 2019
Programming
0
510
Getting Superman X-Ray Vision: Bringing Observability to your Stream Processing
Ricardo Ferreira
March 26, 2019
Tweet
Share
More Decks by Ricardo Ferreira
See All by Ricardo Ferreira
Elastic Community Conference CfP Clinic
riferrei
0
64
Criando Provedores Customizados para Terraform
riferrei
0
80
From Zero to Hero with Observability in Microservices
riferrei
0
110
Construíndo seu Primeiro Provider para Terraform
riferrei
2
530
Understanding Streaming Data and Analytics with Apache Kafka®
riferrei
1
550
Kafka Meetup from March 2020
riferrei
1
380
BU Talk Fev 2020
riferrei
0
160
Boston Meetup, Fev 2020
riferrei
0
56
Building Event Streaming Applications with Serverless Kafka
riferrei
1
470
Other Decks in Programming
See All in Programming
Benchmark
sysong
0
280
NPOでのDevinの活用
codeforeveryone
0
480
Blazing Fast UI Development with Compose Hot Reload (droidcon New York 2025)
zsmb
1
270
地方に住むエンジニアの残酷な現実とキャリア論
ichimichi
5
1.4k
既存デザインを変更せずにタップ領域を広げる方法
tahia910
1
250
Create a website using Spatial Web
akkeylab
0
310
AIプログラマーDevinは PHPerの夢を見るか?
shinyasaita
1
180
アンドパッドの Go 勉強会「 gopher 会」とその内容の紹介
andpad
0
280
GitHub Copilot and GitHub Codespaces Hands-on
ymd65536
1
130
PHPでWebSocketサーバーを実装しよう2025
kubotak
0
240
ソフトウェア品質を数字で捉える技術。事業成長を支えるシステム品質の マネジメント
takuya542
0
280
WebViewの現在地 - SwiftUI時代のWebKit - / The Current State Of WebView
marcy731
0
100
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Producing Creativity
orderedlist
PRO
346
40k
YesSQL, Process and Tooling at Scale
rocio
173
14k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.3k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.4k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Facilitating Awesome Meetings
lara
54
6.4k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Music & Morning Musume
bryan
46
6.6k
How STYLIGHT went responsive
nonsquared
100
5.6k
Docker and Python
trallard
44
3.4k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
Transcript
Getting Superman X-Ray Vision: Bringing Observability to your Stream Processing
Ricardo Ferreira, Developer Advocate
About Me: • Hi, my name is Ricardo Ferreira •
Developer Advocate @ Confluent • Ex-Oracle, Red Hat, IONA Tech • Currently ~70% Dev, ~30% Ops • https://riferrei.net @riferrei
Stream Processing Observability Challenges Best Practices for Kafka • Part
1: • Part 2: Agenda • Part 3:
Stream Processing Observability Challenges Best Practices for Kafka • Part
1: • Part 2: Agenda • Part 3:
Let’s Tweet the Song! 1. Access your Twitter account. 2.
Use #dotcnz in your tweet for correct tracking of your guess. 3. The song name must be within brackets as shown below.
Apache Kafka and Stream Processing O'Reilly Books
Application details: • AWS and Terraform • Confluent Cloud Cluster
• Spring Boot Application • Apache Kafka Connect • Kafka Streams / KSQL • Redis Cache • AWS Lambda • Amazon Alexa
Application details: • AWS and Terraform • Confluent Cloud Cluster
• Spring Boot Application • Apache Kafka Connect • Kafka Streams / KSQL • Redis Cache • AWS Lambda • Amazon Alexa You can find the source-code of this application here:
Stream Processing Observability Challenges Best Practices for Kafka • Part
1: • Part 2: Agenda • Part 3:
Why Observability is Important for Modern Software Architectures? Because it
allows us to focus on solving problems instead of finding where the problem is…
OpenTracing has the elements to implement tracing transparently • Multiple
programming languages • Support for multiple tracers • Large community supporting it
Does it Works with Software Built using Microservices? Of course!
A B
So… What are the Problems? • Collaboration between teams •
Microservices and Serverless • Binary Protocols with no Headers • Pub/Sub based Architectures • Source-Code Unavailability
So… What are the Problems? • Collaboration between teams •
Microservices and Serverless • Binary Protocols with no Headers • Pub/Sub based Architectures • Source-Code Unavailability Stream Processing Arena
Stream Processing Observability Challenges Best Practices for Kafka • Part
1: • Part 2: Agenda • Part 3:
Headers Support on Kafka 0.11.X • Span context can be
set there • All headers are persistent • Headers are agnostic of schema
OpenTracing Contributions • Apache Kafka Instrumentation • Provides support for:
◦ Decorators ◦ Interceptors ◦ Spring Framework ◦ Kafka Streams You can find the source-code of this application here:
Tracing Support for Closed JVMs • Jaeger Tracing support for
REST Proxy, Connect and KSQL • Based on the interceptors model • Instantiate its own tracers based on JSON configuration file set as a environment variable You can find the source-code of this application here:
19 Please Stay in Touch: @riferrei riferrei riferrei
[email protected]
https://riferrei.net
https://cnfl.io/slack
None