Taming I
!
T Data:
Making Sense
!
f Sens
!
rs with
SQL Streaming
Slide 2
Slide 2 text
? streaming ?
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 2
Slide 3
Slide 3 text
"... data processing
that is designed with
infinite data sets
in mind."
— Tyler Akidau
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 3
Slide 4
Slide 4 text
Streaming
is a big deal
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 4
Slide 5
Slide 5 text
EVENTS
...events everywhere
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 5
Slide 6
Slide 6 text
impatient and demanding
NOW!
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 6
Slide 7
Slide 7 text
Streaming Technologies
✓ purpose-built for data-in-motion
✓ events are 1st class citizens
✓ faster results & accurate answers
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 7
Slide 8
Slide 8 text
biggest
Challenge?
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 8
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
not a mess
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 10
Slide 11
Slide 11 text
BUT A MAZE
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 11
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
Apache
Kafka
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 14
Slide 15
Slide 15 text
Apache Kafka
✓ pub / sub to event streams
✓ (permanently) store event streams
✓ process streams in near real-time
➔ horizontal scalability
➔ high fault-tolerance
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 15
Slide 16
Slide 16 text
Event
Streaming
PLATFORM
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 16
Slide 17
Slide 17 text
APIs for Everything
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 17
Slide 18
Slide 18 text
Everything built for Streaming
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 18
Slide 19
Slide 19 text
Kafka's
streaming
SQL
engine
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 19
Slide 20
Slide 20 text
declarative
stream
processing
language
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 20
Slide 21
Slide 21 text
skyrocketing
developer
productivity
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 21
Slide 22
Slide 22 text
unleash
streaming for
the masses
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 22
Slide 23
Slide 23 text
KSQL in your Blood Cell
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 23
Slide 24
Slide 24 text
KSQL in a Nutshell
✓ built on top of Kafka Streams
✓ NO(!) coding skills required
✓ SQL only ➔ not embedded
✓ extremely low entry barrier
✓ familiar syntax & semantics
✓ concise & expressive
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 24
Slide 25
Slide 25 text
KSQL in a Nutshell
the usual suspects OOTB:
✓ projections, filters
✓ joins, aggregations
✓ windowing
something missing?
✓ UDF & UDAF
✓ UDTF pending
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 25
Slide 26
Slide 26 text
KSQL Queries
✓ per-record streaming with ms latency
✓ compiled into Kafka Streams apps
✓ distributed execution: KSQL servers
✓ 2 modes: interactive vs. headless
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 26
Slide 27
Slide 27 text
KSQL's Interactive Mode
✓ KSQL servers accessed via REST API
✓ offers ad-hoc analytics of streams
✓ users can share streams & tables
✓ used for exploration and during development
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 27
Slide 28
Slide 28 text
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 28
Slide 29
Slide 29 text
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 29
Slide 30
Slide 30 text
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 30
Slide 31
Slide 31 text
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 31
Slide 32
Slide 32 text
KSQL'S Headless Mode
✓ application == SQL file
✓ KSQL servers run streaming queries
✓ use case specific isolation
✓ "locked-down" ➡ NO REST API access
✓ used for production deployments
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 32
Slide 33
Slide 33 text
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 33
Slide 34
Slide 34 text
SHOW ME!
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 34
Slide 35
Slide 35 text
Demo Scenario ...
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 35
Slide 36
Slide 36 text
Demo Scenario: Data Ingestion
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 36
Slide 37
Slide 37 text
Demo Scenario: Data Processing
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 37
Slide 38
Slide 38 text
KSQL in Action
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 38
Slide 39
Slide 39 text
"You think that's a
database table
you're querying now?"
— Morpheus
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 39
Slide 40
Slide 40 text
"Instead, only try to
realize the truth - there is
no database table."
— Spoon Boy
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 40
Slide 41
Slide 41 text
Demo Scenario: Data Integration
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 41
Slide 42
Slide 42 text
Demo Scenario: Data Serving
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 42
Slide 43
Slide 43 text
MISSION
accomplished
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 43
Slide 44
Slide 44 text
KSQL Wrap-Up
✓ SQL... and nothing but SQL
✓ use cases of any size (XS ... XXXL)
✓ scalable & fault-tolerant
✓ deployable anywhere in any way
✓ no additional infrastructure
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 44
Slide 45
Slide 45 text
Even if your obsession tells you to do batching,
I'd like you to walk away and stream with
KSQL
The choice is yours folks!
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 45
Slide 46
Slide 46 text
reach out to me
@hpgrahsl
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 46
Slide 47
Slide 47 text
THANK YOU
? Questions ?
https://bit.ly/2NF3CGL
@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 47