Slide 1

Slide 1 text

Taming I ! T Data: Making Sense ! f Sens ! rs with SQL Streaming

Slide 2

Slide 2 text

? streaming ? @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 2

Slide 3

Slide 3 text

"... data processing that is designed with infinite data sets in mind." — Tyler Akidau @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 3

Slide 4

Slide 4 text

Streaming is a big deal @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 4

Slide 5

Slide 5 text

EVENTS ...events everywhere @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 5

Slide 6

Slide 6 text

impatient and demanding NOW! @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 6

Slide 7

Slide 7 text

Streaming Technologies ✓ purpose-built for data-in-motion ✓ events are 1st class citizens ✓ faster results & accurate answers @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 7

Slide 8

Slide 8 text

biggest Challenge? @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 8

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

not a mess @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 10

Slide 11

Slide 11 text

BUT A MAZE @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 11

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Apache Kafka @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 14

Slide 15

Slide 15 text

Apache Kafka ✓ pub / sub to event streams ✓ (permanently) store event streams ✓ process streams in near real-time ➔ horizontal scalability ➔ high fault-tolerance @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 15

Slide 16

Slide 16 text

Event Streaming PLATFORM @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 16

Slide 17

Slide 17 text

APIs for Everything @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 17

Slide 18

Slide 18 text

Everything built for Streaming @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 18

Slide 19

Slide 19 text

Kafka's streaming SQL engine @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 19

Slide 20

Slide 20 text

declarative stream processing language @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 20

Slide 21

Slide 21 text

skyrocketing developer productivity @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 21

Slide 22

Slide 22 text

unleash streaming for the masses @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 22

Slide 23

Slide 23 text

KSQL in your Blood Cell @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 23

Slide 24

Slide 24 text

KSQL in a Nutshell ✓ built on top of Kafka Streams ✓ NO(!) coding skills required ✓ SQL only ➔ not embedded ✓ extremely low entry barrier ✓ familiar syntax & semantics ✓ concise & expressive @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 24

Slide 25

Slide 25 text

KSQL in a Nutshell the usual suspects OOTB: ✓ projections, filters ✓ joins, aggregations ✓ windowing something missing? ✓ UDF & UDAF ✓ UDTF pending @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 25

Slide 26

Slide 26 text

KSQL Queries ✓ per-record streaming with ms latency ✓ compiled into Kafka Streams apps ✓ distributed execution: KSQL servers ✓ 2 modes: interactive vs. headless @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 26

Slide 27

Slide 27 text

KSQL's Interactive Mode ✓ KSQL servers accessed via REST API ✓ offers ad-hoc analytics of streams ✓ users can share streams & tables ✓ used for exploration and during development @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 27

Slide 28

Slide 28 text

@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 28

Slide 29

Slide 29 text

@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 29

Slide 30

Slide 30 text

@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 30

Slide 31

Slide 31 text

@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 31

Slide 32

Slide 32 text

KSQL'S Headless Mode ✓ application == SQL file ✓ KSQL servers run streaming queries ✓ use case specific isolation ✓ "locked-down" ➡ NO REST API access ✓ used for production deployments @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 32

Slide 33

Slide 33 text

@hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 33

Slide 34

Slide 34 text

SHOW ME! @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 34

Slide 35

Slide 35 text

Demo Scenario ... @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 35

Slide 36

Slide 36 text

Demo Scenario: Data Ingestion @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 36

Slide 37

Slide 37 text

Demo Scenario: Data Processing @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 37

Slide 38

Slide 38 text

KSQL in Action @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 38

Slide 39

Slide 39 text

"You think that's a database table you're querying now?" — Morpheus @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 39

Slide 40

Slide 40 text

"Instead, only try to realize the truth - there is no database table." — Spoon Boy @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 40

Slide 41

Slide 41 text

Demo Scenario: Data Integration @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 41

Slide 42

Slide 42 text

Demo Scenario: Data Serving @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 42

Slide 43

Slide 43 text

MISSION accomplished @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 43

Slide 44

Slide 44 text

KSQL Wrap-Up ✓ SQL... and nothing but SQL ✓ use cases of any size (XS ... XXXL) ✓ scalable & fault-tolerant ✓ deployable anywhere in any way ✓ no additional infrastructure @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 44

Slide 45

Slide 45 text

Even if your obsession tells you to do batching, I'd like you to walk away and stream with KSQL The choice is yours folks! @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 45

Slide 46

Slide 46 text

reach out to me @hpgrahsl @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 46

Slide 47

Slide 47 text

THANK YOU ? Questions ? https://bit.ly/2NF3CGL @hpgrahsl | #VoxxedDays Cluj-Napoca, 31st Oct 2019, România 47

Slide 48

Slide 48 text

No content