Tweet
Tweet

Slide 1

Slide 1 text

Making Sense of Sensors with SQL Streaming @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia

Slide 2

Slide 2 text

$ whoami ✓ name: Hans-Peter Grahsl ✓ location: Graz, Austria what I do: ✓ technical trainer @ NETCONOMY ✓ independent consultant & engineer ✓ associate lecturer @ CAMPUS 02 ✓ irregular conference speaker @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 2

Slide 3

Slide 3 text

streaming @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 3

Slide 4

Slide 4 text

"... data processing that is designed with infinite data sets in mind." — Tyler Akidau @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 4

Slide 5

Slide 5 text

Streaming BIG DEAL @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 5

Slide 6

Slide 6 text

EVENTS... events everywhere @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 6

Slide 7

Slide 7 text

NOW! @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 7

Slide 8

Slide 8 text

Streaming alleviates Pain ✓ built for data-in-motion ✓ event-centric & stream-oriented ✓ fast and accurate answers @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 8

Slide 9

Slide 9 text

BIGGEST Challenge? @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 9

Slide 10

Slide 10 text

"Jungle" with many more

Slide 11

Slide 11 text

My choice for today

Slide 12

Slide 12 text

Apache Kafka @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 12

Slide 13

Slide 13 text

Apache Kafka ✓ pub / sub to event streams ✓ (permanently) store event streams ✓ process streams in real-time ➔ horizontal scalability ➔ high fault-tolerance @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 13

Slide 14

Slide 14 text

EVENT STREAMING PLATFORM @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 14

Slide 15

Slide 15 text

APIs for Everything @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 15

Slide 16

Slide 16 text

Everything built for Streaming @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 16

Slide 17

Slide 17 text

Kafka's streaming SQL engine @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 17

Slide 18

Slide 18 text

declarative stream processing language @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 18

Slide 19

Slide 19 text

skyrocketing developer productivity @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 19

Slide 20

Slide 20 text

unleash streaming for the masses @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 20

Slide 21

Slide 21 text

KSQL in a Nutshell ✓ built on top of Kafka Streams ✓ NO(!) coding skills required ✓ SQL only ➔ not embedded ✓ extremely low entry barrier ✓ familiar syntax & semantics ✓ concise & expressive @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 21

Slide 22

Slide 22 text

KSQL in a Nutshell the usual suspects OOTB: ✓ projections, filters ✓ joins, aggregations ✓ windowing something is missing? roll your own: ✓ UDF & UDAF ✓ UDTF pending @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 22

Slide 23

Slide 23 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 23

Slide 24

Slide 24 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 24

Slide 25

Slide 25 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 25

Slide 26

Slide 26 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 26

Slide 27

Slide 27 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 27

Slide 28

Slide 28 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 28

Slide 29

Slide 29 text

KSQL Queries ✓ per-record streaming with ms latency ✓ compiled into Kafka Streams apps ✓ distributed execution on KSQL servers ✓ 2 modes: interactive vs. headless @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 29

Slide 30

Slide 30 text

KSQL Interactive Mode ✓ KSQL servers accessed via REST API ✓ offers ad-hoc stream analytics ✓ share streams & tables across users ✓ used for exploration and during development @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 30

Slide 31

Slide 31 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 31

Slide 32

Slide 32 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 32

Slide 33

Slide 33 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 33

Slide 34

Slide 34 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 34

Slide 35

Slide 35 text

KSQL Headless Mode ✓ application == SQL file ✓ KSQL servers run streaming queries ✓ use case specific isolation ✓ "locked-down" ➡ NO REST API access ✓ used for production deployments @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 35

Slide 36

Slide 36 text

@hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 36

Slide 37

Slide 37 text

SHOW ME! @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 37

Slide 38

Slide 38 text

Demo Scenario @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 38

Slide 39

Slide 39 text

Demo Scenario @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 39

Slide 40

Slide 40 text

Demo Scenario @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 40

Slide 41

Slide 41 text

Demo Scenario @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 41

Slide 42

Slide 42 text

Demo Scenario @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 42

Slide 43

Slide 43 text

Demo Scenario @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 43

Slide 44

Slide 44 text

KSQL in Action @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 44

Slide 45

Slide 45 text

"You think that's a database table you are querying now?" — Morpheus @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 45

Slide 46

Slide 46 text

"Instead, only try to realize the truth... there is no database table." — Spoon Boy @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 46

Slide 47

Slide 47 text

MISSION accomplished @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 47

Slide 48

Slide 48 text

KSQL Wrap-Up ✓ SQL... and nothing but SQL ✓ provides exactly-once semantics ✓ deployable anywhere in any way ✓ no additional infrastructure ✓ use cases of any size (XS ... XXXL) ✓ scalable & fault-tolerant @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 48

Slide 49

Slide 49 text

Your obsession tells you to do batching. I tell you to walk away and stream with KSQL The choice is yours folks! @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 49

Slide 50

Slide 50 text

reach out to me @hpgrahsl @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 50

Slide 51

Slide 51 text

THANK YOU Q & A ? @hpgrahsl | Make IT Conference, 14th October 2019, Portorož, Slovenia 51