STREAM PROCESSING APACHE KAFKA 

WHO AM I? ALEX XANDRA ALBERT SIM ‣ Principal Software Development Engineer at blibli.com ‣ Part of the R&D Team ‣ [email protected] ‣ bertzzie(.sim) 

MAU BAHAS APA? TERSERAH… AGENDA ‣ Stream Core Concept ‣ Stream vs Subscription ‣ Simple Example: Word count ‣ More complicated example: Dota2 Replay Parser 

STREAM CORE CONCEPT WHAT IS A STREAM? unbounded, continuously updating data set ordered, replayable, and fault-tolerant sequence of immutable data records key-value pair 

STREAM CORE CONCEPT STREAM PROCESSOR? input -> process -> output …continuously 

STREAM CORE CONCEPT 

STREAM CORE CONCEPT GUARANTEE at least once exactly once (0.11.0.0+) defaults to “at least once” 

STREAM CORE CONCEPT 

STREAM CORE CONCEPT 

STREAM VS SUBSCRIPTION WHAT’S THE DIFFERENCE? 

STREAM VS SUBSCRIPTION KAFKA DATA SOURCE APP SUBSCRIBER APP publish subscribe OUTPUT SUBSCRIPTION 

STREAM VS SUBSCRIPTION KAFKA DATA SOURCE APP STREAM PROCESSOR publish stream STREAM stream 

STREAM VS SUBSCRIPTION NOTE THAT: ‣ Stream API is built on top of Subscription API ‣ Mostly the difference is on feature and convenience ‣ e.g. stream has interactive query, functional DSL, event- processing, etc ‣ You can do this all in Subscription API if you want to 

DEMO 1 SIMPLE STREAM APPLICATION 

PRE-DEMO 2 WHAT WE’RE GOING TO DO ‣ Read Dota2 Replay File ‣ Get hero position every game tick ‣ Stream to count how many times someone move to certain coordinate ‣ Useful for analysis e.g. heatmap of coordinate, positioning analysis 

DEMO 2 DOTA 2 REPLAY PARSER 

MORE! WANT TO KNOW MORE? ‣ Code is at https://github.com/bertzzie/kafka-clarity ‣ Slides is at https://s.id/jvm5slide2 ‣ Feel free to talk to me anytime (email, social media, etc)! ‣ Also talk to good people at @JVMUserGroup in Telegram 

WORK WITH US, WE HAVE COOKIES! blibli.com . HMM.. 

QUESTIONS?