Apache Kafka Stream Processing

Talk for JVM Meetup #5

https://www.eventbrite.com/e/jvm-meetup-x-tech-it-easy-stream-processing-with-apache-kafka-tickets-37022947633

Videos at https://www.youtube.com/watch?v=zikZEOPDXGk

Transcript

1. STREAM PROCESSING APACHE KAFKA
2. WHO AM I? ALEX XANDRA ALBERT SIM ‣ Principal Software Development Engineer at blibli.com ‣ Part of the R&D Team ‣ alex.sim@gdn-commerce.com ‣ bertzzie(.sim)
3. MAU BAHAS APA? TERSERAH… AGENDA ‣ Stream Core Concept ‣ Stream vs Subscription ‣ Simple Example: Word count ‣ More complicated example: Dota2 Replay Parser
4. STREAM CORE CONCEPT WHAT IS A STREAM? unbounded, continuously updating data set ordered, replayable, and fault-tolerant sequence of immutable data records key-value pair
5. STREAM CORE CONCEPT STREAM PROCESSOR? input -> process -> output …continuously
6. STREAM CORE CONCEPT
7. STREAM CORE CONCEPT GUARANTEE at least once exactly once (0.11.0.0+) defaults to “at least once”
8. STREAM CORE CONCEPT
9. STREAM CORE CONCEPT
10. STREAM VS SUBSCRIPTION WHAT’S THE DIFFERENCE?
11. STREAM VS SUBSCRIPTION KAFKA DATA SOURCE APP SUBSCRIBER APP publish subscribe OUTPUT SUBSCRIPTION
12. STREAM VS SUBSCRIPTION KAFKA DATA SOURCE APP STREAM PROCESSOR publish stream STREAM stream
13. STREAM VS SUBSCRIPTION NOTE THAT: ‣ Stream API is built on top of Subscription API ‣ Mostly the difference is on feature and convenience ‣ e.g. stream has interactive query, functional DSL, event- processing, etc ‣ You can do this all in Subscription API if you want to
14. DEMO 1 SIMPLE STREAM APPLICATION
15. PRE-DEMO 2 WHAT WE’RE GOING TO DO ‣ Read Dota2 Replay File ‣ Get hero position every game tick ‣ Stream to count how many times someone move to certain coordinate ‣ Useful for analysis e.g. heatmap of coordinate, positioning analysis
16. DEMO 2 DOTA 2 REPLAY PARSER
17. MORE! WANT TO KNOW MORE? ‣ Code is at https://github.com/bertzzie/kafka-clarity ‣ Slides is at https://s.id/jvm5slide2 ‣ Feel free to talk to me anytime (email, social media, etc)! ‣ Also talk to good people at @JVMUserGroup in Telegram
18. WORK WITH US, WE HAVE COOKIES! blibli.com . HMM..
19. QUESTIONS?