Writing Stream Processors in Kotlin KotlinConf 2018 - Amsterdam 

Who of you uses ? 

A quick look over the shoulder 3 

Kafka in a Nutshell Source: Wikipedia 

5 What is so special about Kafka? • Persistent • Time and Log compaction • Guaranteed order per partition 

Messages • Have a byte[] key • Have a byte[] value • Contain Metadata like a timestamp and the partition 

Image from http://kafka.apache.org/documentation/ 

Event Streams • null-key and a bare or serialized value • Compaction of the topic by time • Examples: Clickstreams, Logs, IoT events 

Image from http://cloudurable.com/blog/kafka-architecture-log-compaction/index.html Log Compaction 

Stream Processors • Imagine you could do Kotlin Sequences on an infinite list of messages • Stateful • Parallel • And the output is one or more Kafka topic(s) 

What now? • I do home automation, a lot • I use homee, which aggregates multiple protocols (Z-Wave, ZigBee, etc) • It has no official API, yet • But … They (unintentionally) made one of their repos public on Github • And now I have an API Blueprint file ☺ • Guess what? It’s a websocket! 

Let’s get started • The websocket issues text messages when something happens • Attribute Changes of any connected device (motion, temperature, light, door opens, etc.) • System Events (user created, system update, etc.) • We use Kafka Connect to build what is called a “source” and pipe these events into Kafka. 

Let’s look at some code (no live demo, sorry) 

Now, why Kotlin? The Java code did not look that bad… 

An engineering view • Less verbose, better to read • Extension functions enable composable, high level abstractions • Extensions can be called from the Java folks • One can write libraries with business abstractions, reusable within the company, that blend into the original API 

A team leads view • Onboarding people coming from non-JVM languages to Kotlin is easier than Java • The education material available is very rich, based on current teaching standards • People love the language and the feeling of productivity • Unmodifiable / Immutable helps a lot of people to write more stable code, • resulting in better outcome and higher motivation to go on with learning • because “it just works es expected” • You can write good code even without knowing the language in-depth 

Kotlin at trivago • More than 80% of our Kafka related source code is Kotlin • Kafka Connect Sinks, Transforms, Converters • Stream Processors • Custom Solutions, based on Spring Boot 2, Spring Kafka, Spring Integration • My current team writes client facing REST and GRPC services based on Spring Boot 2 entirely in Kotlin • Our recently relaunched Android App is written entirely in Kotlin • Some of our B2B focused teams started to rewrite their PHP applications in Kotlin 

No content

Curious about how we use Apache Kafka? Kafka and Debezium at trivago (code.talks 2017) https://speakerdeck.com/xenji/kafka-and-debezium-at-trivago-code-dot-talks-2017-edition 

No content