Functional Flowing

KotlinX’s Flow data type is extremely powerful, and offers an API that is quite familiar for functional programmers.
This talk will showcase how we can leverage KotlinX Flow to describe powerful programs, and build pipelines to transform and manipulate data in an efficient streaming way.

Transcript

1. Functional Flowing 47deg.com
2. Who am I?
3. KotlinX Flow Pull based Reasoning about Flow Streaming Resources Parallel Processing
4. Pull Based Streaming Upstream Pipeline Downstream
5. Pull Based Streaming Upstream Pipeline Downstream
6. Pull Based Streaming public fun interface FlowCollector<in T> { public suspend fun emit(value: T) }
7. Creating Flows class Circle(val count: Int) suspend fun FlowCollector<Circle>.produceCircles(): Unit { var counter: Int = 0 while(true) { val circle = Circle(counter>+) println("Creating $circle") emit(circle) } }
8. Creating Flows class Circle(val count: Int) suspend fun FlowCollector<Circle>.produceCircles(): Unit { var counter: Int = 0 while(true) { val circle = Circle(counter>+) println("Creating $circle") emit(circle) } } val circles: Flow<Circle> = flow { produceCircles() }
9. Creating Flows class Circle(val count: Int) val circles: Flow<Circle> = flow { var counter: Int = 0 while(true) { val circle = Circle(counter>+) println("Creating $circle") emit(circle) } }
10. Pipeline class Square private constructor(val circle: Circle) { companion object { suspend fun Circle.toSquare(): Square { print("Creating Square($count)") delay(1.seconds) return Square(this).also { println("Created $this") } } } }
11. Putting it all together fun main() = runBlocking { circles .take(3) .collect { circle -> circle.toSquare() } } Creating Circle@63e31ee Creating Square(0) Created Square@6adca536 Creating Circle@23223dd8
12. Reasoning about Flows suspend fun FlowCollector<Circle>.produceCircles(): Unit { var counter: Int = 0 while(true) { val circle = Circle(counter>+) println("Creating $circle") emit(circle) } }
13. Reasoning about Flows class Square private constructor(val circle: Circle) { companion object { suspend fun Circle.toSquare(): Square { print("Creating Square($count)") delay(1.seconds) return Square(this).also { println("Created $this") } } } }
14. Reasoning about Flows while(true) 0 1 2 Coroutine
15. Flowing resources fun Path.readAll(): Flow<String>
16. Flowing resources fun Path.readAll(): Flow<String> = flow { >/ this: FlowCollector<String> useLines { lines -> lines.forEach { line -> emit(line) } } }
17. Flowing resources suspend fun insertIntoBlobStorage(line: String): Unit
18. Flowing resources /** Java SDK */ class BlobStorage(val blobName: String) : AutoCloseable { fun uploadBlob(data: ByteArray): CompletableFuture<UploadResult> override fun close() = Unit } suspend fun BlobStorage.insertIntoBlobStorage(line: String): UploadResult = uploadBlob(line.toByteArray(Charsets.UTF_8)).await()
19. Flowing resources suspend fun uploadData(path: String): Unit = BlobStorage(path).use { storage -> Path(path) .readAll() .collect { line -> storage.insertIntoBlobStorage(line) } }
20. Flowing resources fun blobStorage(path: String): Flow<BlobStorage> = flow { BlobStorage(path).use { emit(it) } }
21. Flowing resources suspend fun uploadData2(path: String): Flow<UploadResult> = blobStorage().flatMapConcat { blobStorage -> Path("data.txt") .readAll() .map { line -> blobStorage.insertIntoBlobStorage(line) } }
22. Parallel Processing Read line Upload blob Coroutine …
23. Parallel Processing Read line Coroutine Upload blob Coroutine Channel(3)
24. Parallel Processing suspend fun uploadData3(path: String): Flow<UploadResult> = blobStorage().flatMapConcat { blobStorage -> Path("data.txt") .readAll() .parMapUnordered(concurrency = 3) { line -> blobStorage.insertIntoBlobStorage(line) } }
25. Parallel Processing @FlowPreview public inline fun <A, B> Flow<A>.parMapUnordered( concurrency: Int = DEFAULT_CONCURRENCY, crossinline transform: suspend (a: A) >> B ): Flow<B> { val flowOfFlows: Flow<Flow<B>> = map { o -> flow { emit(transform(o)) } } return parallelFlows.flattenMerge(concurrency) }
26. Parallel Processing suspend fun uploadData3(path: String): Flow<UploadResult> = blobStorage().flatMapConcat { blobStorage -> Path("data.txt") .readAll() .parMap(concurrency = 3) { line -> blobStorage.insertIntoBlobStorage(line) } }
27. Compose-able streaming Automatic back-pressure Conclusion Suspend everywhere
28. Thanks!