Could Virtual Threads Cast Away the Usage of Kotlin Coroutines

How do Virtual Threads can potentially affect the development of resilient services? If you are implementing services in the JVM, odds are that you are using the Spring Framework. As the development of possibilities for the JVM continues, Spring is constantly evolving with it. This presentation for the London Java Community (LJC) was created to spark that discussion and makes us reflect about out available options so that we can do our best to make the best decisions going forward.

Transcript

1. Could Virtual Threads cast away the usage of Kotlin Coroutines? What experience can tell us… How coroutines have changed the JVM programming world and how Java Virtual Threads can change that? By João Esperancinha 2024/09/10 https://www.eventbrite.co.uk/e/ljc-meet-up-tickets-988279208717
2. Who am I? João is a 10+ years experienced Software Developer, Studied Telecom Engineering at ISEL Lisboa, and worked in different companies in Portugal, Spain, and The Netherlands. He is a certified Kong Champion, a certified Java Developer (OCP11) and is a Certified Spring Professional 2021. Outside the software development Universe, João is someone interested in arts and crafts and making stuff out of old materials creating the occasional modified furniture. João loves plants and has a small potted garden with Yuccas and blueberry bushes. He has one particular YouTube channel called JESPROTECH dedicated to engineering where João shares his experience with the world.
3. Quick Threads History 1995 - Java 1.0 and Green Threads 1998 - Java 1.2 and Transition to Native Threads 2004 - Java 5 (1.5) and Concurrency Utilities 2011 - Kotlin Language Announced 2014 - Project Loom Announced 2017 - Kotlin 1.1 and Introduction of Coroutines 2019 - Project Loom Early Access Builds 2022 - Java 19 and Preview of Virtual Threads 2023 - Java 21 and Official Release of Virtual Threads
4. Fetch all data diagram (Horrendous old way to fetch data for Server Side, but great for mobile … sometimes) Fetching data
5. Structured Concurrency using Virtual Threads try (var scope = new StructuredTaskScope.ShutdownOnFailure()) { var userDeferred = scope.fork(() -> fetchUser(userId)); var postsDeferred = scope.fork(() -> fetchUserPosts(userId)); var commentsDeferred = scope.fork(() -> fetchUserComments(userId)); scope.join(); scope.throwIfFailed(); var processedData = processUserData( userDeferred.get(), postsDeferred.get(), commentsDeferred.get()); updateUI(processedData); logger.info(() -> "Complete!"); } catch (ExecutionException e) { throw new RuntimeException(e); } catch (InterruptedException e) { throw new RuntimeException(e); } Fetching data with Java Virtual Threads (Java) Highly verbose! This is blocking! Alternative: Thread.startVirtualThread ?
6. Structured Concurrency using Virtual Threads ShutdownOnFailure().use { scope -> val userDeferred = scope.fork { fetchUser(userId) } val postsDeferred = scope.fork { fetchUserPosts(userId) } val commentsDeferred = scope.fork { fetchUserComments( userId ) } scope.join() scope.throwIfFailed() val processedData = processUserData( userDeferred.get(), postsDeferred.get(), commentsDeferred.get() ) updateSytem(processedData) logger.info("Complete!") } Fetching data with Java Virtual Threads (Kotlin) Still blocking! Alternative: Thread.startVirtualThread ? Still Highly verbose!
7. Structured Concurrency using Kotlin Coroutines CoroutineScope(IO).launch { val userDeferred = async { fetchUser(userId) } val postsDeferred = async { fetchUserPosts(userId) } val commentsDeferred = async { fetchUserComments(userId) } val user = userDeferred.await() val posts = postsDeferred.await() val comments = commentsDeferred.await() val processedData = processUserData(user, posts, comments) updateSytem(processedData) logger.info("Complete!") } Fetching data Is this not verbose? 🤔 It is better! 😀 Not blocking! 😀
8. Sending messages Send messages diagram (N user messages, packed in 50 messages sent in parallel - Fire and forget)
9. Fire and Forget with Kotlin Coroutines suspend fun sendMessage(text: String, users: List<User>) = coroutineScope { users.chunked(50).map { it.map { async { retry(5, 500) { sendEmail(text, it) } } }.awaitAll() } } Sending Messages Needs a coroutine scope!
10. Fire and Forget with Java Virtual Threads fun sendMessage(text: String, users: List<User>) = users.chunked(50).map { it.map { startVirtualThread { retry(5, 500) { sendEmail(text, it) } } }.forEach { it.join() } } Sending Messages No need for a coroutine scope… but now we wait?🤔 Now it is just a choice! 😀 Example: Thread.startVirtualThread
11. Making tests Using docker-compose 1. services: 2. fulfilment: 3. build: 4. context: . 5. ports: 6. - "8080:8080" 7. expose: 8. - 8080 9. deploy: 10. resources: 11. limits: 12. cpus: 0.5 13. memory: 270M locust -f locust_fulfilment.py --host=http://localhost:8080 Take a part of the machine
12. Python code to make tests Python code called by locust to perform tests from locust import HttpUser, TaskSet, task, between class UserBehavior(TaskSet): @task def get_fulfilment(self): url = "/api/v1/fulfilment" params = {} headers = {"Accept": "application/json"} with self.client.get(url, params=params, headers=headers, catch_response=True) as response: if response.status_code == 200: response.success() print(f"Status Code: {response.status_code}, Response: {response.text[:100]}") else: response.failure(f"Failed with status code: {response.status_code}") class WebsiteUser(HttpUser): tasks = [UserBehavior] wait_time = between(1, 3) def on_start(self): pass def on_stop(self): pass URL + Headers
13. Traditional Spring MVC @RestController @RequestMapping("fulfilment") class FulfilmentController { @GetMapping fun getItems() = (1..10).map { val product = Product(name = "TV", isleType = Room) logger.info("Product: $product") product.name } companion object { val logger: Logger = LoggerFactory.getLogger(FulfilmentController::class.java) } } This was the old way. Or is it? 🤔
14. Locust test - Max 100000 users ; Spawn Rate 1000 users/second (~20 seconds = 20000+ simultaneously - MVC)
15. Traditional Spring MVC Great pattern, but not that capable. Not “fast” enough. • Easy pattern to learn • Good 3 tier model • Model, View, Controller • Domain model • Data transfer object • Converters Problem? • Not many were aware of the inefficient aspect of it • The inefficiency only appeared when raising the load Why was there a need to improve it? • The idea was there all along • When comparing the performance of Spring for higher loads, Spring and other JVM based solutions would fail to be on top. June 2003
16. Reactive Services in Spring MVC - WebFlux @RestController @RequestMapping("fulfilment") class FulfilmentController { @GetMapping fun getItems(): Flux<String> = Flux.fromIterable (1..10).map { val product = Product(name = "TV", isleType = Room) logger.info("Product: $product") product.name } companion object { val logger: Logger = LoggerFactory.getLogger(FulfilmentController::class.java) } } Flux starts reactive revolution, but how good was/is it?
17. Locust test - Max 100000 users ; Spawn Rate 1000 users/second (27 seconds = 27000 + simultaneously - WebFlux)
18. Reactive Programming • Blockhound • Flux / Mono • Flux were used for collections • Mono used for instances • Good support for R2DBC • Good support for Spring • The Observability pattern and publishers were visible Problem? • Coding would get extremely complicated • Particularly zipping functions Spring WebFlux - 2017 (my sea-shell-archiver became too complicated!)_
19. Reactive Services with Spring Kotlin Coroutines @RestController @RequestMapping("fulfilment") class FulfilmentController { @GetMapping fun getItems() = flowOf(1..10).map { val product = Product(name = "TV", isleType = Room) logger.info("Product: $product") product.name } companion object { val logger: Logger = LoggerFactory.getLogger(FulfilmentController::class.java) } } A flow is programmed like flux but the backbone is made with Kotlin Coroutines and it works with the assigned dispatcher by Spring
20. Locust test - Max 100000 users ; Spawn Rate 1000 users/second
21. Why Kotlin Coroutines? Kotlin’s inception year - 2016 • Scope creation instead of a specific structured concurrency block • Structured concurrency • Non-preemptive scheduling • Cooperative multitasking • Better usage of resources • Concurrency • Additional benefit of parallelism • Dispatchers
22. Kotlin Coroutines • Claimed to be simpler than using Flux and Mono • Instead of Mono we started using suspend • Instead collections, we started using flow • The rise of the usage of Channels, hot flows, cold flows • Talks about removing Kafka, RabbitMQ and major dedicated data stream frameworks Problem? • Coroutine complexity was still considerable when using structured concurrency • Because of the non-preemptive model all functions and its instances would have to be marked as suspend • No simple function could call suspend • Many developers still try to use suspend with a returned list or even with flows. In some cases also with Flux. Compatibility with Spring - 2019
23. Coroutines to Virtual Threads (gross simplification)
24. Virtual Threads in Spring # Server server.port=8080 spring.main.web-application-type=servlet spring.mvc.servlet.path=/api/v1 # Enable Virtual Threads for WebFlux spring.threads.virtual.enabled=true # Thread pool configuration #spring.task.execution.pool.max-size=16 #spring.task.execution.pool.queue-capacity=100 #spring.task.execution.pool.keep-alive=10s We can activate virtual threads
25. Virtual Threads in Spring @RestController @RequestMapping("fulfilment") class FulfilmentController { @GetMapping fun getItems() = (1..10).map { val product = Product(name = "TV", isleType = Room) logger.info("Product: $product") product.name } companion object { val logger: Logger = LoggerFactory.getLogger(FulfilmentController::class.java) } } It’s back! And now we are back to the old model!😁
26. Locust test - Max 100000 users ; Spawn Rate 1000 users/second
27. Virtual Threads • Turns everything on its head for Spring • There is no keyword or special objects needed • The application stays reactive • The preemptive/non-preemptive (hybrid) scheduling form makes sure that we can virtualize threads in a code that goes back to the imperative form, without the need to be forced to make major decisions • The use of scopes is off the table and not needed Problem? • Structured concurrency support is very verbose • Many frameworks surrounding Kotlin like Project arrow rely on coroutines for their features. • Streams, channels and flows offer full coroutine support, while it is still cumbersome to do that in Java • Creating mobile applications, namely with android, uses a lot of coroutines Started with Loom as Fibers in 2017 Released in 2023 with the JDK 21
28. Coroutines with Virtual Threads (gross simplification)
29. Coroutines on Virtual Threads in Spring @RestController @RequestMapping("fulfilment") class FulfilmentController { @GetMapping fun getItems(): Flow<String> = (1..10).asFlow().map { val product = Product(name = "TV", isleType = Room) logger.info("Product: $product") product.name }.flowOn( Executors.newVirtualThreadPerTaskExecutor() .asCoroutineDispatcher() ) companion object { val logger: Logger = LoggerFactory.getLogger(FulfilmentController::class.java) } } An odd construction, but does it stand?🤔
30. Locust test - Max 100000 users ; Spawn Rate 1000 users/second
31. Virtual Threads Combined With Kotlin Coroutines • A different perspective on Threads with potential benefits • I never used it in production • Maybe it is not an improvement. • Again, testing is needed and lot of benchmarking is needed in order to make adequate decisions 2023 lot’s of blogs released: I did that too: https://github.com/jesperancinha /good-story
32. GitHub - Good Story - Virtual Thread Dispatcher in combination with Kotlin Coroutines
33. Conclusion • Virtual threads are very impactful because of their support by the Spring Framework. They are easy to use and build code with. Maintain that code is easy. It can be used in: ◦ Fire and forget scenarios ◦ Structured concurrency ◦ Both Java and Kotlin • Coroutines will still be used because of how well and elegantly they adapt to other paradigms: ◦ Flows ◦ Channels ◦ Event Streaming ◦ SSE ◦ Can be used in Both Java and Kotlin, but they are not prepared for Java and can get highly complicated that way • In any case, both can be used with similar or purely equal performance markers, leading to choosing them on the bases of code style, patterns and design. Virtual Theads and Coroutines have a great future still
34. What next? ➔ Java Virtual Threads have a definite future, but for Android at the moment, only up to JDK17 is supported and no support is available for Java Virtual Threads as a result. ➔ Kotlin Coroutines will stick around for Android for a long time. ➔ In Java Server Side the use of Kotlin Coroutines may go down for simple applications and only where certain specific operations are needed will Kotlin coroutines still likely be needed due to their simplicity. ➔ For all operations related to structured concurrency, streams and event handling, the usage of Kotlin Coroutines may actually increase as is the case with Mobile Applications in Android. ➔ Maybe coroutines can one day not be used anymore given the potential of Java Virtual Threads even in the case of Android applications, but that isn’t anything for the near future.
35. Questions? I am an inquisitive cat
36. Resources Online • https://kotlinlang.org/docs/coroutines-basics.html • https://openjdk.org/jeps/444 • https://openjdk.org/projects/jdk/21/ • https://www.jetbrains.com/help/idea/supported-java-versions.html • https://discuss.kotlinlang.org/t/coroutines-java-virtual-threads-and-scoped-values/28004/1 • https://developer.android.com/build/jdks Slides available: • https://www.scribd.com/presentation/768072685/Could-Virtual-Threads-Cast-Away-the-Usage- of-Kotlin-Coroutines • https://www.slideshare.net/slideshow/could-virtual-threads-cast-away-the-usage-of-kotlin-coro utines/271727680
37. Source code and documentation • https://github.com/jesperancinha/virtual-thread-coroutine-cooperation (Source code) • https://github.com/jesperancinha/good-story (CPU bound operations comparison) • https://github.com/jesperancinha/sea-shell-archiver (Coding with WebFlux - Mono/Flux/Zip) • https://github.com/jesperancinha/concert-demos-root (Back Pressure testing in Spring WebFlux and R2DBC) • https://github.com/jesperancinha/kitten-house-care-parent (Basics on resilient services)
38. About me • Homepage - https://joaofilipesabinoesperancinha.nl • LinkedIn - https://www.linkedin.com/in/joaoesperancinha/ • YouTube - JESPROTECH ▪ https://www.youtube.com/channel/UCzS_JK7QsZ7ZH-zTc5kBX_g ▪ https://www.youtube.com/@jesprotech • Bluesky - https://bsky.app/profile/jesperancinha.bsky.social • Mastodon - https://masto.ai/@jesperancinha • GitHub - https://github.com/jesperancinha • Hackernoon - https://hackernoon.com/u/jesperancinha • DevTO - https://dev.to/jofisaes • Medium - https://medium.com/@jofisaes