A minimalist approach to web application development using independent libraries in Scala

The power of vanilla Scala and pushing beyond framework boundaries.

The slides are from the talk ("Breaking framework chains with vanilla Scala") at Functional World #9 on June 28th 2023.

The main topic was exploring why "Less is More", and we delved into Scala 2 and 3 libraries, also discovering unpublished features. We reviewed: Scala Toolkit, Jsoniter-Scala, MacWire, Iron, Sttp, Tapir, Chimney, Ducktape, Quicklens, Dijon, and Diffx. Our discussion also touched on the importance of "thin-wrappers" with the aws-lambda-scala-bridge as an example. These discussions centred on the Serverless Lambda function and managed runtime.

Transcript

1. Breaking framework chains with vanilla Scala A minimalist approach to web application development
2. Agenda • Setting the presentation scope • Sharing motivation linked to the lecture topic • Delving into fresh perspectives on similar approaches • Reviewing noteworthy libraries for common tasks • Showcasing sample code, tailored for this presentation 🚀
3. About Marcin Szałomski (Hapag-Lloyd AG) A software engineering addict with decades of industry experience. Java die-hard turned Scala enthusiast, diving deep into functional programming. Active in open-source, dishing out insights on my blog https://blog.lambdaspot.dev/. Twitter: baldram@twitter Mastodon: @baldram@functional.cafe
4. Hapag-Lloyd
5. Hapag-Lloyd
6. Presentation scope Scala has many flavors: JVM, JS, Native, and this presentation focuses on JVM. Target audience, individuals who: 1. Appreciate static typing, compile-time correctness, and Scala’s additional benefits in this area. 2. Are enthusiasts of functional programming, and prefer higher-level languages like Java, Scala, or Kotlin for enterprise applications. 3. Love JVM, and benefits from accessing its rich ecosystem. 4. Value JVM for its speed compared to interpreted languages. 5. Are willing to accept certain compromises, as we can't ignore certain weaknesses like its long-term pain points: slow startup time, slow time to peak performance, and large footprint. 6. Are aware of various solutions to mitigate these issues, e.g., Project Leyden, CRaC, GraalVM Native Image.
7. The power of asceticism: why less is more Consider these compelling reasons for adopting a minimalist approach: 1. If we can avoid pain points, even when tools for mitigation exist, why trigger the situation? 2. Using such tools invariably leads to trade-offs - from increased build time to dealing with complex configuration options. For example, less dependencies in a Native Image mean fewer potential issues. 3. Fewer transitive dependencies result in decreased maintenance load. This covers updates, dependency conflicts, security risks, bug diagnosis, licensing issues, and performance considerations. 4. Before employing a large framework or extensive library, think about vendor lock-in risks. You may face future licensing issues or become stuck with an outdated stack. 5. Even when using independent libraries, lean towards thin-wrappers or facades Asceticism isn't about giving up tools, it's about smart choices that simplify and enhance performance.
8. To framework or not to framework? Defining "framework" - we're discussing the tool used for building applications. But what about the rest? Frameworks are everywhere, even what our cloud vendors provide can be considered one. Does vendor lock-in pose a problem? Undeniably so. So, let's discuss some options. • Leverage a standard specification such as Cloud Events, or map requests to a custom model for independence. Establish a thin-wrapper on the vendor's SDK for versatility. • Consider an Infrastructure as Code (IaC) solution like Pulumi to manage cloud-agnostic infrastructure code. Remember, dependency on Pulumi is still a trade-off, unless we're up for reinventing the wheel. • Does a serverless cloud environment still require an application framework? The answer: it depends! Navigating the world of frameworks isn't just about choices, it's about making strategic decisions for flexibility and independence.
9. Framework? Common tasks Serverless cloud library MVC architecture – structured way to orginize code wich facititates paralell development, separation of concerns Support for RESTful services; maping HTTP request to controller methods; handling request/response, etc. Data validation, error handling Database and other storage accessing Scalability and performance, consistency Support for asynchronous processing - background jobs Support for unit and integration testing Security Community support Comparing common tasks handled by a framework versus serverless cloud (managed runtime) plus libraries.
10. Sufficient for enterprise? Moreover, we have ability for creating complex solutions using Step Functions. Workflows, with asynchronous processing. Focusing on a business logic only and not worrying about concurrency. Each single serverless function is a simple program. https://aws.amazon.com/blogs/machine-learning/kinect-energy-uses-amazon-sagemaker-to-forecast-energy-prices-with-machine-learning/ https://www.infoq.com/articles/workflows-aws-step-functions/
11. Is standard library the way to go? A curated list of libraries (Scala 2 / 3, JVM , JS , Native), chosen based on: • Usability and Maintenance: Their user-friendly APIs, the responsiveness and availability of their maintainers. • Documentation and Popularity: Well-documented and widely used. • Dependencies and Platform Support: With minimal dependencies. Read more: https://docs.scala-lang.org/toolkit/introduction.html Scala Toolkit. A ±week ago was release of Scala Toolkit: • a suite of libraries for common programming tasks. • supports file handling, JSON parsing, HTTP requests, and unit testing. • ideal for short programs, web apps, and command-line tools.
12. Custom toolkit? There are other toolkits, e.g., Typelevel Toolkit, a purely functional stack based on Cats. Guessing there will be one of libraries under the ZIO umbrella – for type-safe, async and concurrent programming. Let’s set a collection of libraries for common tasks: • Jsoniter-Scala – parsing JSON and serializing objects to JSON • MacWire – dependency injection • Iron – validation, refined types • Sttp – sending HTTP requests • Chimney or Ducktype for automated transformation of similar nested structures • Quicklens or Dijon for modifying nested structures, and Diffx for comparing them • Scalatest and Wiremock, EmbeddedPostgres, DynamoDB Local, Testcontainers, etc. for testing,
13. Example project: triggering the controller Using AWS SAM, we define the handler in the CloudFormation Template (template.yml). Opting for managed runtime, each controller becomes a distinct function. Implementing the AWS Java interface requires a thin Scala-friendly wrapper, enabling constructor arguments for supporting DI and sidestepping the no-arg constructor rule. We use the github.com/lambdaspot/aws-lambda-scala-bridge (a strawman implementation for this presentation, might evolve into a versatile library in the future). In case of RESTful API it’s as follows:
14. Jsoniter-Scala • High-performance, zero-dependency JSON parser and serializer for Scala 2 and Scala 3 (JS, JVM, Native) • Designed for speed, efficiency, with a focus on security. • Supports automatic generation of safe JSON codecs. • Provides customizable encoding/decoding options. • Adheres to RFC-8259, ensuring correctness. • Outperforms all JVM-based JSON parsers, and in JIT mode comparable with top C++ ones Read more: https://github.com/plokhotnyuk/jsoniter-scala https://blog.lambdaspot.dev/the-fastest-and-safest-json-parser-and-serializer-for-scala (practical suggestions and benchmarks)
15. MacWire • A Dependency Injection (DI) library for Scala 2 and Scala 3. • Provides a boilerplate-free manner to wire dependencies. • Resolves all dependencies at compile-time, ensuring type safety. • Simplifies DI, reducing the need for manual wiring. • Promotes clean, understandable, and reusable code. Read more: https://github.com/softwaremill/macwire http://di-in-scala.github.io/ - comprehensive guide on DI with MacWire
16. Iron • A lightweight library for refined types in Scala 3 (JS, JVM, Native). • Enables attaching constraints/assertions to types • Extendable: easy creating own constraints or integrations using classic typeclasses. • Supports both compile-time and runtime evaluation of constraints. • Offers integrations with popular libraries and ecosystems. Read more: https://github.com/Iltotore/iron
17. Iron Or use with JSON parser/serializer: Compose and customize messages: & Jsoniter-Scala
18. Jsoniter-Scala: example problem – vary structure? Safety, fail-fast, correctness, but isn’t it all too strict? Can we have vary structure? What if we deal with: • Two different versions of payload, say v1 and v2, and each with different data structure. • Inconvenient service responding different structure on partial success • Or we want to implement strategy, a “dynamic feature-flag” Use a “discriminator” or implement a custom JsonValueCodec and define decodeValue and encodeValue. Now, simple pattern-matching and we’re good to go with a strategy for further handling! It’s simply enum in Scala 3
19. Sttp • A feature-rich library for making HTTP requests to web servers • Library-agnostic with underlying HTTP client “plugins” (backends) • For Scala 2 and Scala 3 (JS, JVM, Native) • Supports sync, async and WebSockets • Extendable: streaming, logging, telemetry, serialization, … Read more: https://docs.scala-lang.org/toolkit/http-client-intro.html https://sttp.softwaremill.com/en/stable/ Looking for type safety? Try Tapir! https://tapir.softwaremill.com/ • Declarative, type-safe web endpoints library (with server-agnostic approach) • Discoverable API through standard auto-complete • Straightforward generating server, client & docs (OpenAPI / Swagger integration) (or Tapir)
20. Chimney / Ducktape • Supports transformation of data between similar structures • Automatically maps fields that have the same name and type in both case classes • Allows specifying custom transformers for other cases • Chimney supports Scala 2 (migration to 3 in progress) • Ducktape supports Scala 3 (JS, JVM, Native) Read more: https://scalalandio.github.io/chimney/ 🔧 https://github.com/arainko/ducktape
21. Dijon • A dynamically typed JSON library for Scala 2 😞 • Allows creation and manipulation of JSON objects and arrays intuitively • Supports deep access and modifying nested JSON structures. • Enables deep copying and merging of JSON objects • Offers type safety with union types • Supports RFC8259 via Jsoniter-Scala-core • Useful when working with test fixtures Read more: https://github.com/jvican/dijon
22. Quicklens • A lens/optics library for Scala 2 & 3 (JS, JVM, Native) • Simplifies modifying nested data structures. • The “path” to the modified field is a reusable plain value. • Supports field modifications using a function, conditional modifications, modifications of collection elements, and composition of lenses. Let’s double the score of all players in all teams. Read more: https://github.com/softwaremill/quicklens
23. Diffx • Displaying differences between complex structures in pretty way. • JVM, JS, Scala 2 and Scala 3 and various test frameworks integration Read more: https://diffx-scala.readthedocs.io/en/latest 😎 😎 🚀
24. Summary • I've presented a simple approach, using a managed runtime for serverless lambda, where many aspects of application creation disappear, leaving us only to focus on business logic. • We always have the option to use a custom runtime and can build our product as a full-fledged application using heavier libraries or frameworks, handling some technical aspects independently. • We simply choose the tool according to our needs and requirements. • Each serverless lambda function can be independent, and we can mix solutions (custom vs managed runtime) within one project, using what is the lightest and simplest depending on the requirements. • Importantly, in each case, we use Scala, and by writing clean code (pure and platform-agnostic), we can reuse fragments or migrate the entire implementation to another stack, for example, from StdLib to ZIO or JVM to Native.
25. Thank you! 🙏 Questions? References (see also links in earlier slides): • Example hello app from this talk: https://github.com/lambdaspot/vanilla-scala-talk • Jsoniter-Scala intro: https://blog.lambdaspot.dev/the-fastest-and-safest-json-parser-and-serializer-for-scala • Security: JSON Denial of Service: https://www.youtube.com/watch?v=3Cz6D8JLSSA • Dependency Injection overview: https://www.youtube.com/watch?v=Pf8d1VzckRA • Cloud observability: https://aws.amazon.com/blogs/mt/how-hapag-lloyd-established-observability-for- serverless-multi-account-workloads/ • How many teaspoons of sugar (ZIO) do you sweeten with? ;) https://youtu.be/nZAxtQAJaU0