Tech-Stack Overview: Building Scala Microservices

The talk serves as an introduction and how-to of Scala microservices
and will cover the following topics:

- HTTP servers
- Logging
- CRUD with JDBC
- handling JSON
- Application configuration
- SBT plugins to build docker images
- Creating e2e tests
- the functional approach
- dependency injection

Transcript

1. Scala stack
   to build a microservice
   Alexey Novakov, INNOQ
   

   View Slide

2. About me
   4yrs in Scala, 10yrs in Java
   Senior Consultant at INNOQ
   Data Processing,
   Distributed Systems
   and Functional Programming fan
   

   View Slide

3. View Slide

4. Typelevel
   Others
   Lightbend
   Origin of the libraries
   

   View Slide

5. Cats, Shapeless
   

   View Slide

6. (★ GitHub Stars)
   ★
   

   View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. Let’s build a typical microservice
    

    View Slide

15. • HTTP-Server: Akka-HTTP
    • JSON Serialization: Spray-JSON or uPickle
    • Persistence: Slick
    • Logging: Scala-logging
    • Configuration: Typesafe Config, Pure Config, Refined
    • Testing: Scalatest, test-containers-scala
    • FP: Cats
    • Build Tool: SBT
    Tooling
    

    View Slide

16. REST API
    Service
    Architecture
    Domain
    Model Postgres
    SQL
    HTTP Client
    

    View Slide

17. Add: POST /api/v1/trips, body = JSON
    Update: PUT /api/v1/trips/, body = JSON
    Delete: DELETE /api/v1/trips/
    Select all: GET /api/v1/trips?sort=id&page=1&pageSize=100
    Select one: GET /api/v1/trips/
    Ride booking API
    

    View Slide

18. Models
    final case class Trip(
    id: Int, city: String, vehicle: Vehicle,
    price: Int, completed: Boolean, distance: Option[Int],
    endDate: Option[LocalDate]
    )
    final case class Trips(trips: Seq[Trip])
    object Vehicle extends Enumeration {
    type Vehicle = Value
    val Bike, Taxi, Car = Value
    }
    

    View Slide

19. HTTP Server
    • Akka-HTTP
    • Options:
    1. Core Server API (HttpRequest => HttpResponse)
    2. Routing DSL (Directives)
    • helps to make code DRY
    • but requires to learn a DSL (error-prone, time consuming)
    

    View Slide

20. val route: Route = pathPrefix("api" / "v1" / "trips") {
    concat(
    pathEndOrSingleSlash {
    get {
    parameters('sort.?, 'page.as[Int].?, 'pageSize.as[Int].?) {
    (sort, page, pageSize) =>
    log.debug("Select all sorted by '{}'", sort)
    val cars = service.selectAll(page, pageSize, sort)
    complete(cars)
    }
    }
    },
    val serverBinding = Http().bindAndHandle(, server.host, server.port)
    Routing DSL
    ….
    import akka.http.scaladsl.server.Directives._
    

    View Slide

21. JSON En/Decoding
    • Spray-JSON:
    • integrated via akka-http-spray-json library
    • uPickle:
    • String -> AST -> Case Class
    • integrated via de.heikoseeberger:akka-http-upickle library
    

    View Slide

22. JSON Codes
    Spray-JSON
    import spray.json.DefaultJsonProtocol._
    implicit val tripFormat: RootJsonFormat[Trip] = jsonFormat7(Trip)
    implicit val tripsFormat: RootJsonFormat[Trips] = jsonFormat1(Trips)
    Upickle
    import upickle.default._
    implicit val tripRW: ReadWriter[Trip] = macroRW
    implicit val tripsRW: ReadWriter[Trips] = macroRW
    

    View Slide

23. Usage of JSON Codecs
    post {
    entity(as[Trip]) { trip: Trip =>
    log.debug("Create new trip '{}'", trip)
    val inserted = service.insert(trip)
    complete {
    toCommandResponse(inserted, CommandResult)
    }
    }
    }
    

    View Slide

24. Data Layer
    trait Repository[F[_]] {
    def delete(id: Int) : F[Int]
    def update(id: Int, row: Trip): F[Int]
    def createSchema(): F[Unit]
    def insert(row: Trip): F[Int]
    def selectAll(page: Int, pageSize: Int, sort: String): F[Seq[Trip]]
    def select(id: Int): F[Option[Trip]]
    def sortingFields: Set[String]
    }
    

    View Slide

25. Typical Implementation in
    class Trips(tag: Tag) extends Table[Trip](tag, "trips") {
    def id = column[Int]("id", O.AutoInc, O.PrimaryKey)
    def city = column[String]("city")
    def price = column[Int]("price")
    def completed = column[Boolean]("completed")
    override def * =
    (id, city, vehicle, price, completed, distance, endDate) <>
    (Trip.tupled, Trip.unapply)
    }
    

    View Slide

26. def select(id: Int): Future[Option[Trip]] =
    db.run(trips.filter(_.id === id).take(1).result.headOption)
    val trips = TableQuery[Trips]
    class TripRepository(db: Database) extends Repository[Future] {
    }
    ….
    

    View Slide

27. Generic Types
    HTTP Routes:
    RequestContext 㱺 Future[RouteResult]
    TripService[F[_]]
    Repository[F[_]] TripRepo[Future]
    TripService[Future]
    Slick (can be replaced by Doobie, etc.)
    Akka HTTP
    (can be replaced by
    Play, http4s, etc.)
    Implementation
    Application,
    (framework,
    library free)
    

    View Slide

28. FP
    • We want TripService be generic in its return type: F[_]
    def selectAll(…): F[Trips]
    • F[_] - can be later set to Future, Task, IO, Id, etc.
    • Cats Type Classes to the rescue!
    

    View Slide

29. FP: Cats Usage
    def selectAll(page: Option[Int], pageSize: Option[Int], sort: Option[String]):
    F[Trips] = {
    …..
    repo
    .selectAll(pageN, size, sort) // <— F[Seq[Trip]]
    .map(Trips)
    }
    import cats.Functor
    import cats.syntax.functor._
    class TripService[F[_]](repo: Repository[F])(implicit F: Functor[F]) {
    

    View Slide

30. Scala Logging
    • It wraps slf4j and requires logging backend library like logback
    • Usage:
    class MyClass extends LazyLogging {
    logger.debug("This is very convenient ;-)”)
    …
    }
    “com.typesafe.scala-logging:scala-logging"
    if (logger.isDebugEnabled) logger.debug(s"Some $expensive message!")
    • check-enabled-idiom is applied automatically by Scala macros
    

    View Slide

31. Configuration: HOCON
    foo {
    bar = 10
    baz = 12
    }
    foo.bar =10
    foo.baz =12
    Or
    Example:
    Human-Optimized Config Object Notation
    Typesafe Config library
    

    View Slide

32. server {
    host = localhost
    port = 8080
    }
    storage {
    host = localhost
    port = 5432
    dbName = trips
    url = "jdbc:postgresql://"${storage.host}":"${storage.port}"/"${storage.dbName}
    driver = "org.postgresql.Driver"
    user = “trips"
    password = "trips"
    }
    application.conf
    

    View Slide

33. import pureconfig.loadConfig
    import pureconfig.generic.auto._
    val path = … // path to application.conf
    def load: Either[ConfigReaderFailures, Server] = {
    loadConfig[Server](Paths.get(path), “server”)
    }
    “com.github.pureconfig:pureconfig"
    final case class Server(
    host: String = "localhost",
    port: Int = 8080
    )
    

    View Slide

34. “eu.timepit:refined-pureconfig"
    import eu.timepit.refined.types.net.UserPortNumber
    import eu.timepit.refined.types.string.NonEmptyString
    import eu.timepit.refined.pureconfig._
    [info] Compiling 2 Scala sources to /Users/aa/dev/git/akka-crud-service/target/scala-2.12/classes ...
    [error] /Users/aa/dev/git/akka-crud-service/src/main/scala/org/alexeyn/configs.scala:39:84:
    Left predicate of (!(808 < 1024) && !(808 > 49151)) failed: Predicate (808 < 1024) did not fail.
    [error] final case class Server(host: NonEmptyString = "localhost", port: UserPortNumber = 808)
    final case class Server(
    host: NonEmptyString = "localhost",
    port: UserPortNumber = 808
    )
    

    View Slide

35. Custom Types
    type ConnectionTimeout =
    Int Refined Interval.OpenClosed[W.`0`.T, W.`100000`.T]
    type MaxPoolSize =
    Int Refined Interval.OpenClosed[W.`0`.T, W.`100`.T]
    type JdbcUrl =
    String Refined MatchesRegex[W.`"""jdbc:\\w+://\\w+:[0-9]{4,5}/\\w+"""`.T]
    final case class JdbcConfig(
    url: JdbcUrl,
    connectionTimeout: ConnectionTimeout,
    maximumPoolSize: MaxPoolSize
    )
    

    View Slide

36. Testing
    • Scalatest + akka-testkit:
    • path: HTTP Routes Application
    • test-containers-scala + …
    • to test integration
    • path: HTTP Routes Application Postgres Container
    • Note
    mock libraries like Mockito are not really needed
    in Scala projects. Thanks to FP
    

    View Slide

37. private val stubRepo = createStubRepo
    private val service = new TripService[Future](stubRepo)
    private val routes = CommandRoutes.routes(service)
    "CommandRoutes" should {
    "insert new trip and return its id" in {
    val request: HttpRequest = RequestsSupport.insertRequest(berlin)
    request ~> routes ~> check {
    val count = entityAs[CommandResult].count
    count should ===(1)
    }
    }
    Testing Application
    

    View Slide

38. class E2ETest
    extends WordSpec
    …
    with ForAllTestContainer
    override val container = PostgreSQLContainer("postgres:10.4")
    lazy val cfg: Config = ConfigFactory.load(
    ConfigFactory
    .parseMap(
    Map(
    "port" -> container.mappedPort(5432),
    "url" -> container.jdbcUrl,
    "user" -> container.username,
    "password" -> container.password
    ).asJava).atKey("storage")
    )
    lazy val mod = new Module(cfg, createSchema = false)
    

    View Slide

39. Container is started by Scalatest
    

    View Slide

40. SBT Plugins
    plugins.sbt:
    addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % “x.y.z”)
    build.sbt:
    enablePlugins(JavaAppPackaging, AshScriptPlugin)
    FROM openjdk:8-jre-alpine
    WORKDIR /opt/docker
    ADD --chown=daemon:daemon opt /opt
    USER daemon
    ENTRYPOINT ["/opt/docker/bin/akka-crud-service"]
    CMD []
    Generates busybox compatible script
    $ sbt docker:publishLocal
    

    View Slide

41. Useful SBT Plugins
    1. sbt-release
    addSbtPlugin(“com.github.gseitz" % "sbt-release" % “x.y.z")
    Easy to bump version and commit to Git.
    1.2.1-SNAPSHOT -> 1.2.1 -> 1.2.2-SNAPSHOT
    2. sbt-scalafmt
    addSbtPlugin("com.geirsson" % "sbt-scalafmt" % “x.y.z")
    Can check & fail the build, format sources
    

    View Slide

42. Dependency Injection
    val db = Database.forConfig("storage", cfg)
    // class TripRepository(db: Database)
    val repo = wire[TripRepository]
    // class TripService[F[_]](repo: Repository[F])
    val service = wire[TripService[Future]]
    // class QueryRoutes(service: TripService[Future])
    val routes = concat(wire[QueryRoutes].routes, wire[CommandRoutes].routes)
    import com.softwaremill.macwire._
    

    View Slide

43. View Slide

44. More Info
    1. Scalar 2018 whiteboard voting results!
    https://blog.softwaremill.com/scalar-2018-whiteboard-voting-results-c6f50f8fb16d
    2. Scala Developer Survey Results 2018 (link)
    3. Scaladex: https://index.scala-lang.org
    4. Source Code: https://github.com/novakov-alexey/akka-crud-service
    5. Some Scala posts: https://medium.com/se-notes-by-alexey-novakov
    Twitter: @alexey_novakov
    

    View Slide