Tech-Stack Overview: Building Scala Microservices

The talk serves as an introduction and how-to of Scala microservices
and will cover the following topics:

- HTTP servers
- Logging
- CRUD with JDBC
- handling JSON
- Application configuration
- SBT plugins to build docker images
- Creating e2e tests
- the functional approach
- dependency injection

Transcript

1. Scala stack to build a microservice Alexey Novakov, INNOQ
2. About me 4yrs in Scala, 10yrs in Java Senior Consultant at INNOQ Data Processing, Distributed Systems and Functional Programming fan
4. Typelevel Others Lightbend Origin of the libraries
5. Cats, Shapeless
6. (★ GitHub Stars) ★
14. Let’s build a typical microservice
15. • HTTP-Server: Akka-HTTP • JSON Serialization: Spray-JSON or uPickle • Persistence: Slick • Logging: Scala-logging • Configuration: Typesafe Config, Pure Config, Refined • Testing: Scalatest, test-containers-scala • FP: Cats • Build Tool: SBT Tooling
16. REST API Service Architecture Domain Model Postgres SQL HTTP Client
17. Add: POST /api/v1/trips, body = JSON Update: PUT /api/v1/trips/<id>, body = JSON Delete: DELETE /api/v1/trips/<id> Select all: GET /api/v1/trips?sort=id&page=1&pageSize=100 Select one: GET /api/v1/trips/<id> Ride booking API
18. Models final case class Trip( id: Int, city: String, vehicle: Vehicle, price: Int, completed: Boolean, distance: Option[Int], endDate: Option[LocalDate] ) final case class Trips(trips: Seq[Trip]) object Vehicle extends Enumeration { type Vehicle = Value val Bike, Taxi, Car = Value }
19. HTTP Server • Akka-HTTP • Options: 1. Core Server API (HttpRequest => HttpResponse) 2. Routing DSL (Directives) • helps to make code DRY • but requires to learn a DSL (error-prone, time consuming)
20. val route: Route = pathPrefix("api" / "v1" / "trips") { concat( pathEndOrSingleSlash { get { parameters('sort.?, 'page.as[Int].?, 'pageSize.as[Int].?) { (sort, page, pageSize) => log.debug("Select all sorted by '{}'", sort) val cars = service.selectAll(page, pageSize, sort) complete(cars) } } }, val serverBinding = Http().bindAndHandle(<routes…>, server.host, server.port) Routing DSL …. import akka.http.scaladsl.server.Directives._
21. JSON En/Decoding • Spray-JSON: • integrated via akka-http-spray-json library • uPickle: • String -> AST -> Case Class • integrated via de.heikoseeberger:akka-http-upickle library
22. JSON Codes Spray-JSON import spray.json.DefaultJsonProtocol._ implicit val tripFormat: RootJsonFormat[Trip] = jsonFormat7(Trip) implicit val tripsFormat: RootJsonFormat[Trips] = jsonFormat1(Trips) Upickle import upickle.default._ implicit val tripRW: ReadWriter[Trip] = macroRW implicit val tripsRW: ReadWriter[Trips] = macroRW
23. Usage of JSON Codecs post { entity(as[Trip]) { trip: Trip => log.debug("Create new trip '{}'", trip) val inserted = service.insert(trip) complete { toCommandResponse(inserted, CommandResult) } } }
24. Data Layer trait Repository[F[_]] { def delete(id: Int) : F[Int] def update(id: Int, row: Trip): F[Int] def createSchema(): F[Unit] def insert(row: Trip): F[Int] def selectAll(page: Int, pageSize: Int, sort: String): F[Seq[Trip]] def select(id: Int): F[Option[Trip]] def sortingFields: Set[String] }
25. Typical Implementation in class Trips(tag: Tag) extends Table[Trip](tag, "trips") { def id = column[Int]("id", O.AutoInc, O.PrimaryKey) def city = column[String]("city") def price = column[Int]("price") def completed = column[Boolean]("completed") override def * = (id, city, vehicle, price, completed, distance, endDate) <> (Trip.tupled, Trip.unapply) }
26. def select(id: Int): Future[Option[Trip]] = db.run(trips.filter(_.id === id).take(1).result.headOption) val trips = TableQuery[Trips] class TripRepository(db: Database) extends Repository[Future] { } ….
27. Generic Types HTTP Routes: RequestContext 㱺 Future[RouteResult] TripService[F[_]] Repository[F[_]] TripRepo[Future] TripService[Future] Slick (can be replaced by Doobie, etc.) Akka HTTP (can be replaced by Play, http4s, etc.) Implementation Application, (framework, library free)
28. FP • We want TripService be generic in its return type: F[_] def selectAll(…): F[Trips] • F[_] - can be later set to Future, Task, IO, Id, etc. • Cats Type Classes to the rescue!
29. FP: Cats Usage def selectAll(page: Option[Int], pageSize: Option[Int], sort: Option[String]): F[Trips] = { ….. repo .selectAll(pageN, size, sort) // <— F[Seq[Trip]] .map(Trips) } import cats.Functor import cats.syntax.functor._ class TripService[F[_]](repo: Repository[F])(implicit F: Functor[F]) {
30. Scala Logging • It wraps slf4j and requires logging backend library like logback • Usage: class MyClass extends LazyLogging { logger.debug("This is very convenient ;-)”) … } “com.typesafe.scala-logging:scala-logging" if (logger.isDebugEnabled) logger.debug(s"Some $expensive message!") • check-enabled-idiom is applied automatically by Scala macros
31. Configuration: HOCON foo { bar = 10 baz = 12 } foo.bar =10 foo.baz =12 Or Example: Human-Optimized Config Object Notation Typesafe Config library
32. server { host = localhost port = 8080 } storage { host = localhost port = 5432 dbName = trips url = "jdbc:postgresql://"${storage.host}":"${storage.port}"/"${storage.dbName} driver = "org.postgresql.Driver" user = “trips" password = "trips" } application.conf
33. import pureconfig.loadConfig import pureconfig.generic.auto._ val path = … // path to application.conf def load: Either[ConfigReaderFailures, Server] = { loadConfig[Server](Paths.get(path), “server”) } “com.github.pureconfig:pureconfig" final case class Server( host: String = "localhost", port: Int = 8080 )
34. “eu.timepit:refined-pureconfig" import eu.timepit.refined.types.net.UserPortNumber import eu.timepit.refined.types.string.NonEmptyString import eu.timepit.refined.pureconfig._ [info] Compiling 2 Scala sources to /Users/aa/dev/git/akka-crud-service/target/scala-2.12/classes ... [error] /Users/aa/dev/git/akka-crud-service/src/main/scala/org/alexeyn/configs.scala:39:84: Left predicate of (!(808 < 1024) && !(808 > 49151)) failed: Predicate (808 < 1024) did not fail. [error] final case class Server(host: NonEmptyString = "localhost", port: UserPortNumber = 808) final case class Server( host: NonEmptyString = "localhost", port: UserPortNumber = 808 )
35. Custom Types type ConnectionTimeout = Int Refined Interval.OpenClosed[W.`0`.T, W.`100000`.T] type MaxPoolSize = Int Refined Interval.OpenClosed[W.`0`.T, W.`100`.T] type JdbcUrl = String Refined MatchesRegex[W.`"""jdbc:\\w+://\\w+:[0-9]{4,5}/\\w+"""`.T] final case class JdbcConfig( url: JdbcUrl, connectionTimeout: ConnectionTimeout, maximumPoolSize: MaxPoolSize )
36. Testing • Scalatest + akka-testkit: • path: HTTP Routes Application • test-containers-scala + … • to test integration • path: HTTP Routes Application Postgres Container • Note mock libraries like Mockito are not really needed in Scala projects. Thanks to FP
37. private val stubRepo = createStubRepo private val service = new TripService[Future](stubRepo) private val routes = CommandRoutes.routes(service) "CommandRoutes" should { "insert new trip and return its id" in { val request: HttpRequest = RequestsSupport.insertRequest(berlin) request ~> routes ~> check { val count = entityAs[CommandResult].count count should ===(1) } } Testing Application
38. class E2ETest extends WordSpec … with ForAllTestContainer override val container = PostgreSQLContainer("postgres:10.4") lazy val cfg: Config = ConfigFactory.load( ConfigFactory .parseMap( Map( "port" -> container.mappedPort(5432), "url" -> container.jdbcUrl, "user" -> container.username, "password" -> container.password ).asJava).atKey("storage") ) lazy val mod = new Module(cfg, createSchema = false)
39. Container is started by Scalatest
40. SBT Plugins plugins.sbt: addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % “x.y.z”) build.sbt: enablePlugins(JavaAppPackaging, AshScriptPlugin) FROM openjdk:8-jre-alpine WORKDIR /opt/docker ADD --chown=daemon:daemon opt /opt USER daemon ENTRYPOINT ["/opt/docker/bin/akka-crud-service"] CMD [] Generates busybox compatible script $ sbt docker:publishLocal
41. Useful SBT Plugins 1. sbt-release addSbtPlugin(“com.github.gseitz" % "sbt-release" % “x.y.z") Easy to bump version and commit to Git. 1.2.1-SNAPSHOT -> 1.2.1 -> 1.2.2-SNAPSHOT 2. sbt-scalafmt addSbtPlugin("com.geirsson" % "sbt-scalafmt" % “x.y.z") Can check & fail the build, format sources
42. Dependency Injection val db = Database.forConfig("storage", cfg) // class TripRepository(db: Database) val repo = wire[TripRepository] // class TripService[F[_]](repo: Repository[F]) val service = wire[TripService[Future]] // class QueryRoutes(service: TripService[Future]) val routes = concat(wire[QueryRoutes].routes, wire[CommandRoutes].routes) import com.softwaremill.macwire._
44. More Info 1. Scalar 2018 whiteboard voting results! https://blog.softwaremill.com/scalar-2018-whiteboard-voting-results-c6f50f8fb16d 2. Scala Developer Survey Results 2018 (link) 3. Scaladex: https://index.scala-lang.org 4. Source Code: https://github.com/novakov-alexey/akka-crud-service 5. Some Scala posts: https://medium.com/se-notes-by-alexey-novakov Twitter: @alexey_novakov