Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tech-Stack Overview: Building Scala Microservices

Tech-Stack Overview: Building Scala Microservices

The talk serves as an introduction and how-to of Scala microservices
and will cover the following topics:

- HTTP servers
- Logging
- CRUD with JDBC
- handling JSON
- Application configuration
- SBT plugins to build docker images
- Creating e2e tests
- the functional approach
- dependency injection

7a04b88e1469561db6da3818348d4b8f?s=128

Alexey Novakov

January 24, 2019
Tweet

Transcript

  1. Scala stack to build a microservice Alexey Novakov, INNOQ

  2. About me 4yrs in Scala, 10yrs in Java Senior Consultant

    at INNOQ Data Processing, Distributed Systems and Functional Programming fan
  3. None
  4. Typelevel Others Lightbend Origin of the libraries

  5. Cats, Shapeless

  6. (★ GitHub Stars) ★

  7. None
  8. None
  9. None
  10. None
  11. None
  12. None
  13. None
  14. Let’s build a typical microservice

  15. • HTTP-Server: Akka-HTTP • JSON Serialization: Spray-JSON or uPickle •

    Persistence: Slick • Logging: Scala-logging • Configuration: Typesafe Config, Pure Config, Refined • Testing: Scalatest, test-containers-scala • FP: Cats • Build Tool: SBT Tooling
  16. REST API Service Architecture Domain Model Postgres SQL HTTP Client

  17. Add: POST /api/v1/trips, body = JSON Update: PUT /api/v1/trips/<id>, body

    = JSON Delete: DELETE /api/v1/trips/<id> Select all: GET /api/v1/trips?sort=id&page=1&pageSize=100 Select one: GET /api/v1/trips/<id> Ride booking API
  18. Models final case class Trip( id: Int, city: String, vehicle:

    Vehicle, price: Int, completed: Boolean, distance: Option[Int], endDate: Option[LocalDate] ) final case class Trips(trips: Seq[Trip]) object Vehicle extends Enumeration { type Vehicle = Value val Bike, Taxi, Car = Value }
  19. HTTP Server • Akka-HTTP • Options: 1. Core Server API

    (HttpRequest => HttpResponse) 2. Routing DSL (Directives) • helps to make code DRY • but requires to learn a DSL (error-prone, time consuming)
  20. val route: Route = pathPrefix("api" / "v1" / "trips") {

    concat( pathEndOrSingleSlash { get { parameters('sort.?, 'page.as[Int].?, 'pageSize.as[Int].?) { (sort, page, pageSize) => log.debug("Select all sorted by '{}'", sort) val cars = service.selectAll(page, pageSize, sort) complete(cars) } } }, val serverBinding = Http().bindAndHandle(<routes…>, server.host, server.port) Routing DSL …. import akka.http.scaladsl.server.Directives._
  21. JSON En/Decoding • Spray-JSON: • integrated via akka-http-spray-json library •

    uPickle: • String -> AST -> Case Class • integrated via de.heikoseeberger:akka-http-upickle library
  22. JSON Codes Spray-JSON import spray.json.DefaultJsonProtocol._ implicit val tripFormat: RootJsonFormat[Trip] =

    jsonFormat7(Trip) implicit val tripsFormat: RootJsonFormat[Trips] = jsonFormat1(Trips) Upickle import upickle.default._ implicit val tripRW: ReadWriter[Trip] = macroRW implicit val tripsRW: ReadWriter[Trips] = macroRW
  23. Usage of JSON Codecs post { entity(as[Trip]) { trip: Trip

    => log.debug("Create new trip '{}'", trip) val inserted = service.insert(trip) complete { toCommandResponse(inserted, CommandResult) } } }
  24. Data Layer trait Repository[F[_]] { def delete(id: Int) : F[Int]

    def update(id: Int, row: Trip): F[Int] def createSchema(): F[Unit] def insert(row: Trip): F[Int] def selectAll(page: Int, pageSize: Int, sort: String): F[Seq[Trip]] def select(id: Int): F[Option[Trip]] def sortingFields: Set[String] }
  25. Typical Implementation in class Trips(tag: Tag) extends Table[Trip](tag, "trips") {

    def id = column[Int]("id", O.AutoInc, O.PrimaryKey) def city = column[String]("city") def price = column[Int]("price") def completed = column[Boolean]("completed") override def * = (id, city, vehicle, price, completed, distance, endDate) <> (Trip.tupled, Trip.unapply) }
  26. def select(id: Int): Future[Option[Trip]] = db.run(trips.filter(_.id === id).take(1).result.headOption) val trips

    = TableQuery[Trips] class TripRepository(db: Database) extends Repository[Future] { } ….
  27. Generic Types HTTP Routes: RequestContext 㱺 Future[RouteResult] TripService[F[_]] Repository[F[_]] TripRepo[Future]

    TripService[Future] Slick (can be replaced by Doobie, etc.) Akka HTTP (can be replaced by Play, http4s, etc.) Implementation Application, (framework, library free)
  28. FP • We want TripService be generic in its return

    type: F[_] def selectAll(…): F[Trips] • F[_] - can be later set to Future, Task, IO, Id, etc. • Cats Type Classes to the rescue!
  29. FP: Cats Usage def selectAll(page: Option[Int], pageSize: Option[Int], sort: Option[String]):

    F[Trips] = { ….. repo .selectAll(pageN, size, sort) // <— F[Seq[Trip]] .map(Trips) } import cats.Functor import cats.syntax.functor._ class TripService[F[_]](repo: Repository[F])(implicit F: Functor[F]) {
  30. Scala Logging • It wraps slf4j and requires logging backend

    library like logback • Usage: class MyClass extends LazyLogging { logger.debug("This is very convenient ;-)”) … } “com.typesafe.scala-logging:scala-logging" if (logger.isDebugEnabled) logger.debug(s"Some $expensive message!") • check-enabled-idiom is applied automatically by Scala macros
  31. Configuration: HOCON foo { bar = 10 baz = 12

    } foo.bar =10 foo.baz =12 Or Example: Human-Optimized Config Object Notation Typesafe Config library
  32. server { host = localhost port = 8080 } storage

    { host = localhost port = 5432 dbName = trips url = "jdbc:postgresql://"${storage.host}":"${storage.port}"/"${storage.dbName} driver = "org.postgresql.Driver" user = “trips" password = "trips" } application.conf
  33. import pureconfig.loadConfig import pureconfig.generic.auto._ val path = … // path

    to application.conf def load: Either[ConfigReaderFailures, Server] = { loadConfig[Server](Paths.get(path), “server”) } “com.github.pureconfig:pureconfig" final case class Server( host: String = "localhost", port: Int = 8080 )
  34. “eu.timepit:refined-pureconfig" import eu.timepit.refined.types.net.UserPortNumber import eu.timepit.refined.types.string.NonEmptyString import eu.timepit.refined.pureconfig._ [info] Compiling 2

    Scala sources to /Users/aa/dev/git/akka-crud-service/target/scala-2.12/classes ... [error] /Users/aa/dev/git/akka-crud-service/src/main/scala/org/alexeyn/configs.scala:39:84: Left predicate of (!(808 < 1024) && !(808 > 49151)) failed: Predicate (808 < 1024) did not fail. [error] final case class Server(host: NonEmptyString = "localhost", port: UserPortNumber = 808) final case class Server( host: NonEmptyString = "localhost", port: UserPortNumber = 808 )
  35. Custom Types type ConnectionTimeout = Int Refined Interval.OpenClosed[W.`0`.T, W.`100000`.T] type

    MaxPoolSize = Int Refined Interval.OpenClosed[W.`0`.T, W.`100`.T] type JdbcUrl = String Refined MatchesRegex[W.`"""jdbc:\\w+://\\w+:[0-9]{4,5}/\\w+"""`.T] final case class JdbcConfig( url: JdbcUrl, connectionTimeout: ConnectionTimeout, maximumPoolSize: MaxPoolSize )
  36. Testing • Scalatest + akka-testkit: • path: HTTP Routes Application

    • test-containers-scala + … • to test integration • path: HTTP Routes Application Postgres Container • Note mock libraries like Mockito are not really needed in Scala projects. Thanks to FP
  37. private val stubRepo = createStubRepo private val service = new

    TripService[Future](stubRepo) private val routes = CommandRoutes.routes(service) "CommandRoutes" should { "insert new trip and return its id" in { val request: HttpRequest = RequestsSupport.insertRequest(berlin) request ~> routes ~> check { val count = entityAs[CommandResult].count count should ===(1) } } Testing Application
  38. class E2ETest extends WordSpec … with ForAllTestContainer override val container

    = PostgreSQLContainer("postgres:10.4") lazy val cfg: Config = ConfigFactory.load( ConfigFactory .parseMap( Map( "port" -> container.mappedPort(5432), "url" -> container.jdbcUrl, "user" -> container.username, "password" -> container.password ).asJava).atKey("storage") ) lazy val mod = new Module(cfg, createSchema = false)
  39. Container is started by Scalatest

  40. SBT Plugins plugins.sbt: addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % “x.y.z”) build.sbt: enablePlugins(JavaAppPackaging,

    AshScriptPlugin) FROM openjdk:8-jre-alpine WORKDIR /opt/docker ADD --chown=daemon:daemon opt /opt USER daemon ENTRYPOINT ["/opt/docker/bin/akka-crud-service"] CMD [] Generates busybox compatible script $ sbt docker:publishLocal
  41. Useful SBT Plugins 1. sbt-release addSbtPlugin(“com.github.gseitz" % "sbt-release" % “x.y.z")

    Easy to bump version and commit to Git. 1.2.1-SNAPSHOT -> 1.2.1 -> 1.2.2-SNAPSHOT 2. sbt-scalafmt addSbtPlugin("com.geirsson" % "sbt-scalafmt" % “x.y.z") Can check & fail the build, format sources
  42. Dependency Injection val db = Database.forConfig("storage", cfg) // class TripRepository(db:

    Database) val repo = wire[TripRepository] // class TripService[F[_]](repo: Repository[F]) val service = wire[TripService[Future]] // class QueryRoutes(service: TripService[Future]) val routes = concat(wire[QueryRoutes].routes, wire[CommandRoutes].routes) import com.softwaremill.macwire._
  43. None
  44. More Info 1. Scalar 2018 whiteboard voting results! https://blog.softwaremill.com/scalar-2018-whiteboard-voting-results-c6f50f8fb16d 2.

    Scala Developer Survey Results 2018 (link) 3. Scaladex: https://index.scala-lang.org 4. Source Code: https://github.com/novakov-alexey/akka-crud-service 5. Some Scala posts: https://medium.com/se-notes-by-alexey-novakov Twitter: @alexey_novakov