Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Running Docker-Based Integration Tests in Scala...

Matteo Di Pirro
September 29, 2023
Programming
0
67

Running Docker-Based Integration Tests in Scala: A Case Study

When it comes to running integration tests for Scala applications, Docker containers can be a useful tool for mocking infrastructural dependencies. In this talk, we will compare two alternatives for running Docker-based integration tests in Scala: sbt-docker-compose and testcontainers-scala.

We’ll share lessons learned from our own experience using these tools for integration testing in production-ready Scala applications. We’ll discuss the trade-offs involved in choosing between sbt-docker-compose and testcontainers-scala, with a focus on execution time and test design.

Whether you’re a seasoned Scala developer or just getting started with integration testing in Docker, this presentation will provide you with valuable insights and practical tips to improve your testing workflow.

Matteo Di Pirro

September 29, 2023
Tweet

More Decks by Matteo Di Pirro

See All by Matteo Di Pirro
Can we adopt Eclipse IoT projects on Android Embedded Devices
mdipirro
0
3
Eclipse Hara: Updating Embedded Devices with hawkBit Made Easy
mdipirro
0
120
Functional IaC
mdipirro
0
85
Solidity 0.5 - When Typed does not mean Type-Safe
mdipirro
0
54
Design and Deployment in Edge Computing: A Case Study
mdipirro
0
27

Other Decks in Programming

See All in Programming
【Kaigi on Rails 2024】YOUTRUST スポンサーLT
krpk1900
1
330
Contemporary Test Cases
maaretp
0
130
型付き API リクエストを実現するいくつかの手法とその選択 / Typed API Request
euxn23
8
2.2k
タクシーアプリ『GO』のリアルタイムデータ分析基盤における機械学習サービスの活用
mot_techtalk
4
1.4k
Outline View in SwiftUI
1024jp
1
320
macOS でできる リアルタイム動画像処理
biacco42
9
2.4k
ActiveSupport::Notifications supporting instrumentation of Rails apps with OpenTelemetry
ymtdzzz
1
230
最新TCAキャッチアップ
0si43
0
140
Realtime API 入門
riofujimon
0
150
Generative AI Use Cases JP (略称:GenU)奮闘記
hideg
1
290
Enabling DevOps and Team Topologies Through Architecture: Architecting for Fast Flow
cer
PRO
0
310
AWS Lambdaから始まった
Serverlessの「熱」とキャリアパス / It started with AWS Lambda Serverless “fever” and career path
seike460
PRO
1
250

Featured

See All Featured
Building Applications with DynamoDB
mza
90
6.1k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
How STYLIGHT went responsive
nonsquared
95
5.2k
What's in a price? How to price your products and services
michaelherold
243
12k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
Building a Modern Day 
E-commerce SEO Strategy
aleyda
38
6.9k
Facilitating Awesome Meetings
lara
50
6.1k
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
Intergalactic Javascript Robots from Outer Space
tanoku
269
27k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
109
49k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
229
52k

Transcript

  1. Running Docker-Based Integration Tests in Scala A Case Study Matteo

    Di Pirro Kynetics

  2. About me • Software engineer at Kynetics • DevOps +

    application software • Been writing Scala code for ~ 4 years 1/18

  3. Agenda • Why testing? • Testing with Docker in Scala

    • sbt-docker-compose • Test Containers • Conclusions 2/18

  4. Why Testing is so Important • Refactoring ◦ Confidence in

    the application ◦ Developing new features • Documentation 3/18

  5. Why Testing is so Important • Refactoring ◦ Confidence in

    the application ◦ Developing new features • Documentation Maintenance cost increases over time 3/18

  6. The Testing Pyramid 4/18

  7. Typical Scenario 5/18

  8. Docker to the Rescue • Package software into executable images

    ◦ Docker images ◦ Docker containers • Containers as “test doubles” ◦ Application dependencies ◦ Infrastructural dependencies • Suitable for integration tests 6/18

  9. In Scala • Manually running Docker containers ◦ Docker Compose

    ◦ Cumbersome and error-prone ✕ Not automated!!! • Wrapping Docker commands ourselves ✕ Reinventing the wheel! • sbt-docker-compose ◦ SBT plugin • Test Containers ◦ Java library with Scala wrapper 7/18

  10. A Very Simple App 1. @main 2. def bootServer(): Unit

    = 3. val text = "HTTP/1.0 200 OK ... <html>...<body><p>Hello, Madrid!</p></body></html>" 4. val port = 8080 5. val listener = ServerSocket(port) 6. 7. while (true) { 8. val sock = listener.accept() 9. PrintWriter(sock.getOutputStream, true).println(text) 10. sock.shutdownOutput() 11. } Code available at: https://github.com/mdipirro/docker-tests-scaladays2023 8/18

  11. sbt-docker-compose ✓ SBT plugin to integrate Docker Compose into the

    build environment ✓ Interacts with ScalaTest to pass port mappings ✓ Automatically builds a Docker image for the application ✓ Dependencies are specified via a Docker Compose file ✕ Not maintained anymore → We had to fork it to add support for DockerCompose 2 (https://github.com/Kynetics/sbt-docker-compose) ✕ Cumbersome to configure ✕ Difficult to configure the dependent containers programmatically 9/18

  12. sbt-docker-compose flow Create JAR with test files sbt packageBin Configure

    SBT to always build “latest” Docker images Run integration tests via SBT (poor IDE support) sbt dockerComposeTest 10/18

  13. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  14. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  15. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  16. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  17. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  18. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  19. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  20. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  21. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  22. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  23. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  24. TestContainers • Library to manage Docker containers in tests •

    Creates disposable test Docker containers • Support for different programming languages ◦ Including Scala • Lets us create containers in different ways ◦ Built-in modules ◦ Docker Compose file ◦ Programmatically 14/18

  25. TestContainers Configure test task to depend on the publication of

    the Docker image Optionally configure SBT to publish the application version Run integration tests via SBT or IDE sbt Test / test 15/18

  26. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  27. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  28. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  29. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  30. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  31. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  32. Conclusions • Testing code is important • The testing framework/libraries

    impact the design • sbt-docker-compose ◦ Abandoned and difficult to use ◦ Generally faster than TestContainers • TestContainers ◦ Java-based, but with a Scala wrapper ◦ Produces readable code 18/18

  33. When you hear the phrase “it’s just test code”, to

    me that’s a code smell. Alan Page Matteo Di Pirro [email protected]