Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Running Docker-Based Integration Tests in Scala: A Case Study

Matteo Di Pirro
September 29, 2023
Programming
0
47

Running Docker-Based Integration Tests in Scala: A Case Study

When it comes to running integration tests for Scala applications, Docker containers can be a useful tool for mocking infrastructural dependencies. In this talk, we will compare two alternatives for running Docker-based integration tests in Scala: sbt-docker-compose and testcontainers-scala.

We’ll share lessons learned from our own experience using these tools for integration testing in production-ready Scala applications. We’ll discuss the trade-offs involved in choosing between sbt-docker-compose and testcontainers-scala, with a focus on execution time and test design.

Whether you’re a seasoned Scala developer or just getting started with integration testing in Docker, this presentation will provide you with valuable insights and practical tips to improve your testing workflow.

Matteo Di Pirro

September 29, 2023
Tweet

More Decks by Matteo Di Pirro

See All by Matteo Di Pirro
Eclipse Hara: Updating Embedded Devices with hawkBit Made Easy
mdipirro
0
81
Functional IaC
mdipirro
0
65
Solidity 0.5 - When Typed does not mean Type-Safe
mdipirro
0
48
Design and Deployment in Edge Computing: A Case Study
mdipirro
0
17

Other Decks in Programming

See All in Programming
dbtのドメイン分割による データ基盤の改善とDigdagとの連携
sakama
0
430
AWS CDKコントリビュートTIPS / aws-cdk-contribution-tips
gotok365
3
310
StoreKit2によるiOSのアプリ内課金のリニューアル
kangnux
0
120
2 週間で Twitter Bot を作ってみた
contour_gara
0
740
Apache Hive 4 on Treasure Data
ryukobayashi
0
400
Anthropic Cookbook のおすすめレシピ
schroneko
7
1.1k
Amazon SQSコンシューマー疎結合への旅 - 出張！ #DevelopersIO IT技術ブログの中の人が語る勉強会 #3
quiver
0
300
Azure OpenAI Serviceのプロンプトエンジニアリング入門
tomokusaba
3
850
Going beyond Apache Parquet's default settings
xhochy
0
120
Micro Frontends for Java Microservices - Utah JUG 2024
mraible
PRO
1
100
Site Reliability Engineering for GMO
pyama86
8
1.1k
初心者のためのRubyKaigi入門/RubyKaigi Introduction
a_matsuda
8
1.3k

Featured

See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
18
1.6k
The World Runs on Bad Software
bkeepers
PRO
61
6.7k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
34
8.9k
How STYLIGHT went responsive
nonsquared
92
4.8k
Agile that works and the tools we love
rasmusluckow
325
20k
The Cult of Friendly URLs
andyhume
74
5.7k
Typedesign – Prime Four
hannesfritz
36
2.1k
Making the Leap to Tech Lead
cromwellryan
125
8.5k
Become a Pro
speakerdeck
PRO
12
4.6k
Building Better People: How to give real-time feedback that sticks.
wjessup
356
18k
Designing Experiences People Love
moore
136
23k
Building Applications with DynamoDB
mza
88
5.6k

Transcript

  1. Running Docker-Based Integration Tests in Scala A Case Study Matteo

    Di Pirro Kynetics

  2. About me • Software engineer at Kynetics • DevOps +

    application software • Been writing Scala code for ~ 4 years 1/18

  3. Agenda • Why testing? • Testing with Docker in Scala

    • sbt-docker-compose • Test Containers • Conclusions 2/18

  4. Why Testing is so Important • Refactoring ◦ Confidence in

    the application ◦ Developing new features • Documentation 3/18

  5. Why Testing is so Important • Refactoring ◦ Confidence in

    the application ◦ Developing new features • Documentation Maintenance cost increases over time 3/18

  6. The Testing Pyramid 4/18

  7. Typical Scenario 5/18

  8. Docker to the Rescue • Package software into executable images

    ◦ Docker images ◦ Docker containers • Containers as “test doubles” ◦ Application dependencies ◦ Infrastructural dependencies • Suitable for integration tests 6/18

  9. In Scala • Manually running Docker containers ◦ Docker Compose

    ◦ Cumbersome and error-prone ✕ Not automated!!! • Wrapping Docker commands ourselves ✕ Reinventing the wheel! • sbt-docker-compose ◦ SBT plugin • Test Containers ◦ Java library with Scala wrapper 7/18

  10. A Very Simple App 1. @main 2. def bootServer(): Unit

    = 3. val text = "HTTP/1.0 200 OK ... <html>...<body><p>Hello, Madrid!</p></body></html>" 4. val port = 8080 5. val listener = ServerSocket(port) 6. 7. while (true) { 8. val sock = listener.accept() 9. PrintWriter(sock.getOutputStream, true).println(text) 10. sock.shutdownOutput() 11. } Code available at: https://github.com/mdipirro/docker-tests-scaladays2023 8/18

  11. sbt-docker-compose ✓ SBT plugin to integrate Docker Compose into the

    build environment ✓ Interacts with ScalaTest to pass port mappings ✓ Automatically builds a Docker image for the application ✓ Dependencies are specified via a Docker Compose file ✕ Not maintained anymore → We had to fork it to add support for DockerCompose 2 (https://github.com/Kynetics/sbt-docker-compose) ✕ Cumbersome to configure ✕ Difficult to configure the dependent containers programmatically 9/18

  12. sbt-docker-compose flow Create JAR with test files sbt packageBin Configure

    SBT to always build “latest” Docker images Run integration tests via SBT (poor IDE support) sbt dockerComposeTest 10/18

  13. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  14. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  15. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  16. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  17. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  18. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  19. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  20. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  21. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  22. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  23. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  24. TestContainers • Library to manage Docker containers in tests •

    Creates disposable test Docker containers • Support for different programming languages ◦ Including Scala • Lets us create containers in different ways ◦ Built-in modules ◦ Docker Compose file ◦ Programmatically 14/18

  25. TestContainers Configure test task to depend on the publication of

    the Docker image Optionally configure SBT to publish the application version Run integration tests via SBT or IDE sbt Test / test 15/18

  26. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  27. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  28. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  29. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  30. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  31. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  32. Conclusions • Testing code is important • The testing framework/libraries

    impact the design • sbt-docker-compose ◦ Abandoned and difficult to use ◦ Generally faster than TestContainers • TestContainers ◦ Java-based, but with a Scala wrapper ◦ Produces readable code 18/18

  33. When you hear the phrase “it’s just test code”, to

    me that’s a code smell. Alan Page Matteo Di Pirro [email protected]