Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Running Docker-Based Integration Tests in Scala...

Matteo Di Pirro
September 29, 2023
Programming
0
74

Running Docker-Based Integration Tests in Scala: A Case Study

When it comes to running integration tests for Scala applications, Docker containers can be a useful tool for mocking infrastructural dependencies. In this talk, we will compare two alternatives for running Docker-based integration tests in Scala: sbt-docker-compose and testcontainers-scala.

We’ll share lessons learned from our own experience using these tools for integration testing in production-ready Scala applications. We’ll discuss the trade-offs involved in choosing between sbt-docker-compose and testcontainers-scala, with a focus on execution time and test design.

Whether you’re a seasoned Scala developer or just getting started with integration testing in Docker, this presentation will provide you with valuable insights and practical tips to improve your testing workflow.

Matteo Di Pirro

September 29, 2023
Tweet

More Decks by Matteo Di Pirro

See All by Matteo Di Pirro
From Embedded to IaC: Exploring Scala’s Role Across Modern Applications
mdipirro
0
5
Can we adopt Eclipse IoT projects on Android Embedded Devices
mdipirro
0
7
Eclipse Hara: Updating Embedded Devices with hawkBit Made Easy
mdipirro
0
130
Functional IaC
mdipirro
0
92
Solidity 0.5 - When Typed does not mean Type-Safe
mdipirro
0
60
Design and Deployment in Edge Computing: A Case Study
mdipirro
0
32

Other Decks in Programming

See All in Programming
Go の GC の不得意な 部分を克服したい
taiyow
2
770
これが俺の”自分戦略” プロセスを楽しんでいこう！ - Developers CAREER Boost 2024
niftycorp
PRO
0
190
LLM Supervised Fine-tuningの理論と実践
datanalyticslabo
3
1.1k
良いユニットテストを書こう
mototakatsu
5
1.9k
CSC305 Lecture 25
javiergs
PRO
0
130
これでLambdaが不要に？！Step FunctionsのJSONata対応について
iwatatomoya
2
3.6k
Zoneless Testing
rainerhahnekamp
0
120
선언형 UI에서의 상태관리
l2hyunwoo
0
140
ソフトウェアの振る舞いに着目し 複雑な要件の開発に立ち向かう
rickyban
0
890
フロントエンドのディレクトリ構成どうしてる？　Feature-Sliced Design 導入体験談
osakatechlab
8
4.1k
「Chatwork」Android版アプリを 支える単体テストの現在
okuzawats
0
180
ゆるやかにgolangci-lintのルールを強くする / Kyoto.go #56
utgwkk
1
370

Featured

See All Featured
Product Roadmaps are Hard
iamctodd
PRO
49
11k
Code Reviewing Like a Champion
maltzj
520
39k
Done Done
chrislema
181
16k
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Side Projects
sachag
452
42k
The Language of Interfaces
destraynor
154
24k
Unsuck your backbone
ammeep
669
57k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
6
510
Making Projects Easy
brettharned
116
5.9k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
0
97
Bash Introduction
62gerente
608
210k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k

Transcript

  1. Running Docker-Based Integration Tests in Scala A Case Study Matteo

    Di Pirro Kynetics

  2. About me • Software engineer at Kynetics • DevOps +

    application software • Been writing Scala code for ~ 4 years 1/18

  3. Agenda • Why testing? • Testing with Docker in Scala

    • sbt-docker-compose • Test Containers • Conclusions 2/18

  4. Why Testing is so Important • Refactoring ◦ Confidence in

    the application ◦ Developing new features • Documentation 3/18

  5. Why Testing is so Important • Refactoring ◦ Confidence in

    the application ◦ Developing new features • Documentation Maintenance cost increases over time 3/18

  6. The Testing Pyramid 4/18

  7. Typical Scenario 5/18

  8. Docker to the Rescue • Package software into executable images

    ◦ Docker images ◦ Docker containers • Containers as “test doubles” ◦ Application dependencies ◦ Infrastructural dependencies • Suitable for integration tests 6/18

  9. In Scala • Manually running Docker containers ◦ Docker Compose

    ◦ Cumbersome and error-prone ✕ Not automated!!! • Wrapping Docker commands ourselves ✕ Reinventing the wheel! • sbt-docker-compose ◦ SBT plugin • Test Containers ◦ Java library with Scala wrapper 7/18

  10. A Very Simple App 1. @main 2. def bootServer(): Unit

    = 3. val text = "HTTP/1.0 200 OK ... <html>...<body><p>Hello, Madrid!</p></body></html>" 4. val port = 8080 5. val listener = ServerSocket(port) 6. 7. while (true) { 8. val sock = listener.accept() 9. PrintWriter(sock.getOutputStream, true).println(text) 10. sock.shutdownOutput() 11. } Code available at: https://github.com/mdipirro/docker-tests-scaladays2023 8/18

  11. sbt-docker-compose ✓ SBT plugin to integrate Docker Compose into the

    build environment ✓ Interacts with ScalaTest to pass port mappings ✓ Automatically builds a Docker image for the application ✓ Dependencies are specified via a Docker Compose file ✕ Not maintained anymore → We had to fork it to add support for DockerCompose 2 (https://github.com/Kynetics/sbt-docker-compose) ✕ Cumbersome to configure ✕ Difficult to configure the dependent containers programmatically 9/18

  12. sbt-docker-compose flow Create JAR with test files sbt packageBin Configure

    SBT to always build “latest” Docker images Run integration tests via SBT (poor IDE support) sbt dockerComposeTest 10/18

  13. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  14. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  15. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  16. The Docker Compose file 1. version: '3.8' 2. 3. services:

    4. scalaTest: 5. image: scala-days-2023:latest<localBuild> 6. ports: 7. - "0:8080" 11/18

  17. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  18. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  19. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  20. The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture

    2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18

  21. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  22. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  23. Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val

    ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18

  24. TestContainers • Library to manage Docker containers in tests •

    Creates disposable test Docker containers • Support for different programming languages ◦ Including Scala • Lets us create containers in different ways ◦ Built-in modules ◦ Docker Compose file ◦ Programmatically 14/18

  25. TestContainers Configure test task to depend on the publication of

    the Docker image Optionally configure SBT to publish the application version Run integration tests via SBT or IDE sbt Test / test 15/18

  26. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  27. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  28. The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying):

    2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18

  29. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  30. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  31. The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll

    with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18

  32. Conclusions • Testing code is important • The testing framework/libraries

    impact the design • sbt-docker-compose ◦ Abandoned and difficult to use ◦ Generally faster than TestContainers • TestContainers ◦ Java-based, but with a Scala wrapper ◦ Produces readable code 18/18

  33. When you hear the phrase “it’s just test code”, to

    me that’s a code smell. Alan Page Matteo Di Pirro [email protected]