Running Docker-Based Integration Tests in Scala A Case Study Matteo Di Pirro Kynetics 

About me ● Software engineer at Kynetics ● DevOps + application software ● Been writing Scala code for ~ 4 years 1/18 

Agenda ● Why testing? ● Testing with Docker in Scala ● sbt-docker-compose ● Test Containers ● Conclusions 2/18 

Why Testing is so Important ● Refactoring ○ Confidence in the application ○ Developing new features ● Documentation 3/18 

Why Testing is so Important ● Refactoring ○ Confidence in the application ○ Developing new features ● Documentation Maintenance cost increases over time 3/18 

The Testing Pyramid 4/18 

Typical Scenario 5/18 

Docker to the Rescue ● Package software into executable images ○ Docker images ○ Docker containers ● Containers as “test doubles” ○ Application dependencies ○ Infrastructural dependencies ● Suitable for integration tests 6/18 

In Scala ● Manually running Docker containers ○ Docker Compose ○ Cumbersome and error-prone ✕ Not automated!!! ● Wrapping Docker commands ourselves ✕ Reinventing the wheel! ● sbt-docker-compose ○ SBT plugin ● Test Containers ○ Java library with Scala wrapper 7/18 

A Very Simple App 1. @main 2. def bootServer(): Unit = 3. val text = "HTTP/1.0 200 OK ... ...

Hello, Madrid!

" 4. val port = 8080 5. val listener = ServerSocket(port) 6. 7. while (true) { 8. val sock = listener.accept() 9. PrintWriter(sock.getOutputStream, true).println(text) 10. sock.shutdownOutput() 11. } Code available at: https://github.com/mdipirro/docker-tests-scaladays2023 8/18 

sbt-docker-compose ✓ SBT plugin to integrate Docker Compose into the build environment ✓ Interacts with ScalaTest to pass port mappings ✓ Automatically builds a Docker image for the application ✓ Dependencies are specified via a Docker Compose file ✕ Not maintained anymore → We had to fork it to add support for DockerCompose 2 (https://github.com/Kynetics/sbt-docker-compose) ✕ Cumbersome to configure ✕ Difficult to configure the dependent containers programmatically 9/18 

sbt-docker-compose flow Create JAR with test files sbt packageBin Configure SBT to always build “latest” Docker images Run integration tests via SBT (poor IDE support) sbt dockerComposeTest 10/18 

The Docker Compose file 1. version: '3.8' 2. 3. services: 4. scalaTest: 5. image: scala-days-2023:latest 6. ports: 7. - "0:8080" 11/18 

The Docker Compose file 1. version: '3.8' 2. 3. services: 4. scalaTest: 5. image: scala-days-2023:latest 6. ports: 7. - "0:8080" 11/18 

The Docker Compose file 1. version: '3.8' 2. 3. services: 4. scalaTest: 5. image: scala-days-2023:latest 6. ports: 7. - "0:8080" 11/18 

The Docker Compose file 1. version: '3.8' 2. 3. services: 4. scalaTest: 5. image: scala-days-2023:latest 6. ports: 7. - "0:8080" 11/18 

The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture 2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18 

The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture 2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18 

The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture 2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18 

The Test Suite 1. class SampleSbtDockerComposeSpec extends FixtureAnyFunSuite with fixture.ConfigMapFixture 2. with Eventually with IntegrationPatience with ...: 3. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 4. configMap => { 5. val hostInfo = getHostInfo(configMap) 6. val client = SimpleHttpClient() 7. val req = basicRequest.get(uri"http://$hostInfo") 8. eventually { 9. val resp = client.send(req) 10. resp.code.isSuccess shouldBe true 11. resp.body.value should include("Hello, Madrid!" ) 12. } 13. } 14. } 12/18 

Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18 

Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18 

Reading Container’s Conf 1. class SampleSbtDockerComposeSpec extends ...: 2. val ServiceName = "scalaTest" 3. val ServiceHostKey = s"$ServiceName:8080" 4. ... 5. def getHostInfo(configMap: ConfigMap): String = getContainerSetting(configMap, ServiceHostKey) 6. 7. def getContainerSetting(configMap: ConfigMap, key: String): String = { 8. if (configMap.keySet.contains(key)) then configMap(key).toString 9. else { 10. throw TestFailedException( 11. message = s"Cannot find the expected Docker Compose service key ' $key'", 12. failedCodeStackDepth = 10 13. ) 14. } 15. } 13/18 

TestContainers ● Library to manage Docker containers in tests ● Creates disposable test Docker containers ● Support for different programming languages ○ Including Scala ● Lets us create containers in different ways ○ Built-in modules ○ Docker Compose file ○ Programmatically 14/18 

TestContainers Configure test task to depend on the publication of the Docker image Optionally configure SBT to publish the application version Run integration tests via SBT or IDE sbt Test / test 15/18 

The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying): 2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18 

The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying): 2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18 

The Container Definition 1. class AppContainer private(underlying: GenericContainer) extends GenericContainer(underlying): 2. lazy val port: Int = mappedPort(AppContainer. Port) 3. 4. object AppContainer: 5. private val Port = 8080 6. 7. object Def extends GenericContainer. Def[AppContainer]( 8. AppContainer( 9. GenericContainer( 10. dockerImage = System.getenv("DOCKER_IMAGE_FULL_NAME" ), 11. exposedPorts = Seq(Port), 12. waitStrategy = HostPortWaitStrategy() 13. ) 14. ) 15. ) 16/18 

The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18 

The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18 

The Test Suite 1. class SampleTestContainersSpec extends AnyFunSuite with TestContainerForAll with ...: 2. 3. override val containerDef: AppContainer.Def. type = AppContainer.Def 4. 5. test("The app returns a success code and the string 'Hello, Madrid!'" ) { 6. withContainers { app => 7. val host = app.host 8. val port = app.port 9. val client = SimpleHttpClient() 10. val req = basicRequest.get(uri"http://$host:$port") 11. val resp = client.send(req) 12. resp.code.isSuccess shouldBe true 13. resp.body.value should include("Hello, Madrid!" ) 14. } 15. } 17/18 

Conclusions ● Testing code is important ● The testing framework/libraries impact the design ● sbt-docker-compose ○ Abandoned and difficult to use ○ Generally faster than TestContainers ● TestContainers ○ Java-based, but with a Scala wrapper ○ Produces readable code 18/18 

When you hear the phrase “it’s just test code”, to me that’s a code smell. Alan Page Matteo Di Pirro [email protected]