Devoxx MA: Testcontainers deep dive

Transcript

1. Testcontainers
   Integration testing without the hassle
   Sergei @bsideup Egorov
   

   View Slide

2. About me
   • Testcontainers co-maintainer
   
   • Staff Engineer at Pivotal’s Spring R&D, working on Project Reactor ⚛
   
   • Berlin Spring User Group co-organizer
   
   • Developer tools geek
   @bsideup
   

   View Slide

3. Integration testing?
   

   View Slide

4. @bsideup
   

   View Slide

5. @bsideup
   

   View Slide

6. https://martinfowler.com/bliki/IntegrationTest.html @bsideup
   

   View Slide

7. https://martinfowler.com/bliki/IntegrationTest.html @bsideup
   

   View Slide

8. https://martinfowler.com/bliki/IntegrationTest.html @bsideup
   

   View Slide

9. https://labs.spotify.com/2018/01/11/testing-of-microservices/
   “Microservices Testing Honeycomb”
   @bsideup
   

   View Slide

10. https://labs.spotify.com/2018/01/11/testing-of-microservices/
    “Microservices Testing Honeycomb”
    @bsideup
    We’re here
    

    View Slide

11. Some signs of having Integrated Tests are:
    • We spin up other services in a local testing environment
    • We test against other services in a shared testing environment
    • Changes to your system breaks tests for other systems
    https://labs.spotify.com/2018/01/11/testing-of-microservices/ @bsideup
    

    View Slide

12. So…

    Integration testing.
    

    View Slide

13. Verify how your software product will behave in real-
    world conditions
    

    View Slide

14. Isolated from other system components to avoid false negatives
    

    View Slide

15. Real:
    
    databases,
    
    file systems,
    
    network interfaces, …
    

    View Slide

16. Grey box testing - we know some inner details,
    but mostly use public APIs
    

    View Slide

17. View Slide

18. Unit testing
    • Simulation tests are green, yay!
    
    • Everything is mocked
    
    • DB is written by others, why
    should I test it?
    @bsideup
    

    View Slide

19. Integration testing
    https://commons.wikimedia.org/wiki/File:Cd4007.jpg
    • Jeez, how did it passed the
    Unit testing?
    
    • Is it a smoke? @#$%! YES IT
    IS!
    
    • Who knew that 100w
    soldering gun was a bit too
    powerful for it?
    @bsideup
    

    View Slide

20. System testing
    https://commons.wikimedia.org/wiki/File:UART_8250_Microchip.jpg
    • Takes a lot of time to solder
    
    • “Oh no, the power bus is too
    far away from my microchip!”
    
    • Interference between the
    components
    @bsideup
    

    View Slide

21. Production
    @bsideup
    

    View Slide

22. Integration testing
    Real-world, but isolated testing
    
    Spot the issues before the real
    environment
    
    Can be run during the
    development
    You have to start real databases
    
    Should be cross-platform
    
    Slower than Unit testing
    Pros Cons
    @bsideup
    

    View Slide

23. Integration testing transformation
    @bsideup
    

    View Slide

24. Mocking
    Integration testing transformation
    @bsideup
    

    View Slide

25. Mocking
    Local DBs
    Integration testing transformation
    @bsideup
    

    View Slide

26. Mocking
    Local DBs
    VMs

    (Vagrant)
    Integration testing transformation
    @bsideup
    

    View Slide

27. Mocking
    Local DBs
    VMs

    (Vagrant)
    Docker
    Integration testing transformation
    @bsideup
    

    View Slide

28. View Slide

29. Abstraction layer
    

    View Slide

30. CI friendly
    

    View Slide

31. Cross-platform
    

    View Slide

32. Mocking
    Local DBs
    VMs

    (Vagrant)
    Docker
    Integration testing transformation
    @bsideup
    

    View Slide

33. Mocking
    Local DBs
    VMs

    (Vagrant)
    Docker
    Fig

    (aka Docker Compose)
    Integration testing transformation
    @bsideup
    

    View Slide

34. Docker Compose FTW!
    redis:
    image: redis
    ports:
    - "6379:6379"
    postgres:
    image: postgres
    ports:
    - "5432:5432"
    elasticsearch:
    image: elasticsearch:5.0.0
    ports:
    - "9200:9200"
    @bsideup
    

    View Slide

35. But…
    

    View Slide

36. Declarative YAML
    redis:
    image: redis
    ports:
    - "6379:6379"
    postgres:
    image: postgres
    ports:
    - "5432:5432"
    elasticsearch:
    image: elasticsearch:5.0.0
    ports:
    - "9200:9200"
    @bsideup
    

    View Slide

37. No ports randomization
    redis:
    image: redis
    ports:
    - "6379:6379"
    postgres:
    image: postgres
    ports:
    - "5432:5432"
    elasticsearch:
    image: elasticsearch:5.0.0
    ports:
    - "9200:9200"
    @bsideup
    

    View Slide

38. Container per test?
    redis:
    image: redis
    ports:
    - "6379:6379"
    postgres:
    image: postgres
    ports:
    - "5432:5432"
    elasticsearch:
    image: elasticsearch:5.0.0
    ports:
    - "9200:9200"
    @bsideup
    

    View Slide

39. IDE integration?
    @bsideup
    

    View Slide

40. Mocking
    Local DBs
    VMs

    (Vagrant)
    Docker
    Fig

    (aka Docker Compose)
    Integration testing transformation
    @bsideup
    

    View Slide

41. Mocking
    Local DBs
    VMs

    (Vagrant)
    Docker
    Fig

    (aka Docker Compose)
    Docker API
    Integration testing transformation
    @bsideup
    

    View Slide

42. Fighting with Docker environment
    

    View Slide

43. There is no place like
    

    View Slide

44. There is no place like
    … unless there is
    

    View Slide

45. Can we improve that?
    

    View Slide

46. View Slide

47. Testcontainers
    • Created by Richard North in 2015
    
    • github.com/testcontainers/testcontainers-java
    • Wraps docker-java library
    
    • Docker environment discovery (Win, Mac, Linux)
    
    • Containers cleanup on JVM shutdown
    @bsideup
    

    View Slide

48. As simple as
    PostgreSQLContainer postgresql = new PostgreSQLContainer()
    GenericContainer redis = new GenericContainer("redis:3")
    .withExposedPorts(6379)
    @bsideup
    

    View Slide

49. Users
    

    View Slide

50. @bsideup
    

    View Slide

51. Use case: testing of microservices
    • REST service
    
    • Java, Spring Boot
    
    • Redis, Kafka and PostgreSQL
    
    • Calls some other micro-services
    @bsideup
    

    View Slide

52. @RunWith(SpringRunner.class)
    @SpringBootTest(webEnvironment = RANDOM_PORT)
    @ContextConfiguration(initializers = Initializer.class)
    public abstract class AbstractIntegrationTest {
    @ClassRule
    public static GenericContainer redis = new GenericContainer("redis:3.0.6")
    .withExposedPorts(6379);
    @ClassRule
    public static MockServerContainer mockServer = new MockServerContainer();
    }
    @bsideup
    

    View Slide

53. @RunWith(SpringRunner.class)
    @SpringBootTest(webEnvironment = RANDOM_PORT)
    @ContextConfiguration(initializers = Initializer.class)
    public abstract class AbstractIntegrationTest {
    @ClassRule
    public static GenericContainer redis = new GenericContainer("redis:3.0.6")
    .withExposedPorts(6379);
    @ClassRule
    public static MockServerContainer mockServer = new MockServerContainer();
    }
    Still using all the
    Spring goodies!
    @bsideup
    

    View Slide

54. @RunWith(SpringRunner.class)
    @SpringBootTest(webEnvironment = RANDOM_PORT)
    @ContextConfiguration(initializers = Initializer.class)
    public abstract class AbstractIntegrationTest {
    @ClassRule
    public static GenericContainer redis = new GenericContainer("redis:3.0.6")
    .withExposedPorts(6379);
    @ClassRule
    public static MockServerContainer mockServer = new MockServerContainer();
    }
    External
    dependencies
    @bsideup
    

    View Slide

55. View Slide

56. Demo
    

    View Slide

57. Takeaways
    • https://testcontainers.org
    • Works on Linux, Mac and Windows
    • …including CIs like Jenkins, Travis, CircleCI, GH Actions, Azure Pipelines, …
    
    • Provides a great balance between 

    flexibility, usability, speed and features

    @bsideup
    

    View Slide

58. Questions?
    

    View Slide

59. @bsideup
    bsideup
    

    View Slide

60. https://commons.wikimedia.org/wiki/File:Thats_all_folks.svg
    

    View Slide