How fixing a broken window cut down our build time by 50%

The broken windows (not the operating system - 😁) theory not only applies to criminological cases but also codebases. It only takes one lazy developer to break the first window. Chances are high that others will follow a similar path and take workarounds or re-apply a not so optimal pattern thinking, "It passed the code review, I must not be that bad".

For our Spring Boot integration test setup, we had such a broken window that was causing ever-growing build times. While most books about clean architecture emphasize composition over inheritance, we drifted off from this rule and coupled our tests to an AbstractIntegrationTest class that we then annotated with @DirtiesContext...

This resulted in ever-growing build times as each new integration test was launching its own Spring TestContext, adding valuable seconds to the overall build time. Our feedback cycles, as well as the overall satisfaction of the team, started to suffer from this.

This talk describes our journey of getting rid of @DirtiesContext on top of an abstract test class (aka. the "broken window") that made reusing the Spring TestContext impossible and resulted in ever-growing build times. Furthermore, we'll discuss recipes, best practices, and antipatterns for resuing the Spring TestContext and testing Spring Boot applications in general.

After fixing this "broken window", our overall build time went down by 50% from 25 to 12 minutes.

Transcript

1. View Slide

2. @rieckpil
   How Fixing a Broken Window Cut
   Down Our Build Time by 50%
   

   View Slide

3. @rieckpil
   How It Started
   3
   

   View Slide

4. The Broken
   Windows Theory*
   “The broken windows theory is a
   criminological theory that states that
   visible signs of crime, anti-social
   behavior, and civil disorder create an
   urban environment that encourages
   further crime and disorder, including
   serious crimes.
   The theory suggests that policing
   methods that target minor crimes
   such as vandalism, loitering, public
   drinking, jaywalking, and fare evasion
   help to create an atmosphere of order
   and lawfulness.”
   https://en.wikipedia.org/wiki/Broken_windows_theor
   y
   *no relationship to the operating system
   4
   

   View Slide

5. @rieckpil
   It Does Not Always Have to Be a Window
   5
   

   View Slide

6. @rieckpil
   It Does Not Always Have to Be a Window
   6
   

   View Slide

7. @rieckpil
   Impact of Fixing the Broken Window
   Before fixing the broken window
   After fixing the broken window
   7
   

   View Slide

8. @rieckpil
   Agenda for This Talk
   1. Introduction ✅
   2. About Me
   3. Theory: Spring Test Context Caching & @DirtiesContext
   4. Practice: Our Journey of Fixing the Broken Window
   5. Conclusion and Q&A
   8
   

   View Slide

9. @rieckpil
   Goals for This Talk
   Emphasize the importance of well-
   written tests
   Learn more about the Spring Test
   context caching mechanism
   Don’t be the person that breaks the
   first window!
   Understand the impact a small
   workaround can have on the health
   of your project
   9
   🏆
   

   View Slide

10. @rieckpil
    About Me
    - Freelance IT consultant from Berlin, Germany
    - Enjoys writing blog posts on rainy days in cozy cafes (#TeamWordPress)
    - Blogging & content creation for more than five years. Since two years with a
    focus on testing Java and specifically Spring Boot applications
    - Co-authored Stratospheric - Form Zero to Production With Spring Boot and AWS
    together with Björn Wilmsmann (@bwilmsmann) and Tom Hombergs
    (@TomHombergs)
    - Blog: https://rieckpil.de
    - GitHub: https://github.com/rieckpil/
    - YouTube: https://www.youtube.com/c/rieckpil
    10
    

    View Slide

11. Spring TestContext
    Framework:
    Context Caching
    Theory
    - Part of Spring Test (automatically
    part of every Spring Boot project
    via spring-boot-starter-
    test)
    - Spring TestContext Framework:
    caches an already started Spring
    context for later reuse
    - Speed up the test execution time
    by reusing an already started
    context
    11
    

    View Slide

12. @rieckpil
    Spring TestContext Framework Context Caching
    - Configurable cache size (default is 32) with LRU (least recently used) strategy
    - This goes into the cache key (MergedContextConfiguration):
    - activeProfiles (@ActiveProfiles)
    - contextInitializersClasses (@ContextConfiguration)
    - propertySourceLocations (@TestPropertySource)
    - propertySourceProperties (@TestPropertySource)
    - contextCustomizer (@MockBean, @DynamicPropertySource, …)
    - etc.
    12
    

    View Slide

13. @rieckpil
    Can Spring Reuse the Context?
    13
    

    View Slide

14. @rieckpil
    Can Spring Reuse the Context?
    14
    

    View Slide

15. @rieckpil
    Context Caching and Reusability Red Flags
    15
    

    View Slide

16. @rieckpil
    Context Caching Indicators
    16
    

    View Slide

17. @rieckpil
    Context Caching Indicators
    17
    

    View Slide

18. @rieckpil
    @DritiesContext aka. Our Broken Window
    From the @DirtiesContext Javadoc
    > Test annotation which indicates that the ApplicationContext associated with a test is
    dirty and should therefore be closed and removed from the context cache.
    Use this annotation if a test has modified the context — for example, by modifying the
    state of a singleton bean, modifying the state of an embedded database, etc. Subsequent
    tests that request the same context will be supplied a new context.
    18
    

    View Slide

19. @rieckpil
    Our Status Quo
    - Almost all our integration tests (*IT) extended the AbstractIntegrationTest class
    - Every new IT was implicitly inheriting this “bad behavior” and increased our build
    time by 20-30 seconds
    - No effort was made to work on a fix
    - We had multiple attempts of “just remove @DirtiesContext” and see what breaks
    - The increasing build times resulted in a decreasing motivation to add further
    valuable and important integration test
    - Our overall team motivation decreased as the feedback cycles went slower and
    slower
    19
    

    View Slide

20. @rieckpil
    Share the new
    knowledge
    Get rid of
    @DirtiesContext
    Identify hot spots
    and rabbit holes
    Plan this tech debt
    removal accordingly
    Our Strategy
    20
    🚀
    

    View Slide

21. @rieckpil
    Pitfall #1: Message Thefts
    21
    

    View Slide

22. @rieckpil
    Pitfall #1: Message Thefts
    22
    

    View Slide

23. @rieckpil
    Pitfall #1: Message Thefts
    23
    

    View Slide

24. @rieckpil
    Pitfall #1: Message Thefts
    24
    

    View Slide

25. @rieckpil
    Pitfall #1: Message Thefts
    25
    

    View Slide

26. @rieckpil
    Pitfall #2: Data Overflow
    26
    

    View Slide

27. @rieckpil
    Pitfall #2: Data Overflow
    - Avoid any test data leftovers between tests
    - Concepts to understand:
    - EntityManager flushing
    - Second-level vs. first-level cache
    - Transaction handling during tests
    - Ensure every test starts with a clean state
    - Provision the database via Testcontainers
    - Use schema migration tools like Flyway or Liquibase
    - LocalStack for S3 Buckets, etc.
    27
    💽
    

    View Slide

28. @rieckpil
    Pitfall #3: Remote Connections
    28
    

    View Slide

29. @rieckpil
    Pitfall #3: Remote Connections
    29
    - Aim for “airplane” mode
    - Avoid any real HTTP
    communication to external
    services during the build
    - Predictable and reproducible test
    outcomes
    - WireMock to the rescue!
    

    View Slide

30. @rieckpil
    Pitfall #3: Remote Connections
    30
    

    View Slide

31. @rieckpil
    Pitfall #3: Remote Connections
    31
    

    View Slide

32. @rieckpil
    Pitfall #3: Remote Connections
    32
    …
    

    View Slide

33. @rieckpil
    Pitfall #4: @SpringBootTest Obsession
    33
    

    View Slide

34. @rieckpil
    Pitfall #4: @SpringBootTest Obsession
    34
    🙈
    

    View Slide

35. @rieckpil
    Pitfall #4: @SpringBootTest Obsession
    35
    

    View Slide

36. @rieckpil
    Pitfall #4: @SpringBootTest Obsession
    36
    

    View Slide

37. @rieckpil
    Pitfall #5: Setup Variety
    37
    

    View Slide

38. @rieckpil
    Pitfall #5: Setup Variety
    - Group the config to the least amount of profiles needed, for example:
    integration-test and web-test
    - Avoid “unique snowflake” context setups
    - Don’t overuse @MockBean & @SpyBean
    - Avoid manual property overrides where possible
    - Understand what goes into the context cache key
    - (Custom) JUnit Jupiter extensions > extending an abstract test base class
    38
    🥗
    

    View Slide

39. @rieckpil
    The Result
    - Three-day full-time investment
    - Reducing the build time from 26 minutes to 12 minutes
    - New integration tests are only adding a negligible amount of additional build time
    - Final context cache size: 3 (before: 20)
    - Internal knowledge sharing session about the identified pitfalls and applied
    techniques & patterns
    - Increased awareness for this topic when writing tests for Spring Boot applications
    39
    🥳
    

    View Slide

40. @rieckpil
    Testing Spring Boot Applications Masterclass
    - Testing real-world Spring Boot applications
    - 130+ course lessons with 10h+ hands-on video content
    - Tech-Stack: Spring Boot 2.7, PostgreSQL, React 18, Java 17, Keycloak (OAuth
    2 login), communication with a remote HTTP system, async workflows with a
    messaging queue (Amazon SQS)
    - Techniques, pitfalls, and patterns for unit, integration, and end-to-end testing
    - Best of breed testing tools: JUnit 5, Mockito, LocalStack, Testcontainers,
    Awaitility, WireMock, MockWebServer, Selenide, etc.
    - PS: Spring Test context caching is also covered in-depth
    - https://rieckpil.de/testing-spring-boot-applications-masterclass/
    40
    

    View Slide

41. @rieckpil
    Stratospheric - From Zero to Production with Spring
    Boot and AWS
    - 450+ pages long hands-on ebook about getting your Spring Boot app running
    on AWS: Available on Leanpub
    - Java 17, Spring Boot 2.7 with Spring Cloud AWS
    - Fully automated CI/CD pipeline with GitHub Actions
    - AWS Services: ECS, ECR, SQS, CloudWatch, RDS, Cognito, MQ, etc.
    - Infrastructure as Code (written in Java) with the AWS CDK v2
    - Complementary online course is currently in progress
    - More information (discount, mailing list, etc.): https://stratospheric.dev/
    41
    

    View Slide

42. @rieckpil
    Conclusion
    - Think twice before implementing a workaround (aka. breaking the first window)
    - Treat testing code as production code and keep it maintainable and clean
    - Don’t religiously try to minimize the number of contexts: measure & adjust
    - Try to avoid using @DirtiesContext at a central place
    - Show some love for your test suite (your future self and coworkers will thank you)
    - Avoid having unique snowflakes aka. fruit salads when it comes to your test
    ApplicationContext setup
    - Composition over inheritance
    - Short feedback cycles ftw!
    42
    🏆
    

    View Slide

43. @rieckpil
    Q&A
    Questions?
    Slides will be uploaded ✅
    The best way to get in contact is via Twitter @rieckpil (DMs are open) ✅
    Visit my blog and join the mailing list for more testing-related Spring Boot content
    ✅
    Enjoy the rest of the conference ✅
    43
    

    View Slide