$30 off During Our Annual Pro Sale. View Details »

How fixing a broken window cut down our build time by 50%

How fixing a broken window cut down our build time by 50%

The broken windows (not the operating system - 😁) theory not only applies to criminological cases but also codebases. It only takes one lazy developer to break the first window. Chances are high that others will follow a similar path and take workarounds or re-apply a not so optimal pattern thinking, "It passed the code review, I must not be that bad".

For our Spring Boot integration test setup, we had such a broken window that was causing ever-growing build times. While most books about clean architecture emphasize composition over inheritance, we drifted off from this rule and coupled our tests to an AbstractIntegrationTest class that we then annotated with @DirtiesContext...

This resulted in ever-growing build times as each new integration test was launching its own Spring TestContext, adding valuable seconds to the overall build time. Our feedback cycles, as well as the overall satisfaction of the team, started to suffer from this.

This talk describes our journey of getting rid of @DirtiesContext on top of an abstract test class (aka. the "broken window") that made reusing the Spring TestContext impossible and resulted in ever-growing build times. Furthermore, we'll discuss recipes, best practices, and antipatterns for resuing the Spring TestContext and testing Spring Boot applications in general.

After fixing this "broken window", our overall build time went down by 50% from 25 to 12 minutes.

Philip Riecks

May 27, 2022
Tweet

More Decks by Philip Riecks

Other Decks in Programming

Transcript

  1. 1

  2. @rieckpil How Fixing a Broken Window Cut Down Our Build

    Time by 50%
  3. @rieckpil How It Started 3

  4. The Broken Windows Theory* “The broken windows theory is a

    criminological theory that states that visible signs of crime, anti-social behavior, and civil disorder create an urban environment that encourages further crime and disorder, including serious crimes. The theory suggests that policing methods that target minor crimes such as vandalism, loitering, public drinking, jaywalking, and fare evasion help to create an atmosphere of order and lawfulness.” https://en.wikipedia.org/wiki/Broken_windows_theor y *no relationship to the operating system 4
  5. @rieckpil It Does Not Always Have to Be a Window

    5
  6. @rieckpil It Does Not Always Have to Be a Window

    6
  7. @rieckpil Impact of Fixing the Broken Window Before fixing the

    broken window After fixing the broken window 7
  8. @rieckpil Agenda for This Talk 1. Introduction ✅ 2. About

    Me 3. Theory: Spring Test Context Caching & @DirtiesContext 4. Practice: Our Journey of Fixing the Broken Window 5. Conclusion and Q&A 8
  9. @rieckpil Goals for This Talk Emphasize the importance of well-

    written tests Learn more about the Spring Test context caching mechanism Don’t be the person that breaks the first window! Understand the impact a small workaround can have on the health of your project 9 🏆
  10. @rieckpil About Me - Freelance IT consultant from Berlin, Germany

    - Enjoys writing blog posts on rainy days in cozy cafes (#TeamWordPress) - Blogging & content creation for more than five years. Since two years with a focus on testing Java and specifically Spring Boot applications - Co-authored Stratospheric - Form Zero to Production With Spring Boot and AWS together with Björn Wilmsmann (@bwilmsmann) and Tom Hombergs (@TomHombergs) - Blog: https://rieckpil.de - GitHub: https://github.com/rieckpil/ - YouTube: https://www.youtube.com/c/rieckpil 10
  11. Spring TestContext Framework: Context Caching Theory - Part of Spring

    Test (automatically part of every Spring Boot project via spring-boot-starter- test) - Spring TestContext Framework: caches an already started Spring context for later reuse - Speed up the test execution time by reusing an already started context 11
  12. @rieckpil Spring TestContext Framework Context Caching - Configurable cache size

    (default is 32) with LRU (least recently used) strategy - This goes into the cache key (MergedContextConfiguration): - activeProfiles (@ActiveProfiles) - contextInitializersClasses (@ContextConfiguration) - propertySourceLocations (@TestPropertySource) - propertySourceProperties (@TestPropertySource) - contextCustomizer (@MockBean, @DynamicPropertySource, …) - etc. 12
  13. @rieckpil Can Spring Reuse the Context? 13

  14. @rieckpil Can Spring Reuse the Context? 14

  15. @rieckpil Context Caching and Reusability Red Flags 15

  16. @rieckpil Context Caching Indicators 16

  17. @rieckpil Context Caching Indicators 17

  18. @rieckpil @DritiesContext aka. Our Broken Window From the @DirtiesContext Javadoc

    > Test annotation which indicates that the ApplicationContext associated with a test is dirty and should therefore be closed and removed from the context cache. Use this annotation if a test has modified the context — for example, by modifying the state of a singleton bean, modifying the state of an embedded database, etc. Subsequent tests that request the same context will be supplied a new context. 18
  19. @rieckpil Our Status Quo - Almost all our integration tests

    (*IT) extended the AbstractIntegrationTest class - Every new IT was implicitly inheriting this “bad behavior” and increased our build time by 20-30 seconds - No effort was made to work on a fix - We had multiple attempts of “just remove @DirtiesContext” and see what breaks - The increasing build times resulted in a decreasing motivation to add further valuable and important integration test - Our overall team motivation decreased as the feedback cycles went slower and slower 19
  20. @rieckpil Share the new knowledge Get rid of @DirtiesContext Identify

    hot spots and rabbit holes Plan this tech debt removal accordingly Our Strategy 20 🚀
  21. @rieckpil Pitfall #1: Message Thefts 21

  22. @rieckpil Pitfall #1: Message Thefts 22

  23. @rieckpil Pitfall #1: Message Thefts 23

  24. @rieckpil Pitfall #1: Message Thefts 24

  25. @rieckpil Pitfall #1: Message Thefts 25

  26. @rieckpil Pitfall #2: Data Overflow 26

  27. @rieckpil Pitfall #2: Data Overflow - Avoid any test data

    leftovers between tests - Concepts to understand: - EntityManager flushing - Second-level vs. first-level cache - Transaction handling during tests - Ensure every test starts with a clean state - Provision the database via Testcontainers - Use schema migration tools like Flyway or Liquibase - LocalStack for S3 Buckets, etc. 27 💽
  28. @rieckpil Pitfall #3: Remote Connections 28

  29. @rieckpil Pitfall #3: Remote Connections 29 - Aim for “airplane”

    mode - Avoid any real HTTP communication to external services during the build - Predictable and reproducible test outcomes - WireMock to the rescue!
  30. @rieckpil Pitfall #3: Remote Connections 30

  31. @rieckpil Pitfall #3: Remote Connections 31

  32. @rieckpil Pitfall #3: Remote Connections 32 …

  33. @rieckpil Pitfall #4: @SpringBootTest Obsession 33

  34. @rieckpil Pitfall #4: @SpringBootTest Obsession 34 🙈

  35. @rieckpil Pitfall #4: @SpringBootTest Obsession 35

  36. @rieckpil Pitfall #4: @SpringBootTest Obsession 36

  37. @rieckpil Pitfall #5: Setup Variety 37

  38. @rieckpil Pitfall #5: Setup Variety - Group the config to

    the least amount of profiles needed, for example: integration-test and web-test - Avoid “unique snowflake” context setups - Don’t overuse @MockBean & @SpyBean - Avoid manual property overrides where possible - Understand what goes into the context cache key - (Custom) JUnit Jupiter extensions > extending an abstract test base class 38 🥗
  39. @rieckpil The Result - Three-day full-time investment - Reducing the

    build time from 26 minutes to 12 minutes - New integration tests are only adding a negligible amount of additional build time - Final context cache size: 3 (before: 20) - Internal knowledge sharing session about the identified pitfalls and applied techniques & patterns - Increased awareness for this topic when writing tests for Spring Boot applications 39 🥳
  40. @rieckpil Testing Spring Boot Applications Masterclass - Testing real-world Spring

    Boot applications - 130+ course lessons with 10h+ hands-on video content - Tech-Stack: Spring Boot 2.7, PostgreSQL, React 18, Java 17, Keycloak (OAuth 2 login), communication with a remote HTTP system, async workflows with a messaging queue (Amazon SQS) - Techniques, pitfalls, and patterns for unit, integration, and end-to-end testing - Best of breed testing tools: JUnit 5, Mockito, LocalStack, Testcontainers, Awaitility, WireMock, MockWebServer, Selenide, etc. - PS: Spring Test context caching is also covered in-depth - https://rieckpil.de/testing-spring-boot-applications-masterclass/ 40
  41. @rieckpil Stratospheric - From Zero to Production with Spring Boot

    and AWS - 450+ pages long hands-on ebook about getting your Spring Boot app running on AWS: Available on Leanpub - Java 17, Spring Boot 2.7 with Spring Cloud AWS - Fully automated CI/CD pipeline with GitHub Actions - AWS Services: ECS, ECR, SQS, CloudWatch, RDS, Cognito, MQ, etc. - Infrastructure as Code (written in Java) with the AWS CDK v2 - Complementary online course is currently in progress - More information (discount, mailing list, etc.): https://stratospheric.dev/ 41
  42. @rieckpil Conclusion - Think twice before implementing a workaround (aka.

    breaking the first window) - Treat testing code as production code and keep it maintainable and clean - Don’t religiously try to minimize the number of contexts: measure & adjust - Try to avoid using @DirtiesContext at a central place - Show some love for your test suite (your future self and coworkers will thank you) - Avoid having unique snowflakes aka. fruit salads when it comes to your test ApplicationContext setup - Composition over inheritance - Short feedback cycles ftw! 42 🏆
  43. @rieckpil Q&A Questions? Slides will be uploaded ✅ The best

    way to get in contact is via Twitter @rieckpil (DMs are open) ✅ Visit my blog and join the mailing list for more testing-related Spring Boot content ✅ Enjoy the rest of the conference ✅ 43