Migrating Picnic to Spring Boot 3 at Scale

This is not your average version bump: it not only upgrades the very foundation web-apps are built upon, but also transitively upgrades many other libraries with major versions. Get a sneak-peak on how we structure our projects at Picnic, and how we perform the majority
of the migration only once to have it benefit all Picnic services. We shared our process, the blessings and curses of our setup, the challenges we faced and how we overcame them. See how we did it. Schadenfreude guaranteed!

Transcript

1. 1 | 07 November 2018 Migrating to Spring Boot 3 How we did it Pieter Dirk Soels 2 3 Java MeetUp @ Picnic Technologies 2023-09-07
2. 2 | 07 November 2018 Why upgrade? https://spring.io/projects/spring-boot#support What’s all the fuss about? ▪ Java 17 baseline ▪ Jakarta EE 9 ▪ New Spring Micrometer library ▪ AOT and native images ▪ For us, most important at all: ◦ Spring Boot 2.7 OSS support EOL: 18 November 2023 ◦ Unblocking many other dependency upgrades too (Jetty, Reactor, Logback, jOOQ, JAXB, …)
3. 3 | 07 November 2018 It’s just a version bump, right? How hard can it be?
4. 4 | 07 November 2018 Hmm.. it can’t run a test. Let’s run a build!
5. 5 | 07 November 2018 Doh! Of course. Let’s run a build! Package org.springframework.aot Class AotDetector java.lang.Object org.springframework.aot.AotDetector public abstract class AotDetector extends Object Utility for determining if AOT-processed optimizations must be used rather than the regular runtime. Strictly for internal use within the framework. Since: 6.0
6. 6 | 07 November 2018 Ye might be warned Let’s run a build! [INFO] --- maven-enforcer-plugin :3.3.0:enforce (apply-enforcement-rules) @ config-support --- [...] [WARNING] Rule 12: org.apache.maven.enforcer.rules.dependency.RequireUpperBoundDeps failed with message: [...] Require upper bound dependencies error for org.springframework:spring-core:5.3.29 paths to dependency are: [...] +-tech.picnic.shared:config-support:0.0.0-SNAPSHOT +-org.springframework.boot: spring-boot:3.0.0 +-org.springframework: spring-core:5.3.29 (managed) <-- org.springframework:spring-core:6.0.2 [...]
7. 7 | 07 November 2018 And try again… So let’s bump that one too!
8. 8 | 07 November 2018 Sounds familiar... Uhhm what now? private static Validator getValidator() { LocalValidatorFactoryBean factoryBean = new LocalValidatorFactoryBean(); factoryBean.afterPropertiesSet(); return factoryBean.getValidator(); } java: cannot access jakarta.validation.ValidatorFactory class file for jakarta.validation.ValidatorFactory not found
9. 9 | 07 November 2018 There are guides for a reason Okay let’s be fair.. ▪ Spring Boot 3.0 Migration Guide ▪ Upgrading to Spring Framework 6.x ▪ Preparing for 6.0 :: Spring Security ▪ Migrating to 6.0 :: Spring Security
10. 10 | 07 November 2018 Picnic’s Maven and internal library setup Migration process Challenges, blessings and learnings What’s next? Spring Boot 3 Migration at Picnic
11. 11 | 07 November 2018 One POM to rule them all
12. 12 | 07 November 2018 One POM to rule them all
13. 13 | 07 November 2018 One POM to rule them all
14. 14 | 07 November 2018 One POM to rule them all
15. 15 | 07 November 2018 Picnic’s Maven and internal library setup Migration process Challenges, blessings and learnings What’s next? Spring Boot 3 Migration at Picnic
16. 16 | 07 November 2018 Step 0: Pre-upgrade steps Migration process ▪ Adopt JDK 17 ▪ Resolve suppressed deprecated API usages – Oops! ◦ AsyncRestTemplate -> WebClient migration 40 clients!
17. 17 | 07 November 2018 Using Error Prone – an example. Let’s automate the migration import java.time.Duration ; public final class DeadlineHandler { // field, ctor and getter omitted for brevity public void setDeadline(Duration deadline) { this.deadline = deadline; } @Deprecated public void setDeadline(long millis) { setDeadline( Duration.ofMillis(millis)); } }
18. 18 | 07 November 2018 Using Error Prone – an example. Let’s automate the migration import com.google.errorprone.annotations. InlineMe; import java.time.Duration ; public final class DeadlineHandler { // field, ctor and getter omitted for brevity public void setDeadline(Duration deadline) { this.deadline = deadline; } @Deprecated @InlineMe( replacement = "this.setDeadline(Duration.ofMillis(millis))" , imports = { "java.time.Duration" }) public void setDeadline(long millis) { setDeadline( Duration.ofMillis(millis)); } }
19. 19 | 07 November 2018 Using Error Prone – an example. Let’s automate the migration // no imports public class Task { private final DeadlineHandler deadlineHandler = new DeadlineHandler(); public void procrastinate() { // Tomorrow is also fine. deadlineHandler.setDeadline(System.currentTimeMillis() + 86400000); } } [WARNING] /[path]/Task.java: [8,32] [InlineMeInliner] Migrate (via inlining) away from deprecated `DeadlineHandler.setDeadline()`. (see https://errorprone.info/bugpattern/InlineMeInliner ) Did you mean 'deadlineHandler.setDeadline(Duration.ofMillis( System.currentTimeMillis() + 86400000));'?
20. 20 | 07 November 2018 Using Error Prone – an example. Let’s automate the migration -XepPatchChecks:InlineMeInliner -XepPatchLocation:IN_PLACE
21. 21 | 07 November 2018 Our AsyncRestTemplate usage AsyncRestTemplate migration public abstract class AbstractRxHttpClient { private final String baseUrl; private final AsyncRestTemplate asyncRestTemplate; protected AbstractRxHttpClient ( AsyncRestTemplate asyncRestTemplate, String baseUrl) { this.baseUrl = baseUrl; this.asyncRestTemplate = asyncRestTemplate; }
22. 22 | 07 November 2018 Our AsyncRestTemplate usage AsyncRestTemplate migration protected final <T> Observable<T> getObservable( String path, Class<T> responseType, Object... uriVariables) { return observable(send(GET, path, EMPTY, ref(responseType), uriVariables)); }
23. 23 | 07 November 2018 Do the same, but then with WebClient AsyncRestTemplate migration public abstract class AbstractWebClient { private final WebClient webClient; protected AbstractWebClient(WebClient.Builder builder, String baseUrl) { this.webClient = builder.baseUrl(baseUrl).build(); } public abstract class AbstractRxHttpClient { private final String baseUrl; private final AsyncRestTemplate asyncRestTemplate; protected AbstractRxHttpClient (AsyncRestTemplate asyncRestTemplate, String baseUrl) { this.baseUrl = baseUrl; this.asyncRestTemplate = asyncRestTemplate; }
24. 24 | 07 November 2018 And implement the same API AsyncRestTemplate migration protected final <T> Observable<T> getObservable( String path, Class<T> responseType, Object... uriVariables) { return getWebClient() .get() .uri(path, uriVariables) .retrieve() .bodyToMono(responseType) .flux() .as(RxJava2Adapter::fluxToObservable); } protected final <T> Observable<T> getObservable( String path, Class<T> responseType, Object... uriVariables) { return observable(send(GET, path, EMPTY, ref(responseType), uriVariables)); } io.projectreactor.addons: reactor-adapter
25. 25 | 07 November 2018 And let Error Prone do its inlining magic! AsyncRestTemplate migration @InlineMe( replacement = "this.getWebClient().get().uri(path, uriVariables).retrieve().bodyToMono(responseType).flux().as(RxJava2Adapter::fluxTo Observable)", imports = "reactor.adapter.rxjava.RxJava2Adapter" ) @Deprecated protected final <T> Observable<T> getObservable( String path, Class<T> responseType, Object... uriVariables) { AbstractWebClient
26. 26 | 07 November 2018 Example client AsyncRestTemplate migration public class MyClient extends AbstractRxHttpClient { MyClient( AsyncRestTemplate asyncRestTemplate, @Value("${my.url}") String baseUrl) { super(asyncRestTemplate, baseUrl); } public Observable<String> getFoo() { return getObservable("/foo", String.class); } }
27. 27 | 07 November 2018 Migrating the constructor (manual) AsyncRestTemplate migration
28. 28 | 07 November 2018 Boom! Migrated. AsyncRestTemplate migration
29. 29 | 07 November 2018 Step 0: Pre-upgrade steps Migration process ▪ Adopt JDK 17 ▪ Resolve suppressed deprecated API usage – Oops! ◦ AsyncRestTemplate -> WebClient migration ▪ Adopt latest releases in Spring Boot 2, Spring Framework/Security 5 ◦ Deprecation of @EnableGlobalMethodSecurity in 5.8 ▪ Check guides/release notes for what can be done beforehand ◦ Dropped support for RxJava 2: Going Reactor instead
30. 30 | 07 November 2018 Step 1: Bump versions and fix the build in Parent Migration process ▪ Keep scope minimal ▪ If possible, stick with previous behavior ▪ Use automation where possible ▪ Split changes by dependency ▪ Don’t jump to latest
31. 31 | 07 November 2018 Step 1: What did we have to upgrade? Migration process A LOT Spring Security 5.8.5 -> 6.0.5 Spring Framework 5.3.29 -> 6.0.11 Logback 1.2.12 -> 1.4.9 Jakarta Validation 2.0.2 -> 3.0.2 DGS 5.5.3 -> 6.0.0 jOOQ 3.15.12 -> 3.17.14 Springdoc 1.7.0 -> 2.0.4 Jetty 9.4.51 -> 11.0.15 Jakarta Annotations 1.3.5 -> 2.1.1 Jakarta XML Web Services 2.3.6 -> 4.0.1 JAXB 2.3.8 -> 4.0.3 Apache HTTP 4.5.14 -> 5.2.1 Jakarta Servlet 4.0.4 -> 5.0.0 Hibernate Validator 6.2.5 -> 7.0.5 Reactor Core 3.4.31 -> 3.5.9 FlywayDB 8.5.13 -> 9.0.4 And various small libraries. SLF4J 1.7.36 -> 2.0.4 Jakarta Activation 1.2.2 -> 2.1.2 Tomcat EL 9.0.78 -> 10.0.27 Spring Vault 2.3.3 -> 3.0.4 Spring Web Services 3.1.6 -> 4.0.5 Spring Boot 2.7.14 -> 3.0.9
32. 32 | 07 November 2018 Step 1: Automation using OpenRewrite https://docs.openrewrite.org/recipes/java/spring/boot3/upgradespringbo ot_3_0 Migration process And more…
33. 33 | 07 November 2018 Step 1: Automation using OpenRewrite OpenRewrite recipe Migration process
34. 34 | 07 November 2018 Step 2: Resolve must-have tickets Migration process ▪ Apache client auth scheme & timeout changes ▪ JAXB 4.0 class generation ▪ Multi-marker Logback support ▪ Reintroduce Reactor Scheduler metrics ▪ Actuator endpoint sanitization of properties ▪ Make Micrometer 1.11 compatible with jOOQ 3.17 ▪ Many small tweaks and deprecations
35. 35 | 07 November 2018 Step 2: Resolve must-have tickets Migration process
36. 36 | 07 November 2018 Step 3: Test locally on projects Migration process ▪ Build a snapshot version of the Picnic parent POM ▪ Upgrade and migrate services ◦ Use automation where possible (OpenRewrite) ▪ Blackbox testing ▪ Make changes to Picnic parent where needed
37. 37 | 07 November 2018 ▪ Select batch of services ◦ Varying: teams, WebFlux/WebMVC, library usages, size ▪ Run in development environment ▪ All good? Release and deploy to production! ▪ Next batch & repeat Step 4: Getting it out there! Migration process
38. 38 | 07 November 2018 Visualized Migration process
39. 39 | 07 November 2018 Picnic’s Maven and internal library setup Migration process Challenges, blessings and learnings What’s next? Spring Boot 3 Migration at Picnic
40. 40 | 07 November 2018 Jakarta Servlet 5 or 6? Here we go… 🍿
41. 41 | 07 November 2018 Sure let’s do it! Here we go… 🍿
42. 42 | 07 November 2018 Hmm.. it can’t run a test. Let’s run a build! (Using Spring’s MockMvc)
43. 43 | 07 November 2018 https://github.com/spring-projects/spring-boot/issues/33044 Looks like we’re not the only one… “Framework's Servlet-related mocks require Servlet 6.0 and will fail due to the absence of ServletConnection when run against the Servlet 5.0 API. Similarly, Jetty requires Servlet 5.0 and will fail due to the absence of HttpSessionContext when run against the Servlet 6.0 API.”
44. 44 | 07 November 2018 https://github.com/spring-projects/spring-boot/issues/33044 Looks like we’re not the only one… “Framework's Servlet-related mocks require Servlet 6.0 and will fail due to the absence of ServletConnection when run against the Servlet 5.0 API. Similarly, Jetty requires Servlet 5.0 and will fail due to the absence of HttpSessionContext when run against the Servlet 6.0 API.” Our workaround: ▪ Copy ServletConnection into our testing library. ▪ Maven enforcer plugin to identify duplicate classes. ▪ Revert when we upgrade to Jetty 12 and Jakarta Servlet 6.0.
45. 45 | 07 November 2018 Accumulating errors, right? Upgrading Reactor… 🍿 public Mono<Void> processSomeStuff(SomeStuff someStuff) { return someParser .parse(someStuff) .concatMapDelayError(service::report) .onErrorMap(t -> tryHandleCompositeException(t).orElse(t)) .then(); }
46. 46 | 07 November 2018 NOPE Upgrading Reactor… 🍿 java.lang.AssertionError: expectation "expectError(Class)" failed ( expected error of type: CompositeIllegalArgumentException; actual type: java.lang.IllegalArgumentException: client-error)
47. 47 | 07 November 2018 But… the API is not lying right? Upgrading Reactor… 🍿 Errors in the individual publishers will be delayed at the end of the whole concat sequence (possibly getting combined into a composite) if several sources error. Changed in 3.5.0: Have concatMap default to 0 prefetch behavior
48. 48 | 07 November 2018 Ah… Upgrading Reactor… 🍿
49. 49 | 07 November 2018 Just staring at a screen… Oh this one.. Oh this one.. Oh this one..🍿 <?xml version="1.0" encoding="UTF-8"?> <configuration> <include resource="org/springframework/boot/logging/logback/defaults.xml" /> <conversionRule conversionWord="callback" converterClass="tech.picnic.demo.CallbackConverter" /> <property name="CONSOLE_LOG_PATTERN" value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr([%15.15t]){faint} %clr(%.40logger{0}){cyan} \\(%mdc\\) %clr(:){faint} % callback(%m){}%n ${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}" /> <include resource="org/springframework/boot/logging/logback/console-appender.xml" /> <root level="INFO"> <appender-ref ref="CONSOLE"/> </root> </configuration>
50. 50 | 07 November 2018 Logback upgrade… Oh this one.. Oh this one.. Oh this one..🍿 @Test void markerCalled() { BiFunction<ILoggingEvent, String, String> callback = mock(); LOG.info(CallbackMarker.from(callback), "Hello world"); verify(callback).apply(any(ILoggingEvent.class), eq("Hello world")); } 2023-09-01 08:51:26.036 [main] CallbackConverterTest : Hello world 2023-09-01 08:55:43.129 [main] CallbackConverterTest \(\2023-09-01 08:55:43.129 [main] CallbackConverterTest \(\2023-09-01 08:55:43.184 [main] CallbackConverterTest \(\ 2023-09-01 08:55:43.835 [main] CallbackConverterTest \(\ Before: After:
51. 51 | 07 November 2018 Hmmm… Oh this one.. Oh this one.. Oh this one..🍿 <property name="CONSOLE_LOG_PATTERN" value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr([%15.15t]){faint} %clr(%.40logger{0}){cyan} \\(%mdc\\) %clr(:){faint} %callback(%m){}%n ${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}" /> 2022-08-09 Release of logback version 1.3.0-beta0 • Escape sequences are no longer interpreted for the attribute named value defined in the <variable> element in configuration files. By escape sequences we mean the escape sequences for Java String literals such as '\t' or '\n'.
52. 52 | 07 November 2018 Blessings and curses Learnings ▪ One place to do the majority of changes using the Picnic Parent ◦ Complicates the effort ◦ Provides a multiplication factor ▪ ~30K unit tests and 4K blackbox tests ▪ Love / Hate for Maven enforcer plugin ▪ OpenRewrite and Error Prone for automation ◦ Future: self-service at scale ▪ The devil is in the details
53. 53 | 07 November 2018 Picnic’s Maven and internal library setup Migration process Challenges and learnings What’s next? Spring Boot 3 Migration at Picnic
54. 54 | 07 November 2018 Well.. https://spring.io/projects/spring-boot#support What’s next? ▪ Spring Boot 2.7 OSS support EOL: 18 November 2023 ▪ Spring Boot 3.0 OSS support EOL: 24 November 2023 ▪ Continue rollout ◦ 27/58 services migrated ▪ Look at (more) dependency upgrades ▪ See what the fuss is about native images ▪ Prepare for JDK 21 (LTS): 19 September 2023
55. 55 | 07 November 2018 Lexicographically ordered, how we like it. The fellowship of the Java platform team Jarmila Kaiser Oksana Evseeva Pieter Dirk Soels Luca Hennart Nathan Kooij Rick Ossendrijver
56. 56 | Confidential | 07 November 2018 Building the best milkman on earth serving millions of families Climbing a mountain One step at a time