Workshop - Mocks as Code. Modeling services in Config Management integration tests

When managing service configurations at scale, we often need to automate testing of these configurations. In software integration testing we can use Testcontainers - a popular framework for Docker and other container engines. For public cloud environments, there is LocalStack. And for API service providers, we have WireMock. Can we use the same toolchain for configuration management? If you use modern Infrastructure-as-Code tools like Pulumi, yes!

In this edition, we will use Java or Kotlin for EVERYTHING, from the application itself and all the way up to Infrastructure as Code definitions and tests. The workshop also leverages capabilities offered by Gradle, especially for build automation and troubleshooting.

In this workshop, we will go through the following topics:
1. Containerizing Spring app development
2. Integration testing with Testcontainers and LocalStack
3. Mocking APIs with WireMock
4. Infrastructure as Code (IaC) with Pulumi and Pulumi for Java
5. Testing IaC locally, with all tools above combined
6. Advanced troubleshooting with Gradle and Gradle Build Scan

See https://github.com/oleg-nenashev/mocks-as-code-demo for the demo repository we will be using in this workshop. It also contains all the links to the used tools and additional learning materials.

Transcript

1. Oleg Nenashev, @oleg_nenashev CNCF Ambassador / Gradle Inc.
2. > whoami @oleg_nenashev oleg-nenashev Dr. Nenashev / Mr. Jenkins Community builder Developer Tools Hacker #RussiansAgainstPutin #StandWithUkraine
3. Async Q&A: Gradle Community Slack, Testcontainers Slack speakerdeck.com/onenashev
4. 4 Developer Productivity Engineering (DPE)
5. DPE is Like SRE, but for Developer Productivity 5
6. 6 gradle.com/ developer-productivity-engineering
7. Staging & QA Tests Build Integration tests Build Tests Gaining confidence in you software “Shift Left” with modern tools
8. 8 Localstack
9. “Shift Left”, they said? YAML, they gave? 9
10. 10 Java All the Way Down Up Application Unit Tests Build Definition and Pipelines Integration Tests Infrastructure-as-Code Deployment Pipelines Deployment Tests Production Health Checks Java the stack up all the way
11. Today 11 1. Containerizing Spring App Development 2. Integration testing with Testcontainers and LocalStack 3. Mocking APIs with WireMock 4. Infrastructure as Code (IaC) with Pulumi 5. Testing IaC locally 6. Advanced Troubleshooting Localstack
12. Our Repository github.com/oleg-nenashev/mocks-as-code-demo git clone it
13. Workshop Prerequisites • Internet :( • Java 17 & your favorite IDE • Testcontainers-compatible Docker environment ◦ Tested on: Docker Desktop ◦ You can use: Podman, Colima, Rancher Desktop (list) ◦ OR: Testcontainers Cloud • Pulumi CLI + registration ◦ Guide: www.pulumi.com/docs/install/ • Nice2Have: Gradle Build Scan account ◦ scans.gradle.com/ 13
14. Our Application 14
15. 15 AWS S3 SQS External API Our Application
16. Credits 16 • Siva Prasad Reddy and Oleg Shelajev, Docker • The demos are based on: ◦ github.com/testcontainers/tc-guide-testing-aws-service-integrations-using-loca lstack ◦ github.com/testcontainers-community/testcontainers-wiremock-demo
17. Integration Testing? 17 github.com/oleg-nenashev/mocks-as-code-demo/ tree/main/2_api_mocking
21. Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Modeling (aka Mocking) at the API provider level (e.g. WireMock or Microcks) 4. Mocking at the code level Slow Fast Nope Accu- rate * For Testcontainers, see the slides
22. 22 https://martinfowler.com/articles/ mocksArentStubs.html
23. 23 SDKs testcontainers.org … and
24. 24 github.com/testcontainers/ testcontainers-go
25. Example - Java 25
27. 27 LOCAL STACK LocalStack Module K3s Module
28. 28 Localstack www.localstack.cloud
29. LocalStack • Local AWS Services emulation OR mocking • “awslocal” CLI utility • SDKs for languages and test frameworks 29 docs.localstack.cloud/
30. 30 x.com/dastbe/status/1303858170155081728
31. Some of Them 31 • Emulated: S3, SQS, … • Mocked: EC2 (in the FOSS lib) • Thoughts & Prayers: AWS GreenShift ;) • …
32. Demo Time 32 github.com/oleg-nenashev/mocks-as-code-demo/ tree/main/1_application
33. No Implementation? 33 Fake it until you “$ make it”
34. Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Modeling (aka Mocking) at the API provider level (e.g. WireMock or Microcks) 4. Mocking at the code level Slow Fast Nope Accu- rate * For Testcontainers, see the slides
35. wiremock.org/docs/stubbing IF (request_url) THEN (response) * * it gets MUCH more complex WireMock Config JSON: Client library response request Client App Mock API Server HTTP/2 API Mocking 101
36. API Mocking Tools Main implementation in Java Native lib, Docker images and Testcontainers modules Support for GraalVM Rich Ecosystem Sandbox CNCF Project Golang / Microservice arch Docker image, Testcontainers module, All-in K8s Charts Less Rich Ecosystem
37. WireMock Ecosystem & Features wiremock.org/docs
38. WireMock Ecosystem wiremock.org/docs
39. WireMock in Java wiremock.org/docs/stubbing
40. WireMock Ecosystem / Java • Java • Kotlin • Scala • Groovy • Clojure • (?)Android • JUnit 4/5 • Testcontainers • Sprint Boot • Spring Cloud Contract • Quarkus • Maven/Gradle • Spock • Pact
41. Example - JUnit 5 41
42. WireMock Plugin for Gradle 42 William-Hill-Online/wiremock-gradle-plugin
43. 43 WireMock has an official Testcontainers module! testcontainers.com/modules/wiremock
44. Demo Time 44 github.com/oleg-nenashev/mocks-as-code-demo
45. 45 Infrastructure-as-Code, for real
47. 47 And more constructive!
48. HOWTO: JavaDev-Ops 48
49. Enter -as-Code Tools! • Infrastructure as Real Code • SDKs for programming languages • Standard IDEs • Standard Developer Tools • Extensibility via libraries 49
50. • Open-Ecosystem Tool for Infra-as-Code • Open Source SDKs 50
51. • Multiple destinations 51 www.pulumi.com
52. • 120+ extensions: www.pulumi.com/registry/ 52 www.pulumi.com
53. Pulumi for Java 53 pulumi.com/docs/languages-sdks/java/
54. build.gradle 54
55. 55 Java All the Way Down Up Application Unit Tests Build Definition and Pipelines Integration Tests Infrastructure-as-Code Deployment Pipelines Deployment Tests Production Health Checks Java the stack up all the way
56. 56 Localstack
57. Pulumi Local • github.com/localstack/pulumi-local • Setting the environment and the configuration right • Just a script 57 Localstack
60. Hacking Time! 60 github.com/oleg-nenashev/mocks-as-code-demo/ tree/main/3_deployment
61. When it Goes wrong 61
66. 66 gradle --scan Gradle Profiler Java Tools and IDEs Gradle Observability
67. 67 OpenTelemetry plugin for Gradle • Injecting traces and spans to the Gradle execution • Helps to find the slowest parts of your build with the Gradle OpenTelemetry build plugin. • Created by Craig Atkinson • craigatk/opentelemetry-gradle-plugin
68. 68 Gradle tasks tracing github.com/craigatk/ opentelemetry-gradle-plugin
69. Tests traces too! github.com/craigatk/opentelemetry-gradle-plugin
71. But Not Insights? 71
72. Based On True Events 72
73. Gradle Build Scan & Develocity
74. • For many open source projects: Gradle sponsors Develocity licenses/hosting, helps with build optimization and developer productivity • Maven, Bazel and sbt projects welcome gradle/develocity-oss-projects
75. works for Pulumi and Integration Tests Tool! 75
76. Demo Time 76 github.com/oleg-nenashev/mocks-as-code-demo
77. Lessons Learned 77
78. Beware Pulumi for AWS Native • AWS Native configuration is not applied for endpoints • APIs are slightly different in the package 78
83. 83 Java All the Way Down Up Application Unit Tests Build Definition and Pipelines Integration Tests Infrastructure-as-Code Deployment Pipelines Deployment Tests Production Health Checks Java the stack up all the way
84. Move to Kotlin? blog.gradle.org/kotlin-dsl-is-now-the-default -for-new-gradle-builds • 2023 - Kotlin is a default for Gradle build authoring • Scripts, Plugins
85. What I’m looking forward to? • Feedback from Java developers • Is Infra-as-Code with Java interesting to you? • Do you want Pulumi-LocalStack for Java productization? • Any other feedback on Gradle :)
86. References • Main Demo: github.com/oleg-nenashev/mocks-as-code-demo • Gradle Build Scan: scans.gradle.com • Credits: github.com/testcontainers/tc-guide-testing-rest-api-integrations-u sing-wiremock
87. Keep Learning Tools 87 * AKA: Burning Out 101
89. It’s [always] a great time to contribute! gradle.github.io/community/contributing testcontainers.org/contributing wiremock.org/participate 89
90. GitHub: oleg-nenashev Twitter: @oleg_nenashev Gradle Community Slack, #developer-productivity-engineering Questions? Testcontainers Slack 90