CNCF Linz meetup - Taming and Testing the Cloud Environment from localhost to CI

CNCF Meetup in Linz on the 21st of November 2023

Transcript

1. Taming and Testing the Cloud Environment from localhost to CI 21.11.2023 Linz localstack.cloud
2. Anca Ghenade - Developer Advocate - anca.ghenade@localstack.cloud @tinyg210
3. How does one go around developing applications using AWS services?
4. A Little Story About Cloud Development
5. 1. The Software Ninjas are tasked with developing a new web application on AWS Cloud
6. 2. Developing on a local machine, they soon find out that there are lots of dependencies with resources in the cloud
7. 3. They realize that the dev & test loop is extremely slow and tedious. Every local change needs to be packaged and uploaded to the cloud for testing.
8. 4. Now a Software Ninja has a red build on their feature branch, but can’t efficiently test and debug their code in the CI/CD pipeline.
9. 5. The whole team is using Git flow for development - one CI build per feature branch. There is an explosion of different environments required for development (branches x developers).
10. 6. The Ninja manager approaches the team and complains that AWS dev/test resources are not being cleaned up properly causing cost spikes.
11. So things are kind of… complicated. 󰤆
12. HIDDEN
13. Title: Complexity and Risk - Setting up an AWS developer sandbox. - Making sure you have the right permissions to not blow up resources. - “Who is worthy of AWS access?”
14. Title: The Fear That someone leaves resources on overnight… You get an email from your boss’ boss’ boss the next morning.
16. Title: Running out of Time - Your application is never fully tested before release. - AWS is slow to create and handle all those resources.
18. Title: Wasted Time & Money Disaster recovery testing was not done thoroughly dues to costs. You think you’re safe, but you’re not.
19. Title: Never growing It’s hard to learn something new when you don’t get your hands dirty.
20. What about testing your AWS-powered applications?
21. How much of your system you can test Cloud emulation Full cloud emulation with service integration Staging environments Highest fidelity because it actually runs in the cloud Mocking Mock out cloud APIs What are our options in terms of testability? 3 4 Service emulation Replace individual services with local versions 2 1
22. What is LocalStack and how can it help you? A fully functional local cloud stack that was born out of these exact concerns.
23. • Enables a highly efficient dev & test loop for cloud apps • Ships as a Docker image, easy to install and start up • Support for 90-ish services (and growing): ◦ compute (Lambda, ECS, EKS) ◦ various databases (DynamoDB, RDS) ◦ messaging (SQS, Kinesis, MSK) ◦ some sophisticated/exotic APIs (Athena, Glue) • CI integrations & advanced collaboration features • Branching out into other areas: Chaos Engineering, IAM Security Testing, Cloud Ephemeral Environments, 3rd Party Extensions, etc
25. https://github.com/localstack-samples/sample-shipment-list-demo-lambda-dynamodb-s3
27. Using Spring Profiles…
28. Scenario 1 (Production) Scenario 2 (Development)
29. “Your application won’t even know the difference”
30. What about CI/CD pipelines?
31. • Automated Testing • Continuous Deployment • Integration Testing • Code Quality Analysis • Infrastructure as Code (IaC) Validation • Containerization and Orchestration • Feature Branch Testing • Rollbacks and Blue/Green Deployments • Performance Testing and Scalability • Automated Documentation Generation
32. • Automated Testing • Continuous Deployment • Integration Testing • Code Quality Analysis • Infrastructure as Code (IaC) Validation • Containerization and Orchestration • Feature Branch Testing • Rollbacks and Blue/Green Deployments • Performance Testing and Scalability • Automated Documentation Generation
33. Some things cannot just easily be tested, you need a certain amount of reality in order to do that.
34. And if we mock everything for testing, aren’t we just testing the mock itself?
35. What if there’s a breaking change in the latest version of our mocked dependency? The mock doesn’t care, it will just do what you tell it to do.
36. And reality is also expensive, so we also want to test and do CI on a budget.
38. I’ll show you some stuff
39. Source (Testcontainers): https://github.com/tinyg210/testcontainers-shi pment-list-demo
40. 1. Testing your Terraform configuration file in CI
41. Scenario 1 Scenario 2
42. https://github.com/localstack-samples/sa mple-shipment-list-demo-lambda-dynamo db-s3/actions/runs/6917423229/workflow
43. 2. Testing your application workflow
47. Configuring your container *Resources are created programmatically in the @BeforeAll method
48. 1/2
49. 2/2
52. 3.? Bonus: Setting up your infrastructure blazingly fast
53. Cloud Pods Persistent Shareable Sandboxes Cloud Pods are a mechanism that allows you to take a snapshot of the state in your current LocalStack instance, persist it to a storage backend, and easily share it with your team members.
54. We’re giving the OPS back to the DEVS.
55. Thank You
56. Questions…
57. …for you
58. Just FYI - Top 3 podium - Game of correctness and agility - You can play as many times as you want