CNCF Linz meetup - Taming and Testing the Cloud Environment from localhost to CI

Slide 1

Slide 1 text

Taming and Testing the Cloud Environment from localhost to CI 21.11.2023 Linz localstack.cloud

Slide 2

Slide 2 text

Anca Ghenade - Developer Advocate - [email protected] @tinyg210

Slide 3

Slide 3 text

How does one go around developing applications using AWS services?

Slide 4

Slide 4 text

A Little Story About Cloud Development

Slide 5

Slide 5 text

1. The Software Ninjas are tasked with developing a new web application on AWS Cloud

Slide 6

Slide 6 text

2. Developing on a local machine, they soon ﬁnd out that there are lots of dependencies with resources in the cloud

Slide 7

Slide 7 text

3. They realize that the dev & test loop is extremely slow and tedious. Every local change needs to be packaged and uploaded to the cloud for testing.

Slide 8

Slide 8 text

4. Now a Software Ninja has a red build on their feature branch, but can’t efficiently test and debug their code in the CI/CD pipeline.

Slide 9

Slide 9 text

5. The whole team is using Git ﬂow for development - one CI build per feature branch. There is an explosion of different environments required for development (branches x developers).

Slide 10

Slide 10 text

6. The Ninja manager approaches the team and complains that AWS dev/test resources are not being cleaned up properly causing cost spikes.

Slide 11

Slide 11 text

So things are kind of… complicated. 󰤆

Slide 12

Slide 12 text

HIDDEN

Slide 13

Slide 13 text

Title: Complexity and Risk - Setting up an AWS developer sandbox. - Making sure you have the right permissions to not blow up resources. - “Who is worthy of AWS access?”

Slide 14

Slide 14 text

Title: The Fear That someone leaves resources on overnight… You get an email from your boss’ boss’ boss the next morning.

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Title: Running out of Time - Your application is never fully tested before release. - AWS is slow to create and handle all those resources.

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Title: Wasted Time & Money Disaster recovery testing was not done thoroughly dues to costs. You think you’re safe, but you’re not.

Slide 19

Slide 19 text

Title: Never growing It’s hard to learn something new when you don’t get your hands dirty.

Slide 20

Slide 20 text

What about testing your AWS-powered applications?

Slide 21

Slide 21 text

How much of your system you can test Cloud emulation Full cloud emulation with service integration Staging environments Highest ﬁdelity because it actually runs in the cloud Mocking Mock out cloud APIs What are our options in terms of testability? 3 4 Service emulation Replace individual services with local versions 2 1

Slide 22

Slide 22 text

What is LocalStack and how can it help you? A fully functional local cloud stack that was born out of these exact concerns.

Slide 23

Slide 23 text

● Enables a highly eﬃcient dev & test loop for cloud apps ● Ships as a Docker image, easy to install and start up ● Support for 90-ish services (and growing): ○ compute (Lambda, ECS, EKS) ○ various databases (DynamoDB, RDS) ○ messaging (SQS, Kinesis, MSK) ○ some sophisticated/exotic APIs (Athena, Glue) ● CI integrations & advanced collaboration features ● Branching out into other areas: Chaos Engineering, IAM Security Testing, Cloud Ephemeral Environments, 3rd Party Extensions, etc

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

https://github.com/localstack-samples/sample-shipment-list-demo-lambda-dynamodb-s3

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Using Spring Proﬁles…

Slide 28

Slide 28 text

Scenario 1 (Production) Scenario 2 (Development)

Slide 29

Slide 29 text

“Your application won’t even know the difference”

Slide 30

Slide 30 text

What about CI/CD pipelines?

Slide 31

Slide 31 text

● Automated Testing ● Continuous Deployment ● Integration Testing ● Code Quality Analysis ● Infrastructure as Code (IaC) Validation ● Containerization and Orchestration ● Feature Branch Testing ● Rollbacks and Blue/Green Deployments ● Performance Testing and Scalability ● Automated Documentation Generation

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Some things cannot just easily be tested, you need a certain amount of reality in order to do that.

Slide 34

Slide 34 text

And if we mock everything for testing, aren’t we just testing the mock itself?

Slide 35

Slide 35 text

What if there’s a breaking change in the latest version of our mocked dependency? The mock doesn’t care, it will just do what you tell it to do.

Slide 36

Slide 36 text

And reality is also expensive, so we also want to test and do CI on a budget.

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

I’ll show you some stuff

Slide 39

Slide 39 text

Source (Testcontainers): https://github.com/tinyg210/testcontainers-shi pment-list-demo

Slide 40

Slide 40 text

1. Testing your Terraform conﬁguration ﬁle in CI

Slide 41

Slide 41 text

Scenario 1 Scenario 2

Slide 42

Slide 42 text

https://github.com/localstack-samples/sa mple-shipment-list-demo-lambda-dynamo db-s3/actions/runs/6917423229/workﬂow

Slide 43

Slide 43 text

2. Testing your application workﬂow

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

Conﬁguring your container *Resources are created programmatically in the @BeforeAll method

Slide 48

Slide 48 text

1/2

Slide 49

Slide 49 text

2/2

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

3.? Bonus: Setting up your infrastructure blazingly fast

Slide 53

Slide 53 text

Cloud Pods Persistent Shareable Sandboxes Cloud Pods are a mechanism that allows you to take a snapshot of the state in your current LocalStack instance, persist it to a storage backend, and easily share it with your team members.

Slide 54

Slide 54 text

We’re giving the OPS back to the DEVS.

Slide 55

Slide 55 text

Thank You

Slide 56

Slide 56 text

Questions…

Slide 57

Slide 57 text

…for you

Slide 58

Slide 58 text

Just FYI - Top 3 podium - Game of correctness and agility - You can play as many times as you want