Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CNCF Linz meetup - Taming and Testing the Cloud Environment from localhost to CI

CNCF Linz meetup - Taming and Testing the Cloud Environment from localhost to CI

CNCF Meetup in Linz on the 21st of November 2023

Anca Ghenade

November 23, 2023
Tweet

More Decks by Anca Ghenade

Other Decks in Programming

Transcript

  1. Taming and Testing the
    Cloud Environment from
    localhost to CI
    21.11.2023
    Linz
    localstack.cloud

    View full-size slide

  2. Anca Ghenade
    - Developer Advocate -
    [email protected]
    @tinyg210

    View full-size slide

  3. How does one go around
    developing applications using AWS
    services?

    View full-size slide

  4. A Little Story About Cloud
    Development

    View full-size slide

  5. 1. The Software Ninjas are
    tasked with developing
    a new web application
    on AWS Cloud

    View full-size slide

  6. 2. Developing on
    a local machine,
    they soon find out
    that there are lots
    of dependencies
    with resources in
    the cloud

    View full-size slide

  7. 3. They realize that the dev &
    test loop is extremely slow
    and tedious. Every local
    change needs to be packaged
    and uploaded to the cloud for
    testing.

    View full-size slide

  8. 4. Now a Software Ninja
    has a red build on their
    feature branch, but can’t
    efficiently test and debug
    their code in the CI/CD
    pipeline.

    View full-size slide

  9. 5. The whole team is using Git flow for development - one CI
    build per feature branch. There is an explosion of different
    environments required for development
    (branches x developers).

    View full-size slide

  10. 6. The Ninja manager approaches the team
    and complains that AWS dev/test resources
    are not being cleaned up properly causing
    cost spikes.

    View full-size slide

  11. So things are kind of… complicated. 󰤆

    View full-size slide

  12. Title: Complexity and Risk
    - Setting up an AWS developer sandbox.
    - Making sure you have the right
    permissions to not blow up resources.
    - “Who is worthy of AWS access?”

    View full-size slide

  13. Title: The Fear
    That someone leaves resources on
    overnight…
    You get an email from your boss’ boss’
    boss the next morning.

    View full-size slide

  14. Title: Running out of Time
    - Your application is never fully tested before
    release.
    - AWS is slow to create and handle all those
    resources.

    View full-size slide

  15. Title: Wasted Time & Money
    Disaster recovery testing was not done
    thoroughly dues to costs. You think you’re
    safe, but you’re not.

    View full-size slide

  16. Title: Never growing
    It’s hard to learn something new when you
    don’t get your hands dirty.

    View full-size slide

  17. What about testing your AWS-powered
    applications?

    View full-size slide

  18. How much of your
    system you can test
    Cloud emulation
    Full cloud emulation with
    service integration
    Staging environments
    Highest fidelity because it
    actually runs in the cloud
    Mocking
    Mock out cloud APIs
    What are our options in terms of testability?
    3
    4
    Service emulation
    Replace individual services
    with local versions
    2
    1

    View full-size slide

  19. What is LocalStack and how
    can it help you?
    A fully functional local cloud stack that was born out of these exact concerns.

    View full-size slide

  20. ● Enables a highly efficient dev & test loop for cloud apps
    ● Ships as a Docker image, easy to install and start up
    ● Support for 90-ish services (and growing):
    ○ compute (Lambda, ECS, EKS)
    ○ various databases (DynamoDB, RDS)
    ○ messaging (SQS, Kinesis, MSK)
    ○ some sophisticated/exotic APIs (Athena, Glue)
    ● CI integrations & advanced collaboration features
    ● Branching out into other areas: Chaos Engineering, IAM
    Security Testing, Cloud Ephemeral Environments, 3rd Party
    Extensions, etc

    View full-size slide

  21. https://github.com/localstack-samples/sample-shipment-list-demo-lambda-dynamodb-s3

    View full-size slide

  22. Using Spring Profiles…

    View full-size slide

  23. Scenario 1 (Production) Scenario 2 (Development)

    View full-size slide

  24. “Your application won’t even know the difference”

    View full-size slide

  25. What about CI/CD pipelines?

    View full-size slide

  26. ● Automated Testing
    ● Continuous Deployment
    ● Integration Testing
    ● Code Quality Analysis
    ● Infrastructure as Code (IaC) Validation
    ● Containerization and Orchestration
    ● Feature Branch Testing
    ● Rollbacks and Blue/Green Deployments
    ● Performance Testing and Scalability
    ● Automated Documentation Generation

    View full-size slide

  27. ● Automated Testing
    ● Continuous Deployment
    ● Integration Testing
    ● Code Quality Analysis
    ● Infrastructure as Code (IaC) Validation
    ● Containerization and Orchestration
    ● Feature Branch Testing
    ● Rollbacks and Blue/Green Deployments
    ● Performance Testing and Scalability
    ● Automated Documentation Generation

    View full-size slide

  28. Some things cannot just easily be tested, you
    need a certain amount of reality in order to do
    that.

    View full-size slide

  29. And if we mock everything for testing, aren’t
    we just testing the mock itself?

    View full-size slide

  30. What if there’s a breaking change in the latest
    version of our mocked dependency?
    The mock doesn’t care, it will just do what you
    tell it to do.

    View full-size slide

  31. And reality is also expensive, so we also want
    to test and do CI on a budget.

    View full-size slide

  32. I’ll show you some stuff

    View full-size slide

  33. Source (Testcontainers):
    https://github.com/tinyg210/testcontainers-shi
    pment-list-demo

    View full-size slide

  34. 1. Testing your Terraform configuration
    file in CI

    View full-size slide

  35. Scenario 1 Scenario 2

    View full-size slide

  36. https://github.com/localstack-samples/sa
    mple-shipment-list-demo-lambda-dynamo
    db-s3/actions/runs/6917423229/workflow

    View full-size slide

  37. 2. Testing your application workflow

    View full-size slide

  38. Configuring your container
    *Resources are created programmatically in the @BeforeAll method

    View full-size slide

  39. 3.? Bonus: Setting up your
    infrastructure blazingly fast

    View full-size slide

  40. Cloud Pods
    Persistent Shareable
    Sandboxes
    Cloud Pods are a mechanism
    that allows you to take a
    snapshot of the state in your
    current LocalStack instance,
    persist it to a storage backend,
    and easily share it with your
    team members.

    View full-size slide

  41. We’re giving the OPS back to the DEVS.

    View full-size slide

  42. Questions…

    View full-size slide

  43. Just FYI
    - Top 3 podium
    - Game of correctness
    and agility
    - You can play as many
    times as you want

    View full-size slide