Accelerating Your AWS Journey: (Open Source) Tools for getting teams up to speed

Talk given at the AWS Community Day in NL on September 20th 2023.

Embarking on an AWS journey can be both thrilling and overwhelming for teams. There are a lot of concepts to grasp when starting out with AWS. In this talk I’ll share some of the open source tools and libraries I’ve used over the last years while trying get enterprise teams up to speed and flatten the team’s learning curve. These tools help teams benefit from battle-tested solutions, invaluable insights, and time-saving best practices. Tools involved will help with applying best practices, hardening security and compliance, and reduce boilerplate code. After this talks you should have some additional tools on your belt that can help new and existing teams improve their efficiency.

Transcript

1. Accelerating Your AWS Journey: (Open-Source) tools for getting teams up to speed Jeroen Reijn Cloud Solutions Architect @Luminis| AWS Community Builder Twitter/X: @jreijn 🚀
2. Photo by Annie Spratt on Unsplash Working with Dev(Ops) Teams
4. What they hope it’s like What it’s actually like….
5. Learning AWS • AWS Courses / Certification • AWS Workshops • Game Days • Documentation • Tutorials • Blogs • Etc… Learn by doing.
6. Personal Observations • Design: What service(s) should I use to run my application? • Infrastructure as Code • Networking – VPCs, Security Groups, etc. • IAM (Identity & Access Management) • Local development
7. Source: https://www.lastweekinaws.com/blog/the-17-ways-to-run-containers-on-aws/ 🤯
8. 🤔
9. Image by Sasha Kasper: https://sascha-kasper.com/the-bumpy-learning-curve/
11. Engineering productivity
12. Infrastructure as Code
13. Reasoning: • Language Flexibility: TypeScript, Python, Java, C#, etc • Type Safety • High-Level Abstractions • Customization and Reusability • Rich Ecosystem of Constructs • Integrated Deployment • AWS CDK Patterns
14. AWS CDK + Higher level abstractions Source: https://konem.blog/2021/01/17/how-to-structure-your-application-architecture-using-aws-cdk/
16. AWS CDK + AWS Solution Constructs “AWS Solutions Constructs is an open-source extension on top of AWS CDK that provides multi-service, well-architected patterns for quickly defining solutions in code to create predictable and repeatable infrastructure. The goal is to accelerate the experience for developers to build solutions of any size using pattern-based definitions for their architecture.” https://github.com/awslabs/aws-solutions-constructs
17. AWS CDK + AWS Solution Constructs
18. Testcontainers + = 🚀
19. • On-demand isolated infrastructure provisioning • Consistent experience on both local and CI environments • Reliable test setup using wait strategies • Automatic clean up • Support for Java, Python, NodeJs, Go, etc. Test containers:
20. • Local cloud sandbox for development, testing, and experimentation • Emulates 60+ AWS cloud services • Get productive with cloud dev from day 1 • Avoid the slow and tedious deploy-test-redeploy cycles for your cloud based apps LocalStack
24. Amazon CodeWhisperer • AI Coding assistant • (Near) Real-time code suggestions • Optimized for use with AWS services • Support for popular programming languages and IDEs
26. OpenAI / ChatGPT 🤖
27. Security & Compliance
28. Shift Left Shift Left
29. CDK-NAG • Checks CDK applications for best practices using a combination of available rule packs • AWS Solutions • HIPAA Security • NIST 800-53 rev 4 • NIST 800-53 rev 5 • PCI DSS 3.2.1 • Inspired by https://github.com/stelligent/cfn_nag • https://github.com/cdklabs/cdk-nag
32. CDK-NAG output with AwsSolutionsChecks
33. CDK-NAG output with AwsSolutionsChecks [Error at /AwsCdkNagDemoStack/demo-queue/Resource] AwsSolutions- SQS4: The SQS queue does not require requests to use SSL. Without HTTPS (TLS), a network-based attacker can eavesdrop on network traffic or manipulate it, using an attack such as man-in-the-middle. Allow only encrypted connections over HTTPS (TLS) using the aws:SecureTransport condition in the queue policy to force requests to use SSL.
34. CDK-NAG output with HIPAASecurityChecks
35. CDK-NAG output with HIPAASecurityChecks [Error at /AwsCdkNagDemoStack/demo-function/Resource] HIPAA.Security- LambdaConcurrency: The Lambda function is not configured with function-level concurrent execution limits - (Control ID: 164.312(b)). Ensure that a Lambda function's concurrency high and low limits are established. This can assist in baselining the number of requests that your function is serving at any given time. [Error at /AwsCdkNagDemoStack/demo-function/Resource] HIPAA.Security- LambdaDLQ: The Lambda function is not configured with a dead-letter configuration - (Control ID: 164.312(b)). Notify the appropriate personnel through Amazon Simple Queue Service (Amazon SQS) or Amazon Simple Notification Service (Amazon SNS) when a function has failed.
36. • cfn_nag • General-purpose policy-as-code evaluation tool • https://github.com/stelligent/cfn_nag • cfn_guard • Leverages Control Tower proactive controls • Also available as a cdk (validatorplugin) plugin • https://github.com/aws-cloudformation/cloudformation-guard Alternatives
37. KICS - Keeping Infrastructure as Code Secure • Finds security vulnerabilities, compliance issues, and infrastructure misconfigurations early in the development cycle of your infrastructure- as-code. • Works with Cloudformation • Also supports Docker, Helm, Kubernetes, Pulumi, Terraform • https://github.com/Checkmarx/kics
38. Categories • Access Control • Availability • Backup • Best Practices • Build Process • Encryption • Insecure Configurations • Insecure Defaults • Networking and Firewall • Observability • Resource Management • Secret Management • Structure and Semantics • Supply-Chain Reporting options • JSON • SARIF • Gitlab SAST • JUnit • SonarQube • HTML • PDF • CycloneDX • ASFF • CSV • Code Climate • CLI Report
39. CDK Example stack
41. Prowler • Open-Source Security tool for AWS (Azure/GCP) • Cloud Security best practices assessments, audits, incident response, compliance, continuous monitoring, hardening and forensics readiness • Integrates with AWS Security Hub • https://github.com/prowler-cloud/prowler
43. Cost Management & Observability
44. • Integrates directly with VS Code • Calculates Cloud Costs based on Terraform (IaC) • Catch costly typos • https://github.com/infracost/infracost Infracost.io
46. • Direct integration into source control systems or CI/CD • Review cost impact with the team alongside security and code quality • Shows the cost impact of the specific change Infracost.io CI/CD
47. CDK Monitoring Constructs • Easy-to-use CDK constructs for monitoring your AWS infrastructure • Supports a variety of different services • Easily define dashboards, widget and alarms for important metrics https://github.com/cdklabs/cdk-monitoring-constructs
52. Recap Recap
53. Thanks! Questions / Suggestions? Come visit the Luminis booth for your own LEGO figure! http://twitter.com/jreijn https://www.linkedin.com/in/jeroenreijn/