kubectl apply -f cloud-Infrastructure.yaml mit Crossplane et al. @CNNMainz

Simple and efficient development of cloud-native applications still poses significant challenges for many teams. In addition to the implementation of features and microservices, developers are now often also responsible for building the required cloud services with Infrastructure as Code à la Terraform. Unfortunately, this quickly leads to high cognitive overload and suboptimal solutions.

Crossplane, ACK and other open source add-ons for Kubernetes try to address this problem. With these extensions, cloud infrastructure can be declaratively provisioned easily without writing a single line of code. This session shows the practical use of both technologies and its core functions, using AWS and GCP as an example, as well as the seamless integration with a GitOps approach.

Transcript

1. 1 Mario-Leander Reimer mario-leander.reimer@qaware.de @LeanderReimer #gernperDude #CloudNativeNerd #qaware qaware.de Photo by CHUTTERSNAP on Unsplash kubectl apply -f cloud-Infrastructure.yaml mit Crossplane et al.
2. 2 Mario-Leander Reimer Principal Software Architect @LeanderReimer #cloudnativenerd #qaware #gernperDude
3. “Too much cognitive load will become a bottleneck for fast flow and high productivity for many DevOps teams.” QAware | 3 ▪ Intrinsic Cognitive Load Relates to fundamental aspects and knowledge in the problem space (e.g. used languages, APIs, frameworks) ▪ Extraneous Cognitive Load Relates to the environment (e.g. console command, deployment, configuration) ▪ Germane Cognitive Load Relates to specific aspects of the business domain (aka. „value added“ thinking)
4. A Platform team and its engineers are a key enabler for high productivity of stream-aligned DevOps teams. QAware | 4 ▪ Responsible to build and operation a platform to enable and support the teams in their day to day development work. ▪ The platform aims to hide the inherent complexity to reduce the cognitive load for the other teams. – Standardization – Self-Service ▪ Fully automated software delivery is the goal! https://hennyportman.wordpress.com/2020/05/25/review-team-topologies/
5. QAware | 5 You have to work in layers!
6. Cloud-native Application Engineering Cloud-native Platform Engineering The 5 Layers of Cloud-native Software Engineering QAware | 6 IaaS Network, Compute, Storage (VPC, EC2, NLB, ALB, ...) CaaS (Kubernetes Services) PaaS (Software Infrastructure Blueprints with Helm and Continuous Delivery Toolchain) Application-specific Software Infrastructure Cloud-friendly & cloud-native Applications Architect Build Run Amazon SNS AWS IAM Amazon EC2 Amazon EBS
7. The 5 Layers of Cloud-native Software Engineering QAware | 7 IaaS Network, Compute, Storage (VPC, EC2, NLB, ALB, ...) CaaS (Kubernetes Services) PaaS (Software Infrastructure Blueprints with Helm and Continuous Delivery Toolchain) Application-specific Software Infrastructure Cloud-friendly & cloud-native Applications Architect Build Run Amazon SNS AWS IAM Amazon EC2 Amazon EBS ?
8. Why not model cloud infrastructure as Kubernetes resources?
9. Custom Resource Definitions are user-defined, declarative extensions of the Kubernetes API QAware | 9 ▪ Abstraction of complex application constructs and concepts ▪ Definition solely via CustomResourceDefinitions ▪ Structure definition via OpenAPI v3.0 Validation Schema ▪ Default Support for several API Features: CRUD, Watch, Discovery, json-patch, merge-patch, Admission Webhooks, Metadata, RBAC, … ▪ Versioning und Conversion supported via Webhooks
10. QAware | 10
11. QAware | 11 Operator. - Do stuff with my CRDs.
12. Operators are codified Ops procedures! QAware | 12 ▪ Operators are the path towards Zero-Ops. They enable auto-updating, self-monitoring and self-healing infrastructure and applications. ▪ The concept was coined in the Kubernetes world. It’s now been adopted and used widespread in the cloud native world. ▪ Examples: OKD, Sealed Secrets, Kube Monkey, Weave Flux, Crossplane, and many more …
13. Kubernetes Operators Explained QAware | 13
14. Introducing the Operator SDK QAware | 14
15. lreimer/aws-ecr-operator
16. QAware | 16 https://intl.startrek.com/sites/default/files/styles/amp_metadata_content_image_min_696px_wide/public/images/2020-05/memes_002.png Are you serious?!
17. qaware/k8s-native-iac qaware/cloud-native-explab
18. Conceptual Showcase Architecture QAware | 18 Provision GitOps Cluster API AWS Controllers for Kubernetes Config Connector
19. Config Connector Addon for Google Kubernetes Engine QAware | 19 ▪ Define and use Google Cloud resources directly from Kubernetes. No need to define resources outside the cluster using traditional IaC tools. ▪ Config Connector can be added during GKE installation or later ▪ Some in-cluster configuration required after initial setup ▪ Requires a dedicated service account with suitable permissions ▪ Currently all major Google services and resources supported ▪ https://cloud.google.com/config-connector/docs/reference/overview
20. Examples for Config Connector Resources QAware | 20
21. Manage AWS services using the Amazon Controllers for Kubernetes (ACK) QAware | 21 ▪ Define and use AWS service resources directly from Kubernetes. No need to define resources outside the cluster using traditional IaC tools. ▪ Each ACK service controller is packaged into a separate container image and Helm chart ▪ Uses IAM Roles for Service Accounts (IRSA) to automate the provisioning and rotation of temporary IAM credentials ▪ Currently 20 different controllers with RELEASED status available, however, most of these are still in PREVIEW maintenance phase ▪ https://aws-controllers-k8s.github.io/community/
22. Examples for ACK Resources QAware | 22
23. Crossplane in a Nutshell QAware | 23 ▪ Open Source Kubernetes Add-on. Universal Control Plane for Cloud Infrastructure. ▪ Cloud Infrastructure Services can be defined declaratively by application teams ▪ Platform teams can provide relevant cloud infrastructure services via high level self-services APIs ▪ Individual Provider bundle a set of Managed Resources with their controllers. All major cloud providers are supported, e.g. AWS, GCP, Azure, Alibaba, … ▪ Managed Resources are fine granular representations of external cloud resources ▪ Composite Resource Definitions or XRDs enable the definition and creation of new abstractions for composite managed resources ▪ https://crossplane.io
24. Examples for Crossplane AWS Resources QAware | 24 apiVersion: sqs.aws.crossplane.io/v1beta1 kind: Queue metadata: name: test-queue.fifo labels: region: eu-central-1 spec: deletionPolicy: Delete forProvider: region: eu-central-1 contentBasedDeduplication: true delaySeconds: 3 fifoQueue: true # 2 KB message size maximumMessageSize: 2048 # 5 minutes messageRetentionPeriod: 300 providerConfigRef: name: providerconfig-aws apiVersion: s3.aws.crossplane.io/v1beta1 kind: Bucket metadata: name: mastering-gitops annotations: crossplane.io/external-name: mastering-gitops-eu-central-1 labels: region: eu-central-1 spec: deletionPolicy: Delete forProvider: acl: private locationConstraint: eu-central-1 serverSideEncryptionConfiguration: rules: - applyServerSideEncryptionByDefault: sseAlgorithm: AES256 providerConfigRef: name: providerconfig-aws
25. Kubernetes Cluster API QAware | 25 ▪ Official Kubernetes sub-project ▪ Declarative APIs and tooling to provision, upgrade, and operate multiple Kubernetes clusters ▪ Work in different environments, both on-premises and in the cloud ▪ Reuse and integrate existing ecosystem components rather than duplicating
26. More Talks to come … QAware | 26 6.12. online!
27. qaware.de QAware GmbH Aschauer Straße 32 81549 München Tel. +49 89 232315-0 info@qaware.de twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware