Cloud Development Kits - Are we on the road to Infrastructure Nirvana?!

What happens when you apply a software engineering mindset to infrastructure as code? With the rise of Cloud Development Kits, a new world is opening up. How do full-featured programming languages allow us to express the needs of our solutions? And how do you code these higher level abstractions?

It feels like we have yet to unlock the potential that CDKs bring us... what will it take to reach infrastructure Nirvana... can we ever declare our high-level non-functionals and let the infra code make it happen?!

Transcript

1. Are we on the road to Infrastructure Nirvana?! Cloud Development Kits Nico Krijnen
2. The old days
11. Nico Krijnen @nicokrijnen nkrijnen nicokrijnen CDK: Are we on the road to infrastructure Nirvana?!
12. Cloud Development Kits? Are we on the road to infrastructure Nirvana?!
13. Cloud Development Kits? CDKs? – Learn from Software Engineering! – Infrastructure Nirvana?! Are we on the road to infrastructure Nirvana?!
14. TypeScript Python Go Java C#
16. Construct Hub Solution Constructs
17. CloudFormation Terraform Chef Puppet Ansible ARM Bicep resource elasticsearch_xpack_role my_product-all_env-read { role_name = "my_product-all_env-read" indices = { names = [ "logs-kubernetes.container_logs-my_product.uat", "metrics-*-my_product.uat", "traces-apm*-my_product.uat", "logs-apm*-my_product.uat", "logs-kubernetes.container_logs-my_product.pre", "metrics-*-my_product.pre", "traces-apm*-my_product.pre", "logs-apm*-my_product.pre", "logs-kubernetes.container_logs-my_product.prod", "metrics-*-my_product.prod", "traces-apm*-my_product.prod", "logs-apm*-my_product.prod" ] privileges = ["read", "view_index_metadata"] } applications = { application = "kibana-.kibana" privileges = ["space_all"] resources = ["space:my_product"] new XpackRole(scope, this.roleId, { roleName: this.roleId, indices: [ { names: [ ...this.indexPatterns(), ], privileges: ["read", "view_index_metadata"] }, ], applications: [ { application: "kibana-.kibana", privileges: ["space_all"], resources: [`space:${this.space}`] } ], }); TypeScript Python Go Java C#
18. • Rich expressiveness
19. • Rich expressiveness • No more YAML !
20. • Rich expressiveness • No more YAML ! • Rich tooling support • Rich language features
21. • Rich expressiveness • No more YAML ! • Rich tooling support • Rich language features • Abstractions!
24. What can we learn from Software Engineering? CDKs? – Learn from Software Engineering! – Infrastructure Nirvana?! Are we on the road to infrastructure Nirvana?!
25. 10x more time reading “Indeed, the ratio of time spent reading versus writing is well over 10 to 1. We are constantly reading old code as part of the effort to write new code. ... [Therefore,] making it easy to read makes it easier to write.” ― Robert C. Martin, Clean Code: A Handbook of Agile Software Craftsmanship
26. 10x more time reading Mark Seemann
27. 10x more time reading Mark Seemann
28. Optimize? 10x more time reading Mark Seemann
29. Mark Seemann Code That Fits in Your Head No more than 7 (±2)
30. Mark Seemann Code That Fits in Your Head No more than 7 (±2) https://youtu.be/FPEEiX5unWI
32. terraform required_providers elasticsearch { source = "phillbaker/elasticsearch" version = "..." } provider = { elasticsearch = { url = "..." username = "..." } } resource elasticsearch_xpack_role my_product-all_env-read { applications = { application = "kibana-.kibana" privileges = ["space_all"] resources = ["space:my_product"] } indices = { names = [ "logs-kubernetes.container_logs-my_product.uat", "metrics-*-my_product.uat", "traces-apm*-my_product.uat", "logs-apm*-my_product.uat", "logs-kubernetes.container_logs-my_product.pre", "metrics-*-my_product.pre", "traces-apm*-my_product.pre", "logs-apm*-my_product.pre", "logs-kubernetes.container_logs-my_product.prod", "metrics-*-my_product.prod", "traces-apm*-my_product.prod", "logs-apm*-my_product.prod" ] privileges = ["read", "view_index_metadata"] } role_name = "my_product-all_env-read" } resource elasticsearch_xpack_role my_product-prod_env-read { applications = { application = "kibana-.kibana" privileges = ["space_all"] resources = ["space:my_product"] } indices = { names = [ "logs-kubernetes.container_logs-my_product.prod", "metrics-*-my_product.prod", "traces-apm*-my_product.prod", "logs-apm*-my_product.prod" ] privileges = ["read", "view_index_metadata"] } role_name = "my_product-prod_env-read" } resource elasticsearch_xpack_role_mapping my_product_support_team { role_mapping_name = "my_product_support_team" roles = ["my_product-prod_env-read"] rules = "{\"all\":[{\"field\":{\"realm.name\": \"azure_saml\"}},{\"field\":{\"groups\": \"af8c120a-547b-49df-9c4f-32320816666bb\"}}]}" } resource elasticsearch_xpack_role_mapping my_product_team { role_mapping_name = "my_product_team" roles = ["my_product-all_env-read"] rules = "{\"all\":[{\"field\":{\"realm.name\": \"azure_saml\"}},{\"any\":[{\"field\":{\"groups\": \"630d6107-8a7f-4632-932b-2ca201c639a8\"}},{\"field\":{\"groups\": \"d45fbb82-8759-423e-a889-f53b120bad65\"}}]}]}" }
34. function grantAccessForProductTeams(scope: Construct) { const allEnvsRole = new EnvironmentRole(scope, ["my_product.uat", "my_product.pre", "my_product.prod"] ); const onlyProdRole = new EnvironmentRole(scope, ["my_product.prod"] ); allEnvsRole.grantAccessTo("my_product_team", azure_ad_groups.product.dev, azure_ad_groups.product.management, ); onlyProdRole.grantAccessTo("my_product_support_team", azure_ad_groups.product.support, ); }
35. What can we learn from Software Engineering? CDKs? – Learn from Software Engineering! – Infrastructure Nirvana?! Are we on the road to infrastructure Nirvana?!
36. Value of software
37. Value of software Flexible
38. Value of software Works Can change
39. Value of software Works Can change =
40. Value of software Works Can change =
41. Value of software Works Can change =
42. Value of software Works Can change =
43. Is infrastructure just hardware?
44. Cloud changes everything!
45. What makes code easy to change?
46. What makes code easy to change? • Easy to read and understand intention
47. What makes code easy to change? • Easy to read and understand intention • Version control
48. What makes code easy to change? • Easy to read and understand intention • Version control • Automated tests
49. What makes code easy to change? • Easy to read and understand intention • Version control • Automated tests Safety net }
50. Optimize your architecture to be easy to change
51. SOLID • Single-responsibility principle • Open–closed principle • Liskov substitution principle • Interface segregation principle • Dependency inversion principle Robert C. Martin – Uncle Bob
52. CUPID – for joyful coding • Composable • Unix philosophy • Predictable • Idiomatic • Domain-based https://dannorth.net/2022/02/10/cupid-for-joyful-coding/ Daniel Terhorst-North
53. CUPID – for joyful coding • Composable • Unix philosophy • Predictable • Idiomatic • Domain-based https://dannorth.net/2022/02/10/cupid-for-joyful-coding/ Daniel Terhorst-North
54. CUPID – Composable Plays well with others • Small surface area • Intention-revealing • Minimal dependencies Daniel Terhorst-North
55. CUPID – Unix philosophy Does one thing well • A simple, consistent model • Single purpose vs. single responsibility Daniel Terhorst-North
56. CUPID – Predictable Does what you expect • Behaves as expected • Deterministic • Observable Daniel Terhorst-North
57. CUPID – Predictable Does what you expect • Behaves as expected • Deterministic • Observable Daniel Terhorst-North
58. CUPID – Idiomatic Feels natural • Language idioms • Local idioms Daniel Terhorst-North
59. CUPID – Domain-based The solution domain models the problem domain in language and structure • Domain-based language • Domain-based structure • Domain-based boundaries Daniel Terhorst-North
60. CUPID – Domain-based The solution domain models the problem domain in language and structure • Domain-based language • Domain-based structure • Domain-based boundaries Daniel Terhorst-North I've seen way too many stacks with 700 lines of code instead of being split into domain driven constructs.
61. CUPID – for joyful coding • Composable • Unix philosophy • Predictable • Idiomatic • Domain-based https://dannorth.net/2022/02/10/cupid-for-joyful-coding/ Daniel Terhorst-North
62. CUPID – for joyful coding • Composable • Unix philosophy • Predictable • Idiomatic • Domain-based https://dannorth.net/2022/02/10/cupid-for-joyful-coding/ Daniel Terhorst-North
63. Right level of abstraction? • Test code != App code • Infra code != App code
64. Right level of abstraction? Code is not done until it is absolutely gorgeous (simple design rules): 1.Works 2.Communicates 3.No duplicates 4.Fewest methods & classes
65. Refactoring infrastructure code
66. Common refactor recipes • Rename • Extract method & Inline method • Extract variable & Inline variable
67. Logic + Infra refactoring
68. Logic + Infra refactoring
69. Logic + Infra refactoring
70. Logic + Infra refactoring
71. Infrastructure Nirvana?! CDKs? – Learn from Software Engineering! – Infrastructure Nirvana?! Are we on the road to infrastructure Nirvana?!
72. Value of a software system?
73. Value of a software system? Value it brings to users
74. Value of a software system? Features Value it brings to users
75. Value of a software system? Features Value it brings to users Availability Performance Security
76. Value of a software system? Features Value it brings to users Non-functional Functional Availability Performance Security
77. Where do you want to spend your time? Adding business value Infrastructure details
78. Single-Page App [TypeScript, Svelte, CloudFront, S3] Backend API [Node, Hapi.js, AWS Fargate] SQL Database [AWS RDS Aurora]
79. Single-Page App [TypeScript, Svelte, CloudFront, S3] Backend API [Node, Hapi.js, AWS Fargate] SQL Database [AWS RDS Aurora] const productionApp = new ThreeTierApp(this, "productionApp" { redundancy: Redundancy.twoAvailabilityZones, autoScaling: true, backendLatencyAlertThreshold: Duration.ofSeconds(0.8), backendMemory: MemorySize.ofGB(4), databaseMemory: DatabaseMemorySize.ofGB(8), encryptionAtRest: true, });
80. Single-Page App [TypeScript, Svelte, CloudFront, S3] Backend API [Node, Hapi.js, AWS Fargate] SQL Database [AWS RDS Aurora] const productionApp = new ThreeTierApp(this, "productionApp" { redundancy: Redundancy.twoAvailabilityZones, autoScaling: true, backendLatencyAlertThreshold: Duration.ofSeconds(0.8), backendMemory: MemorySize.ofGB(4), databaseMemory: DatabaseMemorySize.ofGB(8), encryptionAtRest: true, }); const stagingApp = new ThreeTierApp(this, "stagingApp" { redundancy: Redundancy.noRedundancy, autoScaling: false, backendMemory: MemorySize.ofGB(4), databaseMemory: DatabaseMemorySize.ofGB(4), encryptionAtRest: false, });
81. Single-Page App [TypeScript, Svelte, CloudFront, S3] Backend API [Node, Hapi.js, AWS Fargate] SQL Database [AWS RDS Aurora] const productionApp = new ThreeTierApp(this, "productionApp" { redundancy: Redundancy.twoAvailabilityZones, autoScaling: true, backendLatencyAlertThreshold: Duration.ofSeconds(0.8), backendMemory: MemorySize.ofGB(4), databaseMemory: DatabaseMemorySize.ofGB(8), encryptionAtRest: true, }); const stagingApp = new ThreeTierApp(this, "stagingApp" { redundancy: Redundancy.noRedundancy, autoScaling: false, backendMemory: MemorySize.ofGB(4), databaseMemory: DatabaseMemorySize.ofGB(4), encryptionAtRest: false, }); export class ThreeTierApp extends cdk.Construct { constructor(scope: cdk.Construct, id: string, props: ThreeTierAppProps) { super(scope, id); ... } } export interface ThreeTierAppProps { redundancy: Redundancy; autoScaling: boolean; backendLatencyAlertThreshold?: Duration; backendMemory: MemorySize; databaseMemory: DatabaseMemorySize; encryptionAtRest: boolean; } export enum Redundancy { noRedundancy, twoDatacenters, twoRegions } export class MemorySize { private constructor(readonly sizeInGB: number) { if (this.sizeInGB < 0.5) throw new Error("Memory cannot be less than 512 MiB"); if (this.sizeInGB > 120) throw new Error("Memory cannot be more than 120 GB"); } static ofGB(sizeInGB: number): MemorySize { return new MemorySize(sizeInGB); } }
82. Infrastructure Nirvana?!
83. Infrastructure Nirvana?!
84. Infrastructure Nirvana?!
85. Write Joyful infra code, that anyone can understand. Infrastructure Nirvana?!
86. @nicokrijnen nkrijnen nicokrijnen Infrastructure Nirvana?! Write Joyful infra code, that anyone can understand. Cloud Development Kits