Vendor-Agnostic Cloud-Native Apps

When designing Cloud Native applications, developers end up making choices that lead to vendor lock-in early on in the lifecycle of a project.

Drawing from speakers own experience of making such mistakes, this talk covers some good practises in areas such a data storage, compute, observability, application code design, authentication / authorization and infrastructure provisioning to avoid vendor lock-in without giving up on features and flexibility. It also touches upon when and how to introduce vendor specific code/tooling in a safe manner.

Transcript

1. 1 | Copyright appCD Developing Applications in a Vendor Agnostic Cloud Native Way Kunal Dabir
2. 2 | Copyright appCD About Me • VP of Engineering at appCD ◦ appCD makes devops easy by generating Infra as Code • Polyglot Developer • Author of Gradle Essentials • @kdabir (github/twitter)
3. 3 | Copyright appCD Letʼs Begin • Based on my own experiences (read mistakes) • Best Practises for Developers, DevOps, Eng Managers, & Architects • No offence to any Cloud Provider or SaaS Vendor
4. 4 | Copyright appCD How Lock-in Creeps in
5. 5 | Copyright appCD The Tale of a Startup • Small Team • Ship fast (read everyday) • Managed NoSQL database
6. 6 | Copyright appCD From Startup to Scale-up 🚀 2 years down the line • 20 Million Users MAU • Throughput 1000 req/sec • API Latency 70 ms • DynamoDB Latency 58 ms
7. 7 | Copyright appCD From Aha to Uh-Oh! • Querying DynamoDB was hard • Exporting data was hard and expensive ◦ Redshift job consumed lot of RCUs every day • Competing cloud gave us massive Discount ◦ Application code tied to DynamoDB ◦ We could not migrate to any other cloud • We were Locked-in 😭
8. 8 | Copyright appCD Lock-in Cost of making the Switch
9. 9 | Copyright appCD • With rising costs itʼs always good to have options open • Adopt multi-cloud strategy • Save future troubles • Possible Reverse migration Why Vendor Agnostic Solutions
10. 10 | Copyright appCD May be except when... building a proof of concept to figure out Product Market Fit time to market is the most important factor have resources to refactor or rewrite in future
11. 11 | Copyright appCD Conscious Decision v/s Accidental Lock-in
12. 12 | Copyright appCD Accidental Lock-ins: Different priorities Developer • Latest & Shiny • Easy API • Looks cool on Resume • Familiarity Engineering Management • Cheapest to operate • Easy to hire people for • Availability of support DevOps / SRE / Platform Engineer • Stable version • Easy to provision • Smooth to operate • Control and flexibility
13. 13 | Copyright appCD The “since we already useˮ Syndrome We are using __ anyways, and __ comes by default and works like magic Why add credit card on yet another vendor. No oneʼs gonna approve Oh! All the example code we found was using __ on __ so we used it too 💳 🪄 󰣻
14. 14 | Copyright appCD Examples of “since we already useˮ • Since we are required to use AWS API Gateway for Lambda, letʼs just use it for other services as well • Since we use Firebase to store data, letʼs just use Google Cloud Functions for implementing business logic • Since EKS ships logs to CloudWatch Logs by default, letʼs just use it for all observability needs • Since we already use GitHub as SCM, letʼs just use GitHub Actions • Since we already use AWS, letʼs just use SES for sending emails * BTW, I absolutely love all of the above services
15. 15 | Copyright appCD Bundle and Lock-in • Vendor Specific solutions can give you incredible development speed, runtime performance, infinite scale and ease of use. • But may prove costly in the long run, especially when you need to exit out
16. 16 | Copyright appCD • We chose to use serverless & managed services • Event driven architecture, ability to scale to up and down • We were happy with the services we were using • We used wrappers and abstractions where possible • It was a conscious decision ◦ we wanted quick time to market ◦ Couldnʼt afford lot of devops engineers and cycle time ◦ Scale up and down based on traffic Another Startup: Serverless FTW
17. 17 | Copyright appCD Strategies for Designing Vendor Agnostic Applications
18. 18 | Copyright appCD Cost of Making Changes Application Code Business logic, Performance optimizations, Re-test thoroughly 3 Data Layer Data migration, Data-Model changes, and potentially all of the above layers 4 Configuration Environment Variables, Secrets, Keys 1 Build Logic Dependencies like Libraries, Drivers, Agents, and OS 2 * IaC always needs to be rewritten depending on target cloud
19. 19 | Copyright appCD Data Layer • Database, Blob Storage, Queues, Pub-Subs and Caches • Use Products that are available on multiple providers like PostgreSQL, MySQL, Redis, MongoDB, ElasticSearch etc. • Use proprietary services only when you have a solid reason ◦ Exporting huge volume of data can be a challenge Ex. DynamoDB, Google Datastore, Firebase) ◦ Rewrite of application code may be required
20. 20 | Copyright appCD Types for services Un-managed You provision it, host it, patch it, back it up, restore it when it goes down, scale it up and down E.g. Hosting PostgreSQL on EC2 Managed Provider manages infra, You manage scaling. Scaling require some planning. You pay for idle resources. E.g. AWS RDS Provider manages scaling as well, generally scale from 0 to hypothetical infinity in seconds and can scale back to 0. You pay by use. E.g AWS Aurora Serverless Serverless
21. 21 | Copyright appCD • In my next startup we used RDS/Aurora • App Code is guaranteed to be portable right? • But some services ended up using RDS Data API 󰣻 RDS Data API Data API) provides a secure HTTP endpoint to run SQL statements without managing connections. Startup Story: Careful this time
22. 22 | Copyright appCD Application Code • Stay on top of your dependencies in: ◦ build.gradle, pom.xml, go.mod, package.json, Gemfile, requirements.txt • Use Static Code analysis and build tools to detect if Cloud SDKs are used in inappropriate places (core business logic) • Use Standards APIs wherever possible e.g. Open Telemetry, JDBC, SMTP ◦ Standard Database connectivity like JDBC & SQL to connect to DB
23. 23 | Copyright appCD Example: SES Instead of using AWS SDK SES client, we could use the SMTP API so that the code is portable ❌ ✅
24. 24 | Copyright appCD Example: Password Rotation • Move from code Dependency to build Dependency ◦ https://github.com/aws/aws-secretsmanager-jdbc ◦ just build/config changes • OR Move it outside the code (requires restarting the container) ❌ ✅
25. 25 | Copyright appCD Application Code: Use Portable Cloud Abstraction • Go - https://go.dev/blog/go-cloud • Python - https://libcloud.apache.org/ • Ruby - https://github.com/fog/fog • Java - https://jclouds.apache.org/start/
26. 26 | Copyright appCD Abstract out vendor specific code • Move vendor specific code behind interface • The implementation can be swapped in future • Have it implemented in a separate module 🫣
27. 27 | Copyright appCD Compute • Containerize your applications and itʼs dependency on other services • Run it locally Kubernetes) - minikube, kind, k3s • If you are using Lambda, ◦ use Spring boot, Micronaut or Express JS adapters for lambda, so that you can test the services locally ◦ Explore OpenFaaS / KNative • Authentication / Authorization ◦ Use OAuth / JWT ◦ You may choose portable solutions like Dex, KeyCloak 📦
28. 28 | Copyright appCD Keep operations in mind • Log in JSON format ◦ If using JVM, use Slf4J Facade • Use OpenTelemetry or Micrometer for Metrics, Traces ( and Logs) ◦ Code to interfaces, ◦ donʼt use vendor specific code for capturing traces/spans • Use frameworks that emit important metrics and logs by default • Prefer sidecars for service-mesh and cross cutting concerns
29. 29 | Copyright appCD The Litmus Test Can You: 1. Develop (test & run) locally without connecting internet? 2. Use the same version of services as in production (db, queue, caches, proxy...? 3. Run locally using Docker Compose, (and Deploy using Kubernetes)? 4. Host it on any cloud with just configuration or build changes?
30. 30 | Copyright appCD The CI/CD Layer • Cloud Vendor Specific: AWS CodePipeline CodeDeploy, Azure DevOps • Provided by SCM Github Actions, Gitlab Pipeline, BitBucket Pipeline, • Independent SaaS CircleCI, Travis • Self Hosted: Jenkins, GoCD.org • Adaptable: Earthly.dev, Dagger.io •
31. 31 | Copyright appCD Infrastructure as Code Supports multiple clouds • Terraform / OpenTofu, • Pulumi, • Cloud Development Kit for Terraform CDKTF • K8s (helm, kustomize) * Learn once write everywhere * same code is not portable Vendor Specific • CloudFormation, • AWS CDK, • Azure Resource Manager, • Google Cloud Deployment Manager
32. 32 | Copyright appCD Summary: Choosing the right stack Works on major clouds • Allowed to self host • Offered as Managed service Provides Standard API • Multiple implementations available • Libraries in various languages • Doesnʼt depend on Cloud Vendorʼs SDK Stable and Well Maintained • Has active community around it • Preferably) backed by a company • Preferably) Open Source
33. 33 | Copyright appCD Summary • Choose cloud services consciously not accidentally • Containerize everything • Stick to Standard APIs in application code • Litmus Test: Can you run your entire stack locally offline, using same version of services as in production, without costing you any money? • Make sure you can export data easily (egress cost and format) • Choose vendor agnostic IAC - (learn once, rewrite everywhere)
34. 34 | Copyright appCD https://xkcd.com/927
35. 35 | Copyright appCD Thank You @kdabir | appcd.com