Living in 
 Cloud-Native World Liam Hwang Buzzvil 

1. What is Cloud-Native Application? 2. 12 Factor Application 3. Beyond the 12 Factor Application 4. Current Status 5. Case. Migrating Squid to K8S 

No content

Cloud-Native Application “A cloud-native application is an application that has been designed and implemented to run on a Platform-as-a-Service installation and to embrace horizontal elastic scaling.” - Kevin Hoffman 

Cloud-Native Application Characteristics • Resiliency • Embrace failure instead of trying to prevent • Takes advantage of dynamic nature of running on a platform • Agility • Fast deployment, quick iteration 

Cloud-Native Application Characteristics • Operability • Control of application life cycles from inside the app instead of relying on external processes and monitors • Observability • Provide information about application state 

How to achieve • Microservice • Health Reporting(application life cycle) • Telemetry Data(application business objective - alert) • Resiliency • Declarative, not Reactive 

Cloud-Native Landscape 

Twelve-Factor App • Brought by Heroku • Just code without worrying about infrastructure • Good starting point toward building application running on cloud. • Little bit outdated… • https://12factor.net/ 

Twelve-Factor App 1. Codebase 2. Dependencies 3. Configuration 4. Backing Services 5. Build, Release, Run 6. Processes 7. Port Binding 8. Concurrency 9. Disposability 10. Dev/Prod Parity 11. Logs 12. Admin Processes 

Beyond the 12 factor app • Outdated… • Container • "works well on heroku" 

Beyond the 12 factor app 1. One Codebase, One Application 2. API First 3. Dependency Management 4. Design, Build, Release, Run 5. Configuration, Credentials, and Code 6. Logs 7. Disposability 8. Backing Services 9. Environment Parity 10. Administrative Processes 11. Port Binding 12. Stateless Processes 13. Concurrency 14. Telemetry 15. Authentication and Authorization 

1. One Codebase, One Application • Consist of a single codebase tracked in a VCS • Potentially splitting monoliths off into microservices • Violations • Application is made up of multiple repositories • main application(en-queuer) + tightly coupled worker(de-queuer): multiple codebase supporting single application • Single codebase with multiple launch script • Shared code is separately released product. Can be vendored • "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations."
 — M. Conway 

2. API First • Not in the original 12-factor app • API as first-class artifact of the dev process • To be consumed by client apps or by other services • Contract-first development • API Blueprint, OpenAPI 3.0 

3. Dependency Management • Never rely on implicit existence of system-wide package • pip, npm, bundler, godep, … • Deployment should be repeatable 

4. Design, Build, Release, Run • Strict separation of build, run environment is required • Design: small design changes per iteration, how dependencies is vendored • Build: artifacts generated from code repository. build - deployments is 1:N. • Release: Build artifact combined with env/app-specific configuration • Run: Keeping app alive, monitor health, log aggregation, and so on. 

No content

Clear sign that the four stages of this process are likely not as separate as they should be 

5. Configuration, Credentials, Code • Configuration: any value can vary across deployments • URL, other information about backing service(web server, smtp, database) • Third-party service credentials(AWS, api token) • Property file, configuration YML, XML • Should keep configuration separate from code (Imagine repository being open sourced on GitHub) • Cloud Provider usually provides externalized configuration 

6. Logs • “truly cloud-native application never concerns itself with routing or storage of its output stream” • Write all log entries to stdout and stderr • Aggregation, processing and storage are taken care by Platform • Decouple application from the knowledge of log storage, processing, analysis • Modify log storage without touching application 

6. Logs Web Nginx DB Ad server Admin UI Web Logging Backend(ELK, S3, Splunk, Fluentd) stdout stdout stdout stdout stdout stdout 

7. Disposability • Application Lifecycle is ephemeral • Start rapidly, shutdown gracefully • health check 

8. Backing Services An application relying on backing services (Beyond the Twelve-Factor App, 2016) 

8. Backing Services • Treat backing services as “bound resources”
 (in relate to Factor 4) • Attaching/detaching backing system should be possible at will, without re-deploying the app • What if the database is completely broken and need to change the binding to newly brought up db instance? 

9. Environment Parity • Keep all environment as similar as possible • Give confidence that the application WILL WORK EVERYWHERE • Time: time until change is pushed to production should remain short(mins ~ same day) • People: human should never be deploying application. it should be done by clicking a button or triggered by event • Resources: backing service should be similar across all environments 

10. Administrative Processes • Use of administrative processes is generally bad • Database migrations • Interactive programming consoles (REPLs) • Running timed scripts, such as a nightly batch job or hourly 
 import • Running a one-off job that executes custom code only once 

11. Port Binding • Do not micromanage port binding by yourself • Expose standard port, and let PaaS do the routing for you appserver1 app1 :8080 app2 :8081 app3 :8082 

12. Stateless Processes • Pure stateless is impossible • Applications tend to be stateful • Stateless: all lasting state must be external(backing services) to the application • Cache? 

13. Concurrency • Scale up isn’t way to go(end up with bigger monolithic app) • Scale out horizontally using the process model • Most cloud provider is excellent at horizontal scaling 

14. Telemetry • Application performance monitoring(APM) • Domain specific telemetry • Health and system log 

15. Authentication and Authorization • Every request for application’s resource should know • who is making the request, • the role to which that consumer belongs • RBAC 

Case study.
 Up & running legacy service on cloud • Squid: Caching proxy • At Buzzvil, we mainly use as transparent forward proxy when requesting 3rd-party API endpoints requiring whitelisted IP • Forward proxying HTTPS via Connect 

Problems we had • Inability to scale out(whitedlisted IP pools hardly scale) • Limited concurrency • Low resource utilization • Hard to monitor • Lack of log aggregation • Hard to configure across multiple instances 

Enter Helm • https://helm.sh/ • package manager for kubernetes • Even complex application can be easily deployed as a package 

Squid as a helm package • https://github.com/Buzzvil/buzz-k8s-resources/tree/master/ helm/squid/squid 

Problems solved • Inability to scale out(whitedlisted IP pools hardly scale) → Private k8s cluster with egress NAT Gateway • Limited concurrency → horizontal scaling • Low resource utilization → multiple pods per instance • Hard to monitor → exporter & prometheus & grafana • Lack of log aggregation → stream access.log to stdout • Hard to configure across multiple instances -> manage as a helm package 

Current problems • AMI • 1:1 mapping of instance to release • Configuration is coupled with codebase • Log path • Deploy script • Provisioning script involved while scaling out • “Ecosystem of monoliths” 

From DevOps’ POV • Some best(or better) practice could be derived • Eliminate bad practices hinder us from going fast • Discuss how we transform to move&ship faster 

Thank You