Share
Embed
The Journey of Software Delivery
nghialv
January 25, 2021
Transcript
35 slides
-
Le Van Nghia, CyberAgent, Jan 20, 2021 @nghialv The Journey
of Software Delivery Developer Productivity Study -
About me - CyberAgent • Developer Productivity • OpenSaasStudio •
ABEMA • Ameba • Building PipeCD • Building Work f l ow Automation System • Building Feature Flags, A/B Testing System • Building Monitoring System • Building Deployment Tool • Infrastructure/Microservices Related Works • Building FRESH iOS App • Rebuilding Ameba iOS App -
Agenda • The Fundamentals • Principles and Practices • Progressive
Delivery • GitOps -
The Fundamentals Why do we need How do we achive
the goal Basic concepts Common pitfalls -
What is goal of software professionals If somebody thinks of
a good idea, how do we deliver it to users as quickly as possible? -
How do we achieve that goal The feedback must be
received as soon as possible Every change should trigger the feedback process Analyse and act on the feedback -
CI/CD Apply CI and CD systems to accelerate the delivery
process Actions -
CI/CD When people say “CI/CD,” they are only talking about
Continuous Integration. Nobody is talking about (or practicing) Continuous Deployment. AT ALL. It’s like we have all forgotten it exists. It's time to change that. Charity Majors -
CI != CD - They should be seperated Artifact Storage
Verifying and Analysing the Impact Application Code (.go, .java, .js...) Infrastructure Code (.tf ...) Con f i guration Code (.yaml ...) DockerHub, GCR, ECR... GCS, S3... Git Repository Code Storage Actions Continuous Integration Test Code Git Repository Host Environment Artifact Continuous Delivery Artifact Build and Save Artfacts Cloud User Low-risk actions: release strategy, rollback Deployment Dependency Management Provisioning, Installing Artifact -
Continous Delivery != Continuous Deployment Continuous Deployment means that every
change goes through the pipeline and automatically gets put into production, resulting in many production deployments every day. In order to do Continuous Deployment you must be doing Continuous Delivery. Continuous Delivery just means that you are able to do frequent deployments but may choose not to do it, usually due to businesses preferring a slower rate of deployment. Artifact Continuous Deployment Dev, Test Env Artifact Continuous Delivery Prod Env An example -
Deploy != Release Deployment is the process for installing the
new version of artifact on prod environment. When we say a new version of software is deployed, we mean it is running somewhere in the production environment. Releasing is the process of moving production tra f f i c to the new version. When we say a version of a software is released, we mean that it is responsible for serving production tra f f i c. Deployment need not expose customers to a new version of your service. Given this definition, deployment can be an almost zero-risk activity. Turbine Labs -
Reliable Process Uni f i ed process for all
Idempotent and repeatable process Automation Visibility and Auditability Everyone is responsible Take DX seriously -
Reliable Process Uni f i ed process for all
Idempotent and repeatable process Automation Visibility and Auditability Everyone is responsible Take DX seriously The key to deploying any application in a realiable, consistent manner is constant practice. Use the same process to do all operations of all applications to every environments Deploying new version Scaling service Rolling back DB migration Emergency f i x Kubernetes app Serverless app Infrastructure app Development Testing Staging Production -
Reliable Process Uni f i ed process for all
Idempotent and repeatable process Automation Visibility and Auditability Everyone is responsible Take DX seriously Avoid side effect in the process. The version of tools/dependencies should be speci f i ed. Artifacts should be immutable and reused in all environment . containers: - image: gcr.io/awesome-docker-image:latest bases: - [email protected]:awesome/kustomization-base?ref=v0.1.2-15 containers: - image: gcr.io/awesome-docker-image:v0.1.0 bases: - [email protected]:awesome/kustomization-base Do not use latest Do not use branch -
Reliable Process Uni f i ed process for all
Idempotent and repeatable process Automation Visibility and Auditability Everyone is responsible Take DX seriously People introduce errors, and are not auditable -
Reliable Process Uni f i ed process for all
Idempotent and repeatable process Automation Visibility and Auditability Everyone is responsible Take DX seriously Clearly log the activities Keep everything in version control Make developer loves the process Less repository is better life Everybody is reponsible to build/use/improve the delivery process Even business man, manager can easy to release their features -
Low-risk Process Test Static analysis, unit test, integration test, e2e
test Small changes Veri f i cation based on metrics, logs, events Rolling back Automation Progressive Delivery -
What is Progressive Delivery Continuous Delivery helps team move faster
by making each change small and manageable. Progressive Delivery helps team move safer by reducing the risk of each change by controlling the audience exposed. Progressive Delivery is continous delivery with f i ne-grained control over the blast radius. James Governor, ReadMonk -
How to enable Progressive Delivery Progressive Delivery comes in two
complementary f l avors: environment and user. You can deliver progressively to environments using canary or blue-green deployments to limit exposure when new version is deployed (instance, region, env...). Progressive Delivery for users, on the other hand, lets you limit the exposure of which users see a new change. Decide the strategy of exposure Analyse the feedback to decide whether to go on or rollback -
How to enable Progressive Delivery - Canary Canary Release +
Canary Analysis + Automated Rollback Image Source -
How to enable Progressive Delivery - Feature Flags Image Source
Dev Team Beta Testers All Users Deploy f i rst and control the exposure from management page or by automation -
What is GitOps? Operations by Pull Request - weaveworks The
core idea Store the desired state of application declaratively in the Git repository and Sync them to host environment. Git Repository Host Environment Sync to match Cloud User What artifact should be released What strategy should be used What is the desired con f i guration Agent PR to change image from v1 to v2 PR to scale app from 100 to 200 replicas PR to update revert commit to rollback PR to update LB con f i guration -
What is GitOps? GitOps is only for CD not CI
Artifact Storage Application Code (.go, .java, .js...) Infrastructure Code (.tf ...) Con f i guration Code (.yaml ...) DockerHub, GCR, ECR... GCS, S3... Git Repository Code Storage Actions Continuous Integration Git Repository Host Environment Continuous Delivery Artifact Cloud User GitOps Agent Git Repository What artifact should be released What strategy should be used What is the desired con f i guration -
Benefits of GitOps Unifying Deployment, Monitoring and Management. Git as
the single source of truth of a system's desired state. ALL intended operations are committed by pull request ALL di f f s between inteneded and observed state with automatic convergence ALL changes are observable, veri f i able and auditable https://www.weave.works/blog/what-is-gitops-really -
Conclusion • Same process for all operations, all applications, all
environments • Small change, frequent deployment • Automate almost everything • Delivery process is the whole teams's responsibility • Take developer experience seriously