$30 off During Our Annual Pro Sale. View Details »

The Journey of Software Delivery

nghialv
January 25, 2021

The Journey of Software Delivery

nghialv

January 25, 2021
Tweet

More Decks by nghialv

Other Decks in Technology

Transcript

  1. Le Van Nghia, CyberAgent, Jan 20, 2021

    @nghialv
    The Journey of Software Delivery
    Developer Productivity Study

    View Slide

  2. About me
    @nghialv
    @nghialv2607

    View Slide

  3. About me - CyberAgent
    • Developer Productivity

    • OpenSaasStudio

    • ABEMA

    • Ameba
    • Building PipeCD

    • Building Work
    f
    l
    ow Automation System

    • Building Feature Flags, A/B Testing System

    • Building Monitoring System

    • Building Deployment Tool

    • Infrastructure/Microservices Related Works

    • Building FRESH iOS App

    • Rebuilding Ameba iOS App

    View Slide

  4. About me - DIY
    https://twitter.com/nghialv2607/status/1345936214407274496?s=20

    View Slide

  5. Agenda
    • The Fundamentals

    • Principles and Practices

    • Progressive Delivery

    • GitOps

    View Slide

  6. The Fundamentals
    Why do we need

    How do we achive the goal

    Basic concepts

    Common pitfalls

    View Slide

  7. What is goal of software professionals
    If somebody thinks of a good idea, 

    how do we deliver it to users as quickly as possible?

    View Slide

  8. How do we achieve that goal
    The feedback must be received as soon as possible
    Every change should trigger the feedback process
    Analyse and act on the feedback

    View Slide

  9. Requiring a robust Delivery Process
    Reliable Rapid Low-Risk

    View Slide

  10. CI/CD
    Apply CI and CD systems to accelerate the delivery process
    Actions

    View Slide

  11. CI/CD
    When people say “CI/CD,” they are only talking about Continuous Integration. 

    Nobody is talking about (or practicing) Continuous Deployment. AT ALL. 

    It’s like we have all forgotten it exists. It's time to change that.

    Charity Majors

    View Slide

  12. CI != CD - They should be seperated
    Artifact Storage
    Verifying and Analysing the Impact
    Application Code 

    (.go, .java, .js...)
    Infrastructure Code 

    (.tf ...)
    Con
    f
    i
    guration Code 

    (.yaml ...)
    DockerHub, GCR, ECR...
    GCS, S3...
    Git Repository
    Code Storage
    Actions
    Continuous Integration
    Test Code
    Git Repository
    Host Environment
    Artifact
    Continuous Delivery
    Artifact
    Build and Save Artfacts
    Cloud
    User
    Low-risk actions: release strategy, rollback
    Deployment Dependency Management
    Provisioning, Installing Artifact

    View Slide

  13. Continous Delivery != Continuous Deployment
    Continuous Deployment means that every change goes through the pipeline and
    automatically gets put into production, resulting in many production deployments every day.
    In order to do Continuous Deployment you must be doing Continuous Delivery.
    Continuous Delivery just means that you are able to do frequent deployments but may
    choose not to do it, usually due to businesses preferring a slower rate of deployment.
    Artifact
    Continuous Deployment
    Dev, Test Env
    Artifact
    Continuous Delivery
    Prod Env
    An example

    View Slide

  14. Deploy != Release
    Deployment is the process for installing the new version of artifact on prod environment. 

    When we say a new version of software is deployed, we mean it is running somewhere in
    the production environment.
    Releasing is the process of moving production tra
    f
    f
    i
    c to the new version.

    When we say a version of a software is released, we mean that it is responsible for
    serving production tra
    f
    f
    i
    c.
    Deployment need not expose customers to a new version of your service.


    Given this definition, deployment can be an almost zero-risk activity.


    Turbine Labs

    View Slide

  15. Principles and Practices of Software Delivery

    View Slide

  16. Achieving a robust Delivery Process
    Reliable Rapid Low-Risk

    View Slide

  17. Reliable Process
    Uni
    f
    i
    ed process for all


    Idempotent and repeatable process


    Automation

    Visibility and Auditability


    Everyone is responsible


    Take DX seriously

    View Slide

  18. Reliable Process
    Uni
    f
    i
    ed process for all


    Idempotent and repeatable process


    Automation

    Visibility and Auditability


    Everyone is responsible


    Take DX seriously
    The key to deploying any application in a realiable, consistent manner is
    constant practice.

    Use the same process to do all operations of all applications to every
    environments
    Deploying new version

    Scaling service

    Rolling back

    DB migration

    Emergency
    f
    i
    x

    Kubernetes app

    Serverless app

    Infrastructure app
    Development

    Testing

    Staging

    Production

    View Slide

  19. Reliable Process
    Uni
    f
    i
    ed process for all


    Idempotent and repeatable process


    Automation

    Visibility and Auditability


    Everyone is responsible


    Take DX seriously
    Avoid side effect in the process.
    The version of tools/dependencies should be speci
    f
    i
    ed.

    Artifacts should be immutable and reused in all environment
    .


    containers:


    - image: gcr.io/awesome-docker-image:latest
    bases:


    - [email protected]:awesome/kustomization-base?ref=v0.1.2-15
    containers:


    - image: gcr.io/awesome-docker-image:v0.1.0
    bases:


    - [email protected]:awesome/kustomization-base
    Do not use latest
    Do not use branch

    View Slide

  20. Reliable Process
    Uni
    f
    i
    ed process for all


    Idempotent and repeatable process


    Automation

    Visibility and Auditability


    Everyone is responsible


    Take DX seriously People introduce errors, and are not auditable

    View Slide

  21. Reliable Process
    Uni
    f
    i
    ed process for all


    Idempotent and repeatable process


    Automation

    Visibility and Auditability


    Everyone is responsible


    Take DX seriously
    Clearly log the activities

    Keep everything in version control
    Make developer loves the process

    Less repository is better life
    Everybody is reponsible to build/use/improve the delivery process

    Even business man, manager can easy to release their features

    View Slide

  22. Rapid Process
    Small change

    Do more frequently


    Automate almost everything

    View Slide

  23. Low-risk Process
    Test

    Static analysis, unit test, integration test, e2e test

    Small changes

    Veri
    f
    i
    cation based on metrics, logs, events

    Rolling back

    Automation

    Progressive Delivery

    View Slide

  24. Progressive Delivery

    View Slide

  25. What is Progressive Delivery
    Continuous Delivery helps team move faster by making each change small and manageable.


    Progressive Delivery helps team move safer by reducing the risk of each change by controlling
    the audience exposed.
    Progressive Delivery is continous delivery with
    f
    i
    ne-grained control over the blast radius.

    James Governor, ReadMonk

    View Slide

  26. How to enable Progressive Delivery
    Progressive Delivery comes in two complementary
    f
    l
    avors: environment and user.

    You can deliver progressively to environments using canary or blue-green
    deployments to limit exposure when new version is deployed (instance, region, env...).

    Progressive Delivery for users, on the other hand, lets you limit the exposure of which
    users see a new change.
    Decide the strategy of exposure
    Analyse the feedback to decide whether to go on or rollback

    View Slide

  27. How to enable Progressive Delivery - Canary
    Canary Release + Canary Analysis + Automated Rollback
    Image Source

    View Slide

  28. How to enable Progressive Delivery - Feature Flags
    Image Source
    Dev Team
    Beta Testers
    All Users
    Deploy
    f
    i
    rst and control the exposure from management page or by automation

    View Slide

  29. GitOps

    View Slide

  30. What is GitOps?
    Operations by Pull Request - weaveworks
    The core idea

    Store the desired state of application declaratively in the Git repository 

    and Sync them to host environment.
    Git Repository
    Host Environment
    Sync to match
    Cloud
    User
    What artifact should be released
    What strategy should be used
    What is the desired con
    f
    i
    guration
    Agent
    PR to change image from v1 to v2
    PR to scale app from 100 to 200 replicas
    PR to update revert commit to rollback
    PR to update LB con
    f
    i
    guration

    View Slide

  31. What is GitOps?
    GitOps is only for CD not CI
    Artifact Storage
    Application Code 

    (.go, .java, .js...)
    Infrastructure Code 

    (.tf ...)
    Con
    f
    i
    guration Code 

    (.yaml ...)
    DockerHub, GCR, ECR...
    GCS, S3...
    Git Repository
    Code Storage
    Actions
    Continuous Integration
    Git Repository
    Host Environment
    Continuous Delivery
    Artifact
    Cloud
    User
    GitOps Agent
    Git Repository
    What artifact should be released
    What strategy should be used
    What is the desired con
    f
    i
    guration

    View Slide

  32. Benefits of GitOps
    Unifying Deployment, Monitoring and
    Management.

    Git as the single source of truth of a
    system's desired state.


    ALL intended operations are committed

    by pull request


    ALL di
    f
    f
    s between inteneded and observed
    state with automatic convergence


    ALL changes are observable, veri
    f
    i
    able

    and auditable


    https://www.weave.works/blog/what-is-gitops-really

    View Slide

  33. Conclusion
    • Same process for all operations, all applications, all environments

    • Small change, frequent deployment

    • Automate almost everything

    • Delivery process is the whole teams's responsibility

    • Take developer experience seriously

    View Slide

  34. What is next
    Let's try PipeCD!
    https://pipecd.dev

    View Slide

  35. Thank You

    View Slide