Software Delivery in 2016 - A Continuous Delivery Approach

The speech "Software Delivery in 2016" was held by Giovanni Toraldo (Lead Developer at ClouDesire) on July 1st 2016 in Pisa, Italy.
Event: Apericoder
Organizer: Coders TUG

#coderstug #cloudesire #devops #continuousdelivery #CD #jenkins #chef #flyway #git

Transcript

1. Software Delivery
   in 2016
   A Continuous Delivery approach.
   

   View Slide

2. Who am I?
   Giovanni Toraldo (gionn)
   Lead developer ClouDesire.
   com
   --
   Lord of Jenkins
   Protector of the build
   Watcher of the stacktraces
   

   View Slide

3. What is ClouDesire?
   Application Marketplace to help software vendors to sell and provision
   applications
   ● Web Applications:
   ○ provision VM
   ○ on multiple cloud providers
   ○ deploy/upgrade docker containers
   ○ application logging
   ○ resource monitoring
   ● With multi-tenant applications/SaaS:
   ○ expose REST hooks and API for billing lifecycle
   ● manage subscriptions, billing, pay-per-use, invoicing, payments.
   3
   

   View Slide

4. What is Continuous Delivery?
   “Continuous delivery (CD) is a software engineering approach
   in which teams produce software in short cycles, ensuring that
   the software can be reliably released at any time. It aims at
   building, testing, and releasing software faster and more
   frequently. The approach helps reduce the cost, time, and risk
   of delivering changes by allowing for more incremental
   updates to applications in production.”
   https://en.wikipedia.org/wiki/Continuous_delivery
   

   View Slide

5. View Slide

6. The road to enlightenment
   ● Configuration Management
   ● Data management
   ● Continuous Integration
   ● Automated Deployment
   

   View Slide

7. Configuration Management
   ● Keep *everything* in VCS.
   ● Managed software dependencies
   ● Your application should be configurable
   ● Avoid long-living branches
   

   View Slide

8. Data management
   ● Scripted schema migrations
   ● Dataset for development
   ● Be backward and forward compatible (when possible)
   

   View Slide

9. Continuous Integration
   ● Requirements:
   ○ Versioned sources
   ○ Build tool
   ○ Automated tests
   ● Features
   ○ Early feedback on how things are going
   ○ Feedback shared with all the team
   ○ Keep track on how things are going
   ○ Dashboard for automation
   

   View Slide

10. Jenkins for the CI
    ● Easy installation: Just run java -jar jenkins.war, or deploy it
    in a servlet container, or system packages.
    ● Easy configuration: via friendly web GUI with on-the-fly
    error checks and inline help.
    ● Extensibility: Most parts of Jenkins can be extended and
    modified, and there plugins for every needs.
    ● Distributed builds: Jenkins can distribute build/test loads
    to multiple computers with different operating systems.
    

    View Slide

11. What is a pipeline?
    “is an automated manifestation of your process for getting
    software from version control into the hands of your users.”
    Jez Humble and David Farley - Continuous Delivery
    

    View Slide

12. View Slide

13. One job “pipeline” for simple java app
    Prerequisites:
    ● Version Control System (GIT, Mercurial, SVN..)
    ● Dependency management (NPM, Gem, Composer, …)
    ● Build system (Maven, Gradle, ...)
    Jenkins will:
    1. Grab the latest code
    2. Execute commands you want
    3. Report status
    

    View Slide

14. View Slide

15. View Slide

16. View Slide

17. View Slide

18. Pipeline with different stages
    Every stage is a different job, for example:
    1. Compilation / dependencies check job (faster)
    2. Static analysis job (fast)
    3. Unit test job (fastest)
    4. Component test job (slowish)
    5. Integration test job (slow)
    6. Packaging job (fast)
    

    View Slide

19. Configure multiple jobs pipeline
    ● Configure different jobs, each with different commands
    ● Enable notification of another job upon completion
    ● If a job fail, pipeline is interrupted
    

    View Slide

20. More complex pipelines (parallelization)
    ● Parallel job execution
    ○ Speed-up feedback loop
    ● Per-node limit to avoid job clash via Throttle Concurrent Builds Plugin
    

    View Slide

21. More complex pipelines (diamond job)
    Execute multiple build in parallel, wait for completion, then execute another job.
    ● Join plugin
    

    View Slide

22. Static analysis reports (Java centric)
    ● Checkstyle: coding standards
    ● FindBugs: find bugs on bytecode
    ● PMD: find bugs on source code
    ● CPD: copy-paste detector
    ● Open task: scan for FIXME, HACK, XXX...
    

    View Slide

23. View Slide

24. View Slide

25. View Slide

26. View Slide

27. View Slide

28. View Slide

29. GitHub
    ● GIT repository hosting
    ● Simple issue management
    ● Pull requests workflow
    

    View Slide

30. Building GH pull requests
    Pull requests are great for:
    ● Enabling code review
    ● Execute all the tests before merging to master
    GitHub pull request builder plugin
    

    View Slide

31. View Slide

32. Code review
    Pros:
    ● Ensure use-cases are respected
    ● Catch bugs early
    ● Catch distraction errors
    ● Enforce coding quality
    ● Rise the bar among developers
    

    View Slide

33. View Slide

34. Automated Deployment
    ● Requirements:
    ○ Versioned sources
    ○ Build tool
    ○ Automated tests
    ● Features
    ○ Automated deploy procedure
    ○ Easy roll-back when things go wrong
    ○ Easy management of multiple environments
    

    View Slide

35. Chef-zero (ex Chef-Solo)
    ● Ruby DSL
    ● Recipes organized in
    cookbooks
    ● Everything is versioned in
    a git repository
    ● Environment
    management
    ● Single node
    management
    ● Data in JSON
    

    View Slide

36. Chef cookbooks tests and packaging
    ● Multiple cookbooks for multiple stacks
    ○ Test-kitchen
    ■ GCE backend
    1. Each cookbook job notifies downstream cookbooks.
    2. Every cookbook notifies the gatekeeper job
    3. If gatekeeper is green, cookbook-builder job build a tar.gz
    with all the current cookbooks and upload via SCP.
    

    View Slide

37. View Slide

38. View Slide

39. Cookbook Gatekeeper job
    Ensure that all cookbooks are green before packaging the uber .tar.gz
    

    View Slide

40. Parameterized builds
    Configurable environment variables can be injected before the build
    

    View Slide

41. Push parameters while notifying job
    Build another job, with a chosen environment variable populated.
    

    View Slide

42. Multi-cloud integration tests
    50+ parallels tests:
    ● of different applications with different stacks
    ○ Wordpress php+mysql
    ○ ...
    ○ Docker applications
    ● on multiple cloud providers
    ○ Empty VM
    ○ Sample applications
    

    View Slide

43. View Slide

44. View Slide

45. Questions?
    Don’t be shy!
    

    View Slide

46. We are hiring!
    https://cloudesire.cloud/jobs/
    jobs@cloudesire.com
    

    View Slide

47. Thanks!
    

    View Slide