Pipeline Cookbooks Meetup Talk Chef Seattle

Transcript

2. Pipeline Automated delivery of cookbooks using Berkshelf and Jenkins
3. Stephen Lauck lauck@getchef.com @stephenlauck https://github.com/stephenlauck
4. Live Demo?
5. Why? Control the flow of cookbooks through your environments * Automate testing and delivery of cookbooks * Track all cookbooks for an org or enterprise * Uniform space to do integration testing * Manage dependency and versions of cookbooks for many environments using one codebase * Teams collaborate/integrate in source code only ie git * Developers work locally, no direct access to chef servers/orgs via knife
6. Automation of your workflow is a good thing. ! • one way moving of code towards production • one place for integration • local development before integration
7. Wrapper Cookbook Wrap the pipeline cookbook with your settings
8. How it works ! • 1. create berksfile with all the cookbooks • 2. pipeline job for the berksfile • 4. CD/CI job per cookbook uploads to artifact chef server • 3. promote jobs move cookbooks across environments
9. Berksfile site :opscode ! group :community do # opscode community cookbook 'nginx', '= 2.0.8' cookbook 'runit', '= 1.4.0' cookbook 'yum', '= 2.4.4' cookbook 'git', '= 2.8.4' cookbook 'pipeline', git: 'git@github.com:stephenlauck/pipeline.git' end ! group :sml do cookbook 'thresholder_pipeline', git: 'git@github.com:thresholderio/thresholder_pipeline.git' cookbook 'quasar_app', git: 'git@github.com:thresholderio/quasar_app.git' cookbook 'appliation_ruby', git: 'git@github.com:stephenlauck/application_ruby', branch: 'fix_integration' end Example berksfile
10. Continuous Delivery reads a master berksfile creates jenkins job per cookbook watches for changes on cookbooks runs tests uploads cookbook to chef server/org artifact? ! creates spiceweasel job watches for changes on yml for chef artifacts (data_bags, environments and roles) uploads artifacts
11. Berksfile Job sudo chef-client reads a master berksfile creates jenkins job per cookbook watches for changes on cookbooks runs tests uploads cookbook to chef server/org artifact? ! creates spiceweasel job watches for changes on yml for chef artifacts (data_bags, environments and roles) uploads artifacts
12. Cookbook Job foodcritic -f correctness . if [ -f Berksfile.lock ]; then berks update -c /var/lib/jenkins/.berkshelf/config.json else berks install -c /var/lib/jenkins/.berkshelf/config.json fi berks upload -c /var/lib/jenkins/.berkshelf/config.json reads a master berksfile creates jenkins job per cookbook watches for changes on cookbooks runs tests uploads cookbook to chef server/org artifact? ! creates spiceweasel job watches for changes on yml for chef artifacts (data_bags, environments and roles) uploads artifacts
13. Deployment export PATH=&quot;/opt/chef/embedded/bin:$PATH&quot; /opt/chef/embedded/bin/spiceweasel -e <%= @yml_file %> -c /var/lib/jenkins/.chef/<%= @chef_org %>.rb --novalidation if [ -f Berksfile.lock ]; then /opt/chef/embedded/bin/berks update -c /var/lib/jenkins/.berkshelf/<%= @chef_artifact_server %>- config.json else /opt/chef/embedded/bin/berks install -c /var/lib/jenkins/.berkshelf/<%= @chef_artifact_server %>- config.json fi /opt/chef/embedded/bin/berks upload -c /var/lib/jenkins/.berkshelf/<%= @chef_org %>-config.json /opt/chef/embedded/bin/berks apply <%= @env %> -c /var/lib/jenkins/.berkshelf/<%= @chef_org %>- config.json create promote jobs per environment AND/OR chef server/org uploads cookbooks/artifacts from one chef server/org to another pin versions in environments
14. Deployment spiceweasel upload berks install from artifact chef server berks upload to PROD chef server berks apply PROD create promote jobs per environment AND/OR chef server/org uploads cookbooks/artifacts from one chef server/org to another pin versions in environments
15. * file[/var/lib/jenkins/.chef/chef-zero-validator.pem] action create [2014-01-21T22:15:48+00:00] INFO: Processing file[/var/lib/ jenkins/.chef/chef-zero-validator.pem] action create (pipeline::knife line 36) [2014-01-21T22:15:48+00:00] INFO: file[/var/lib/jenkins/.chef/chef-zero-validator.pem] created file /var/lib/jenkins/.chef/chef-zero- validator.pem - create new file /var/lib/jenkins/.chef/chef-zero-validator.pem [2014-01-21T22:15:48+00:00] INFO: file[/var/lib/jenkins/.chef/chef-zero-validator.pem] updated file contents /var/lib/jenkins/.chef/chef- zero-validator.pem - update content in file /var/lib/jenkins/.chef/chef-zero-validator.pem from none to 6fe8f1 --- /var/lib/jenkins/.chef/chef-zero-validator.pem 2014-01-21 22:15:48.136061549 +0000 +++ /tmp/.chef-zero-validator.pem20140121-1306-1s3u300 2014-01-21 22:15:48.136061549 +0000 @@ -1 +1,28 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEogIBAAKCAQEAyd4VInXBK+XlGFSkFv80Kgifr8/5LOi/HBF3VhLRxM/GkgnN +fds/lhzNq3NH3ZjDhMbS5vOisrVW00DgvlyzsFJAjVGKNEW+S9BqTiYAQzmpRp6Y +QRYAm5sEuhZgY5m7WWuFrW/zayR2wllanirBdIlfY9TnF+w1NecKAqbW4QH5XIWk +dri2aq/wW9to7X0mdotAhZxfZ5sRPoyN9Kno0qjqJ6+zzmRUOY76Rq3CD4FDYEMJ +v3hMT2yOzqavLq2rUaApENLrnKd9SJXFrnuhfhKGZ21hTVn4kkiD5BAHst6k8+m9 +j3KcJXmHXYkQjNME8JZ3iwpBtN+xKyoknYGRNwIDAQABAoIBACFbz8ZIC0oDzZ39 +rrgWKDqh/jGBfr3LIHm08TGKHpwVcc0ETa70okdeLyacAE5ARl8UtBlyPXqmuNhk +Kj+K9i63CO/Rf7Mvq0jAAjEz2mtBhhWjc6mdxy/vqBJQTFFpQCqAuDB3BZS5C98G +ARGOIzXs1ZSbxCyR3iEwMtlJVM0NyQj7XOEMvbPXll3ODYuhLIYCFzRYE9Doa2QX +XDwTTQ6e2NtEUI2l9PgV1l27oQxuKFQ9EiZdCj74BgsNuoUcB45ZvSsFWbvXJdhD +lCoO8oTG7cR/hrf/LhdyFck/AaX0zqIfkfN0YFhUmXkbR9pDgibNsRIoSrAcAE1c +f08/umECgYEA+DTqCdfAQzwGtFbXSeqGimVZwEKCvjPYnJLcJJAVNUzBCXW0TOF9 ++DynHEU4o/FwRPfm2H9UC/ebzJnUAvsBMT4R86mlk2rmogeuXw1X5w6iIzLR/nPs Fail ! keys, auth, credentials users deploy/jenkins/github cookbook dependency and berkshelf upstream cookbooks
16. Improve better ssh key management partials for job commands LWRP pipeline git tagging / metadata bump berkshelf DSL test before merge data bags for chef orgs documentation (derp) better ssh key management partials all providers tagging back berkshelf DSL test before merge data in data bags how to test the pipeline cookbook
17. Patterns wrapper cookbook berkshelf / spiceweasel manifest vagrant / test-kitchen cookbook / wrapper cookbook berkshelf / spiceweasel manifest vagrant / test-kitchen
18. Actual Code https://github.com/stephenlauck/pipeline https://github.com/stephenlauck/pipeline_chef https://github.com/stephenlauck/wrapper_pipeline
19. Questions? cookbook / wrapper cookbook berkshelf / spiceweasel manifest vagrant / test-kitchen