Vendorificator

Lightning talk on FOSDEM 2013 introducing/demoing Vendorificator, a tool to manage third-party upstream dependencies in a git repository.

Transcript

1. Vendorificator Maciej Pasternacki <maciej@pasternacki.net> 3ofcoins @mpasternacki
2. github.com/3ofcoins/vendorificator Vendorificator – “Vendor Everything” ”google this! – Keep upstream dependencies right there with your code, in your repository – Keep track of what came when, from where, and what changes were applied
3. github.com/3ofcoins/vendorificator Vendorificator – Works with Git. Written in Ruby. – Pristine branch for each upstream module. – Annotations for people and machines.
4. github.com/3ofcoins/vendorificator Vendorificator $ gem install vendorificator $ git init $ $EDITOR Vendorfile ! ! ! ! ! ! # or config/vendor.rb $ vendor sync
5. github.com/3ofcoins/vendorificator chef_cookbook_ignore_dependencies %w|mysql rsyslog| archive :virtualenv, :url => 'http://…/virtualenv-1.8.4.tar.gz', :version => '1.8.4' chef_cookbook 'omnibus_updater' chef_cookbook 'lvm' chef_cookbook 'munin' class << git 'git://github.com/mpasternacki/nagios.git', :branch => 'COOK-1997', :category => :cookbooks, :version => '0.20130124.2' include Vendorificator::Hooks::ChefCookbookDependencies end Vendorfile
6. github.com/3ofcoins/vendorificator v-demo|master ⇒ be vendor status new virtualenv/1.8.4 new omnibus_updater/0.1.2 WARNING Metadata of omnibus_updater does not exist at … /metadata.rb, could not gather dependencies new lvm/0.8.4 WARNING Metadata of lvm does not exist at …/metadata.rb, could not gather dependencies new munin/1.1.2 WARNING Metadata of munin does not exist at …/metadata.rb, could not gather dependencies new nagios/0.20130124.2 WARNING Metadata of nagios does not exist at …/metadata.rb, could not gather dependencies
7. github.com/3ofcoins/vendorificator gather dependencies new munin/1.1.2 WARNING Metadata of munin does not exist at /Users/japhy/tmp/v-demo/vendor/cookbooks/munin/metadata.rb, could not gather dependencies new nagios/0.20130124.2 WARNING Metadata of nagios does not exist at /Users/japhy/tmp/v-demo/vendor/cookbooks/nagios/metadata.rb, coul d not gather dependencies v-demo|master ⇒ be vendor sync module virtualenv fetching virtualenv/1.8.4 download http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.8.4.tar.gz unpack virtualenv-1.8.4.tar.gz tag vendor/virtualenv/1.8.4 module omnibus_updater fetching omnibus_updater/0.1.2 download http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/3005/original/omn ibus_updater.tgz?1359690436 unpack omnibus_updater.tgz tag vendor/cookbooks/omnibus_updater/0.1.2 module lvm fetching lvm/0.8.4 download http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/2607/original/lvm .tgz?1355345356 unpack lvm.tgz tag vendor/cookbooks/lvm/0.8.4 module munin fetching munin/1.1.2 download http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/2460/original/mun in.tgz?1353970236 unpack munin.tgz tag vendor/cookbooks/munin/1.1.2 module nagios fetching nagios/0.20130124.2 clone git://github.com/mpasternacki/nagios.git tag vendor/cookbooks/nagios/0.20130124.2 module apache2 fetching apache2/1.4.2 download http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/2801/original/apa che2.tgz?1357667719 unpack apache2.tgz tag vendor/cookbooks/apache2/1.4.2 module nginx fetching nginx/1.2.0 download http://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/2947/original/ngi
8. github.com/3ofcoins/vendorificator v-demo|master ⇒ be vendor status up to date virtualenv/1.8.4 up to date omnibus_updater/0.1.2 up to date lvm/0.8.4 up to date munin/1.1.2 up to date nagios/0.20130124.2 up to date apache2/1.4.2 up to date nginx/1.2.0 up to date apt/1.8.4 up to date build-essential/1.3.2 up to date php/1.1.4 up to date nginx_simplecgi/0.1.0 up to date ohai/1.1.6 up to date xml/1.1.2 up to date perl/1.1.0 up to date runit/1.0.0 up to date bluepill/2.2.0
9. github.com/3ofcoins/vendorificator v-demo|master ⇒ git branch * master vendor/cookbooks/apache2 vendor/cookbooks/apt vendor/cookbooks/bluepill vendor/cookbooks/build-essential vendor/cookbooks/lvm vendor/cookbooks/munin vendor/cookbooks/nagios vendor/cookbooks/nginx vendor/cookbooks/nginx_simplecgi vendor/cookbooks/ohai vendor/cookbooks/omnibus_updater vendor/cookbooks/perl vendor/cookbooks/php vendor/cookbooks/runit vendor/cookbooks/xml vendor/virtualenv
10. github.com/3ofcoins/vendorificator v-demo|master ⇒ git tag vendor/cookbooks/apache2/1.4.2 vendor/cookbooks/apt/1.8.4 vendor/cookbooks/bluepill/2.2.0 vendor/cookbooks/build-essential/1.3.2 vendor/cookbooks/lvm/0.8.4 vendor/cookbooks/munin/1.1.2 vendor/cookbooks/nagios/0.20130124.2 vendor/cookbooks/nginx/1.2.0 vendor/cookbooks/nginx_simplecgi/0.1.0 vendor/cookbooks/ohai/1.1.6 vendor/cookbooks/omnibus_updater/0.1.2 vendor/cookbooks/perl/1.1.0 vendor/cookbooks/php/1.1.4 vendor/cookbooks/runit/1.0.0 vendor/cookbooks/xml/1.1.2 vendor/virtualenv/1.8.4
11. github.com/3ofcoins/vendorificator v-demo|master ⇒ git lg --color | cat * c633c6c (HEAD, master) Merge branch 'vendor/cookbooks/bluepill' |\ | * 2d7f57d (tag: vendor/cookbooks/bluepill/2.2.0, vendor/cookbooks/bluepill) Conjured cookbook bluepill version 2.2 .0 Origin: http://s3.amazonaws.com/ community-files.opscode.com/cookbook_versions/tarballs/2806/original/ bluepill.tgz ?1357674185 Checksum: d59ab8d1785264a06ebf89f08f377a25e98292ce10084d5c94cfcf5947b4cc61 * d24472c Merge branch 'vendor/cookbooks/runit' |\ | * 1c79974 (tag: vendor/cookbooks/runit/1.0.0, vendor/cookbooks/runit) Conjured cookbook runit version 1.0.0 Origin : http://s3.amazonaws.com/ community-files.opscode.com/cookbook_versions/tarballs/2959/original/ runit.tgz?1359413695 Checksum: 14f4f217fc7cd39cdcd83994437e199d65e5fe7401f9df5f1af2de825fa34bb3 * 50bfb63 Merge branch 'vendor/cookbooks/perl' |\ | * 37c0698 (tag: vendor/cookbooks/perl/1.1.0, vendor/cookbooks/perl) Conjured cookbook perl version 1.1.0 Origin: http://s3.amazonaws.com/community- files.opscode.com/cookbook_versions/tarballs/2652/original/perl.tgz?1355811654 Chec ksum: dedd691eed22df67d574b40595eb838ec8a67791399fe9cb7aa2fef9f3153dfd * 33ebf2d Merge branch 'vendor/cookbooks/xml' |\ | * aa2dac5 (tag: vendor/cookbooks/xml/1.1.2, vendor/cookbooks/xml) Conjured cookbook xml version 1.1.2 Origin: http ://s3.amazonaws.com/community- files.opscode.com/cookbook_versions/tarballs/2638/original/xml.tgz?1355781843 Checksum : 911cf6bce431288f0b58ff9be06662e588e634d29f0ee6f91e52ff384c07d420 …
12. github.com/3ofcoins/vendorificator v-demo|master ⇒ tree -L 2 vendor/ vendor/ ├── cookbooks │ ├── apache2 │ ├── apt │ ├── bluepill │ ├── build-essential │ ├── lvm │ ├── munin │ ├── nagios │ ├── nginx │ ├── nginx_simplecgi │ ├── ohai │ ├── omnibus_updater │ ├── perl │ ├── php │ ├── runit │ └── xml └── virtualenv ├── AUTHORS.txt ├── LICENSE.txt ├── MANIFEST.in ├── PKG-INFO ├── README.rst ├── bin ├── docs ├── scripts ├── setup.cfg ├── setup.py ├── virtualenv.egg-info ├── virtualenv.py ├── virtualenv_embedded └── virtualenv_support
13. github.com/3ofcoins/vendorificator archive :virtualenv, :url => 'http://…/virtualenv-1.8.4.tar.gz', :version => '1.8.4-1' do FileUtils::rm_rf Dir.entries('.') - %w|.. . AUTHORS.txt LICENSE.txt README.rst virtualenv.py| end Vendorfile
14. github.com/3ofcoins/vendorificator up to date nginx_simplecgi/0.1.0 up to date ohai/1.1.6 up to date xml/1.1.2 up to date perl/1.1.0 up to date runit/1.0.0 up to date bluepill/2.2.0 v-demo|master ⇒ git commit -a --amend [master 259318e] modify Vendorfile 1 files changed, 6 insertions(+), 1 deletions(-) v-demo|master ⇒ be vendor status outdated virtualenv/1.8.4-1 up to date omnibus_updater/0.1.2 up to date lvm/0.8.4 up to date munin/1.1.2 up to date nagios/0.20130124.2 up to date apache2/1.4.2 up to date nginx/1.2.0 up to date apt/1.8.4 up to date build-essential/1.3.2 up to date php/1.1.4 up to date nginx_simplecgi/0.1.0 up to date ohai/1.1.6
15. github.com/3ofcoins/vendorificator up to date build-essential/1.3.2 up to date php/1.1.4 up to date nginx_simplecgi/0.1.0 up to date ohai/1.1.6 up to date xml/1.1.2 up to date perl/1.1.0 up to date runit/1.0.0 up to date bluepill/2.2.0 v-demo|master ⇒ be vendor sync module virtualenv fetching virtualenv/1.8.4-1 download http://pypi.python.org/packages/source/ unpack virtualenv-1.8.4.tar.gz tag vendor/virtualenv/1.8.4-1 module omnibus_updater up to date omnibus_updater/0.1.2 module lvm up to date lvm/0.8.4 module munin up to date munin/1.1.2 module nagios up to date nagios/0.20130124.2 module apache2 up to date apache2/1.4.2 module nginx
16. github.com/3ofcoins/vendorificator v-demo|master vendor/ ├── cookbooks │ └── … └── virtualenv ├── AUTHORS.txt ├── LICENSE.txt ├── README.rst └── virtualenv.py
17. github.com/3ofcoins/vendorificator v-demo|master ⇒ git lg * a9c5ac0 (HEAD, master) Merge branch 'vendor/virtualenv' |\ | * 7268996 (tag: vendor/virtualenv/1.8.4-1, vendor/virtualenv) Conjured archive virtualenv from virtualenv-1.8.4.ta r.gz Origin: http://pypi.python.org/packages/source/v/virtualenv/ virtualenv-1.8.4.tar.gz Checksum: a16ee99f4a3b72be0 4704b8bd28c2f21d510af545492501ca3cd50fde9a70cb6 Version: 1.8.4-1 * | 259318e modify Vendorfile * | c779a86 Merge branch 'vendor/virtualenv' |\ \ |/ | * af6b67c (tag: vendor/virtualenv/1.8.4) Conjured archive virtualenv from virtualenv-1.8.4.tar.gz Origin: http://p ypi.python.org/packages/source/v/virtualenv/ virtualenv-1.8.4.tar.gz Checksum: a16ee99f4a3b72be04704b8bd28c2f21d510af 545492501ca3cd50fde9a70cb6 Version: 1.8.4 * d35f013 Vendorfile
18. github.com/3ofcoins/vendorificator v-demo|master ⇒ git tag vendor/cookbooks/apache2/1.4.2 … vendor/cookbooks/xml/1.1.2 vendor/virtualenv/1.8.4 vendor/virtualenv/1.8.4-1
19. github.com/3ofcoins/vendorificator Now let’s add and commit a small change in vendor/cookbooks/munin/metadata.rb…
20. github.com/3ofcoins/vendorificator v-demo|master ⇒ be vendor diff unchanged virtualenv/1.8.4-1 changed munin/1.1.2 diff --git c/vendor/cookbooks/munin/metadata.rb w/vendor/cookbooks/ munin/metadata.rb index b6e10db..5ed1c58 100644 --- c/vendor/cookbooks/munin/metadata.rb +++ w/vendor/cookbooks/munin/metadata.rb @@ -1,3 +1,5 @@ +# test change + name "munin" maintainer "Opscode, Inc." maintainer_email "cookbooks@opscode.com" unchanged nagios/0.20130124.2
21. github.com/3ofcoins/vendorificator v-demo|master ⇒ be vendor log unchanged virtualenv/1.8.4-1 changed munin/1.1.2 commit 2f51d8167bf32c730dceb6ff3dd1915fbed89ea8 Author: Maciej Pasternacki <maciej@pasternacki.net> Date: Sun Feb 3 14:07:18 2013 +0100 test change unchanged nagios/0.20130124.2
22. github.com/ 3ofcoins/vendorificator