Eat the Whole Bowl: Building a Full-Stack Installer with Omnibus

An introduction to Omnibus, Opscode's tool for creating cross-platform full-stack installers.

Video recording of the presentation available here:
http://www.youtube.com/watch?v=q8iJAntXCNY

Transcript

1. EAT THE WHOLE BOWL
   Building a Full-Stack Installer with Omnibus
   Seth Chisamore
   Christopher Maier
   Opscode
   Engineering
   

   View Slide

2. THE
   FULL STACK
   

   View Slide

3. View Slide

4. View Slide

5. TWO
   MEN
   HAD A
   DREAM
   

   View Slide

6. I was playing 80's power
   rock ballads. Probably
   wrote much of it while I
   was listening to Warrant's
   “Heaven”.
   

   View Slide

7. I called it “Fatty”,
   because it was making
   “fat” binary packages.
   I didn't really realize I
   was being offensive until
   later.
   

   View Slide

8. How about “Omnibus”?
   

   View Slide

9. *hugs*
   

   View Slide

10. View Slide

11. HOW
    OPSCODE
    USES
    OMNIBUS
    

    View Slide

12. A LOOK INSIDE
    

    View Slide

13. PROJECT DSL
    

    View Slide

14. name "chef-server"
    maintainer "Opscode, Inc."
    homepage "http://www.opscode.com"
    replaces "chef-server"
    install_path "/opt/chef-server"
    build_version Omnibus::BuildVersion.new.semver
    build_iteration 1
    dependency "postgresql"
    dependency "rabbitmq"
    dependency "chef-solr"
    dependency "erchef"
    # more...
    exclude "\.git*"
    exclude "bundler\/git"
    

    View Slide

15. SOFTWARE DSL
    

    View Slide

16. name "erchef"
    version "1.2.6"
    dependency "erlang"
    dependency "rsync"
    dependency "curl"
    source :git => "git://github.com/opscode/erchef"
    relative_path "erchef"
    env = {
    "PATH" => "#{install_dir}/embedded/bin:#{ENV["PATH"]}",
    "LDFLAGS" => "-L#{install_dir}/embedded/lib
    -I#{install_dir}/embedded/include",
    # more...
    }
    continued...
    

    View Slide

17. build do
    command "make distclean", :env => env
    command "make rel", :env => env
    command "mkdir -p
    #{install_dir}/embedded/service/erchef"
    command "#{install_dir}/embedded/bin/rsync -a --delete
    --exclude=.git/*** --exclude=.gitignore
    ./rel/erchef/
    #{install_dir}/embedded/service/erchef/"
    command "rm -rf
    #{install_dir}/embedded/service/erchef/log"
    end
    

    View Slide

18. COMMAND
    LINE
    INTERFACE
    

    View Slide

19. PROJECT
    

    View Slide

20. $ omnibus project foobar
    create foobar/config/software/README.md
    create foobar/Gemfile
    create foobar/.gitignore
    create foobar/config/projects/foobar.rb
    create foobar/README.md
    create foobar/package-scripts/foobar/makeselfinst
    create foobar/package-scripts/foobar/postinst
    create foobar/package-scripts/foobar/postrm
    ...
    

    View Slide

21. BUILD
    PROJECT
    

    View Slide

22. $ omnibus build project foobar
    Building foobar
    mkdir -p /var/cache/omnibus/build/opt_foobar
    mkdir -p /var/cache/omnibus/src/preparation
    touch /var/cache/omnibus/build/opt_foobar/preparation.fetch
    [builder:preparation] building preparation
    [builder:preparation] preparation build succeeded,0.000755s
    touch /var/cache/omnibus/build/opt_foobar/preparation.manif
    mkdir -p /var/cache/omnibus/src/version-manifest
    touch /var/cache/omnibus/build/opt_foobar/version-manifest.
    [builder:version-manifest] building version-manifest
    [builder:version-manifest] version-manifest build succeeded
    ...
    

    View Slide

23. BUILD
    SOFTWARE
    

    View Slide

24. $ omnibus build software foobar version-manifest
    Building version-manifest for foobar project
    [builder:preparation] building preparation
    [builder:preparation] preparation build succeeded,
    0.000383s
    touch /var/cache/omnibus/build/opt_foobar/
    preparation.manifest
    [builder:version-manifest] building version-manifest
    [builder:version-manifest] version-manifest build
    succeeded, 0.000229s
    touch /var/cache/omnibus/build/opt_foobar/
    version-manifest.manifest
    ...
    

    View Slide

25. CLEAN
    

    View Slide

26. $ omnibus clean foobar --purge
    remove /var/cache/omnibus/src/preparation
    remove /var/cache/omnibus/src/version-manifest
    remove /var/cache/omnibus/build/opt_foobar
    remove /var/cache/omnibus/build/opt_foobar/preparation.fet
    remove /var/cache/omnibus/build/opt_foobar/preparation.man
    remove /var/cache/omnibus/build/opt_foobar/version-manifes
    remove /var/cache/omnibus/build/opt_foobar/version-manifes
    remove /var/cache/omnibus/pkg/foobar-0.0.0+20130405022120_
    remove /opt/foobar
    

    View Slide

27. ROADMAP
    

    View Slide

28. TESTING
    

    View Slide

29. DEPENDENCY
    MANAGEMENT
    name "opscode-push-jobs-server"
    maintainer "Opscode, Inc."
    homepage "http://www.opscode.com"
    replaces "opscode-push-jobs-server"
    install_path "/opt/opscode-push-jobs-server"
    build_version Omnibus::BuildVersion.new.semver
    build_iteration 1
    runtime_dependency "private-chef"
    # creates required build directories
    dependency "preparation"
    ...
    

    View Slide

30. (REAL) RE-USE
    • * Software definitions become specialized cookbooks
    • * These cookbooks live in standalone Git repositories
    source 'https://rubygems.org'
    gem 'omnibus', '~> 1.0.0'
    gem 'omnibus-software', :git => 'git://github.com/opscode/omnibus-software.git'
    Current:
    Future:
    

    View Slide

31. RELEASE COMMAND
    • Based on lessons learned from Omnitruck (think install.sh)
    • Will perform checksumming (md5, sha256)
    • Initially will push packages to an S3 bucket
    • Would like to support “real” repos using something like PRM
    

    View Slide

32. View Slide

33. THANKS
    _cwm_
    christophermaier
    cm@opscode.com
    schisamo
    schisamo
    schisamo@opscode.com
    

    View Slide

34. Photo Credits
    Noodle Bowl http://www.insockmonkeyslippers.com/swiss-chard-and-noodle-soup-with-poached-egg
    Suitcases http://www.flickr.com/photos/rubyblossom/7413242342/
    Stones http://www.flickr.com/photos/erix/4645279944/
    Worried Man http://www.flickr.com/photos/pedrosimoes7/145220445/
    Skateboard http://www.flickr.com/photos/14degrees/2714047400/
    Gears http://www.flickr.com/photos/wwarby/4782904694/
    Blueprints http://www.flickr.com/photos/pulpolux/838363289/
    Tool Shed Interior http://www.flickr.com/photos/matthigh/4050429719/
    Compass Rose http://www.flickr.com/photos/marfis75/5374308475/
    Measuring Tape http://www.flickr.com/photos/hippie/2466583497/
    Factory Pipes http://www.flickr.com/photos/cycloctopus/8181800285/
    Glass Bottles http://www.flickr.com/photos/afropicmusing/8089260503/
    Freddie Mercury Statue http://www.flickr.com/photos/nick_c/112111560/
    

    View Slide