Linecook (A Chef Alternative)

Slides from my talk at DeRailed in Denver on April 26, 2011

Transcript

1. Linecook (A Chef Alternative) Simon Chiang Tuesday, April 26, 2011
2. Chef Tuesday, April 26, 2011
3. Chef Ruby Tuesday, April 26, 2011
4. Chef Ruby Opscode Tuesday, April 26, 2011
5. Chef Ruby Opscode “Systems Integration Framework” Tuesday, April 26, 2011
6. Chef Ruby Opscode “Systems Integration Framework” Open Source Tuesday, April 26, 2011
7. Chef Ruby Opscode “Systems Integration Framework” Automated Open Source Tuesday, April 26, 2011
8. Chef Ruby Opscode “Systems Integration Framework” Automated Scalable Open Source Tuesday, April 26, 2011
9. Chef Ruby Opscode “Systems Integration Framework” Automated Scalable Open Source Server Provisioning Tuesday, April 26, 2011
10. Chef Ruby Opscode “Systems Integration Framework” Automated Scalable Open Source Community Server Provisioning Tuesday, April 26, 2011
11. Chef Ruby Opscode “Systems Integration Framework” Automated Scalable Hotness Open Source Community Server Provisioning Tuesday, April 26, 2011
12. An alternative to shell scripts Tuesday, April 26, 2011 Not dismissing Chef when I say that. The advantages over manually writing setup/maintenance script are not to be underestimated. But for the most part things you do with Chef are things you would otherwise do with shell scripts, and there are problems.
13. Never a quick Quickstart “Before installing Chef, you should take a moment to understand the various "flavors" of chef: client-server, chef-solo, and the Opscode Platform. Deciding which one is right for you will impact your installation process. You may also want to take a quick look at Chef's architecture to get an idea of what you're installing before you proceed.” http:/ /wiki.opscode.com/display/chef/Installation Tuesday, April 26, 2011
14. Not a script, but sort of... [lib/chef/provider/package/zypper.rb] @ 0.10.0.rc.0 def install_package(name, version) if version run_command( :command => "zypper -n --no-gpg-checks install -l #{name}=#{version}" ) Tuesday, April 26, 2011
15. Not a script, but sort of... [lib/chef/provider/package/zypper.rb] @ 0.10.0.rc.0 def install_package(name, version) if version run_command( :command => "zypper -n --no-gpg-checks install -l #{name}=#{version}" ) Fixed options :( Tuesday, April 26, 2011
16. Not a script, but sort of... [lib/chef/provider/package/zypper.rb] @ 0.10.0.rc.0 def install_package(name, version) if version run_command( :command => "zypper -n --no-gpg-checks install -l #{name}=#{version}" ) https:/ /github.com/opscode/chef/pull/27 Error for version="" Fixed options :( Tuesday, April 26, 2011
17. Many moving parts http:/ /wiki.opscode.com/display/chef/Architecture Tuesday, April 26, 2011
18. Many moving parts http:/ /wiki.opscode.com/display/chef/Vagrant http:/ /wiki.opscode.com/display/chef/Architecture Tuesday, April 26, 2011
19. Many moving parts http:/ /wiki.opscode.com/display/chef/Vagrant http:/ /wiki.opscode.com/display/chef/Architecture http:/ /community.opscode.com/cookbooks Tuesday, April 26, 2011
20. Alternative? Tuesday, April 26, 2011
21. Shell Scripts! Tuesday, April 26, 2011
22. Linecook A Shell Script Generator Tuesday, April 26, 2011
23. Established Tools (bash, ssh, gems, VirtualBox) Tuesday, April 26, 2011 Just use the existing systems. Some ‘advanced’ techniques but all established, known systems Prior experience applies any learning will feed back into existing workflow
24. Start with a Script End with a Script Tuesday, April 26, 2011 Able to bring an existing script to Linecook, then rework to make it more powerful or maintainable. You will work with scripts directly. Again all prior experience applies.
25. Pretty quick Quickstart Make a server Make a script Run script on server Tuesday, April 26, 2011
26. Pretty quick Quickstart Make a server Make a script Run script on server 1 Tuesday, April 26, 2011
27. Pretty quick Quickstart Make a server Make a script Run script on server 1 2 Tuesday, April 26, 2011
28. Pretty quick Quickstart Make a server Make a script Run script on server 1 2 3 Tuesday, April 26, 2011
29. An ERB Trick... Tuesday, April 26, 2011
30. How to Make a Server VirtualBox Install OS SSH Key Exchange localhost Tuesday, April 26, 2011
31. How to Make a Server VirtualBox Install OS SSH Key Exchange VirtualBox localhost Tuesday, April 26, 2011
32. How to Make a Server VirtualBox Install OS SSH Key Exchange VirtualBox abox-ubuntu localhost Tuesday, April 26, 2011
33. How to Make a Server VirtualBox Install OS SSH Key Exchange VirtualBox abox-ubuntu localhost Tuesday, April 26, 2011
34. abox-ubuntu localhost VirtualBox Port Forwarding Access VM by ssh to localhost ssh -p 2220 linecook@localhost VBoxManage modifyvm abox --natpf1 'abox-ssh,tcp,, 2220,,22' Tuesday, April 26, 2011
35. abox-ubuntu localhost bbox-ubuntu Lather, rinse Repeat as needed ssh -p 2220 linecook@localhost ssh -p 2221 linecook@localhost ... VBoxManage modifyvm bbox --natpf1 'bbox-ssh,tcp,, 2221,,22' Tuesday, April 26, 2011
36. Advantages No special prerequisites on servers Fast Access (fast enough for TDD) Cmdline control Multiple VMs Snapshots Tuesday, April 26, 2011
37. How to Run Scripts cat > script.sh <<"DOC" echo "# $(whoami)@$(hostname): hello world!" DOC chmod +x script.sh scp -P 2220 script.sh linecook@localhost:/tmp/script.sh ssh -p 2220 linecook@localhost -- /tmp/script.sh # linecook@abox-ubuntu: hello world! Tuesday, April 26, 2011
38. Redundany cat > script.sh <<"DOC" echo "# $(whoami)@$(hostname): hello world!" DOC chmod +x script.sh scp -P 2220 script.sh linecook@localhost:/tmp/script.sh ssh -p 2220 linecook@localhost -- /tmp/script.sh # linecook@abox-ubuntu: hello world! Tuesday, April 26, 2011
39. Use SSH Config mkdir config cat > config/ssh <<"DOC" Host abox Port 2220 User linecook Hostname localhost DOC scp -F config/ssh script.sh abox:/tmp/script.sh ssh -F config/ssh abox -- /tmp/script.sh # linecook@abox-ubuntu: hello world! Tuesday, April 26, 2011
40. Multiple Hosts cat > config/ssh <<"DOC" Host abox Port 2220 Host bbox Port 2221 Host * User linecook Hostname localhost DOC scp -F config/ssh script.sh bbox:/tmp/script.sh ssh -F config/ssh bbox -- /tmp/script.sh # linecook@bbox-ubuntu: hello world! Tuesday, April 26, 2011
41. Redundancy cat > config/ssh <<"DOC" Host abox Port 2220 Host bbox Port 2221 Host * User linecook Hostname localhost DOC scp -F config/ssh script.sh bbox:/tmp/script.sh ssh -F config/ssh bbox -- /tmp/script.sh # linecook@bbox-ubuntu: hello world! Tuesday, April 26, 2011
42. Redundancy cat > config/ssh <<"DOC" Host abox Port 2220 Host bbox Port 2221 Host * User linecook Hostname localhost DOC scp -F config/ssh script.sh bbox:/tmp/script.sh ssh -F config/ssh bbox -- /tmp/script.sh # linecook@bbox-ubuntu: hello world! Tuesday, April 26, 2011
43. Make a “Package” Run with Linecook [config/ssh] Host abox Port 2220 Host bbox Port 2221 Host * User linecook Hostname localhost [packages/abox/script.sh] echo "# $(whoami)@$(hostname): hello world!" [packages/bbox/script.sh] echo "# $(whoami)@$(hostname): Hullo Wurld!" linecook run --script script.sh --remote-dir /tmp abox bbox # linecook@abox-ubuntu: hello world! # linecook@bbox-ubuntu: Hullo Wurld! Tuesday, April 26, 2011
44. Leverge Defaults [config/ssh] Host abox Port 2220 Host bbox Port 2221 Host * User linecook Hostname localhost [packages/abox/run] echo "# $(whoami)@$(hostname): hello world!" [packages/bbox/run] echo "# $(whoami)@$(hostname): Hullo Wurld!" linecook run # linecook@abox-ubuntu: hello world! # linecook@bbox-ubuntu: Hullo Wurld! script: run remote dir: ~/linecook hosts: * Tuesday, April 26, 2011
45. Easy Way To Test! [packages/abox/run] echo "hello world" > /tmp/message.txt [packages/abox/test] if [ $(cat /tmp/message.txt) == "hello world" ] then echo "# success" else echo "# fail" fi linecook run --script test # fail linecook run linecook run --script test # success Tuesday, April 26, 2011
46. Cmdline Dev Cycle linecook start linecook run linecook run --script test linecook ssh abox linecook snapshot modified linecook stop linecook start --snapshot modified ... Tuesday, April 26, 2011
47. Advantages Standard use of SSH One standard config file Ordinary inputs (directories, scripts) Multiple VMs Flexible! Tuesday, April 26, 2011
48. How to Make a Script Tuesday, April 26, 2011
49. How to Make a Script (boring alert) Tuesday, April 26, 2011
50. Start with a Script [packages/abox/run] echo "# I will not manually configure my server" linecook run # I will not manually configure my server Tuesday, April 26, 2011
51. Convert to Recipe [packages/abox.yml] linecook: package: recipes: run: abox [recipes/abox.rb] target <<"SCRIPT" echo "# I will not manually configure my server" SCRIPT A package file Tuesday, April 26, 2011
52. Convert to Recipe [packages/abox.yml] linecook: package: recipes: run: abox [recipes/abox.rb] target <<"SCRIPT" echo "# I will not manually configure my server" SCRIPT A tempfile (packages/abox/run) Tuesday, April 26, 2011
53. Simplify [packages/abox.yml] {} If default, no manifest needed [recipes/abox.rb] target <<"SCRIPT" echo "# I will not manually configure my server" SCRIPT Tuesday, April 26, 2011
54. Build and Run [packages/abox.yml] {} [recipes/abox.rb] target <<"SCRIPT" echo "# I will not manually configure my server" SCRIPT linecook build linecook run # I will not manually configure my server Tuesday, April 26, 2011
55. Ok... that was boring. Tuesday, April 26, 2011 Seems pointless, but here is where it gets cool.
56. Use Recipe as Context for ERB Tuesday, April 26, 2011
57. ERB Compiles to Ruby require 'erb' compiler = ERB::Compiler.new("<>") compiler.put_cmd = "target<<" compiler.insert_cmd = "target<<" compiler.compile "got <%= obj %>" # => "target<<\"got \"; target<<(( obj ).to_s)" Tuesday, April 26, 2011
58. InstanceEval for Context class Recipe attr_accessor :target def initialize @target = "" end def obj "milk" end end code = "target<<\"got \"; target<<(( obj ).to_s)" recipe = Recipe.new recipe.instance_eval(code) recipe.target # => "got milk" Tuesday, April 26, 2011
59. Make a Module module Helper def get(obj) target<<"got "; target<<(( obj ).to_s) end end recipe = Recipe.new recipe.extend Helper recipe.instance_eval %q{ get "milk" target << ", " get "cookies" } recipe.target # => "got milk, got cookies" Tuesday, April 26, 2011
60. Make a Module module Helper def get(obj) target<<"got "; target<<(( obj ).to_s) end end recipe = Recipe.new recipe.extend Helper recipe.instance_eval %q{ get "milk" target << ", " get "cookies" } recipe.target # => "got milk, got cookies" This is a recipe! Tuesday, April 26, 2011
61. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" Tuesday, April 26, 2011
62. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" [lib/example.rb] module Example # Write an echo ... def echo(str) target<< "echo ";... end end Tuesday, April 26, 2011
63. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" [lib/example.rb] module Example # Write an echo ... def echo(str) target<< "echo ";... end end Tuesday, April 26, 2011
64. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" [lib/example.rb] module Example # Write an echo ... def echo(str) target<< "echo ";... end end Tuesday, April 26, 2011
65. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" [lib/example.rb] module Example # Write an echo ... def echo(str) target<< "echo ";... end end Tuesday, April 26, 2011
66. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" [lib/example.rb] module Example # Write an echo ... def echo(str) target<< "echo ";... end end Tuesday, April 26, 2011
67. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" [lib/example.rb] module Example # Write an echo ... def echo(str) target<< "echo ";... end end Tuesday, April 26, 2011
68. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" require "example" extend Example Tuesday, April 26, 2011
69. Helpers [helpers/example/echo.erb] Write an echo statement (str) -- echo "<%= str %>" [recipes/abox.rb] helpers "example" echo "# I will not manually configure my server" linecook build linecook run # I will not manually configure my server Tuesday, April 26, 2011
70. Capture (no write) [helpers/example/color.erb] Add color to a string (color, str) codes = Hash[*%W{red 0;31 white 1;37 blue 0;34}] -- \033[<%= codes[color.to_s] %>m<%= str %>\033[0m [recipes/abox.rb] helpers "example" msg = "# I will not manually configure my server" echo _color("blue", msg) Tuesday, April 26, 2011
71. Capture (no write) [helpers/example/color.erb] Add color to a string (color, str) codes = Hash[*%W{red 0;31 white 1;37 blue 0;34}] -- \033[<%= codes[color.to_s] %>m<%= str %>\033[0m [recipes/abox.rb] helpers "example" msg = "# I will not manually configure my server" echo _color("blue", msg) Prefix with underscore String output used as input Tuesday, April 26, 2011
72. Capture (no write) [helpers/example/color.erb] Add color to a string (color, str) codes = Hash[*%W{red 0;31 white 1;37 blue 0;34}] -- \033[<%= codes[color.to_s] %>m<%= str %>\033[0m [recipes/abox.rb] helpers "example" msg = "# I will not manually configure my server" echo _color("blue", msg) linecook build linecook run # I will not manually configure my server Tuesday, April 26, 2011
73. Recipes are Ruby [recipes/abox.rb] helpers "example" msg = "# I will not manually configure my server" 3.times do echo _color("blue", msg) end linecook build linecook run # I will not manually configure my server # I will not manually configure my server # I will not manually configure my server Tuesday, April 26, 2011
74. Advantages Easily extensible DSL Produces ordinary modules Test as any other Ruby Distribute as Gems (versions, bundler) Reprocessing of output Also, kind of cool... Tuesday, April 26, 2011
75. Linebook (Standard Library for Linecook) Tuesday, April 26, 2011
76. Logic + Pipelines [recipes/demo.rb] helpers 'linebook/shell' unless_ _file?('/tmp/message') do cat.to('/tmp/message').heredoc do writeln 'hello world!' end end cat('/tmp/message') Tuesday, April 26, 2011
77. Logic + Pipelines [packages/demo/run] if ! [ -f "/tmp/message" ] then cat > /tmp/message << HEREDOC_0 hello world! HEREDOC_0 fi cat "/tmp/message" Tuesday, April 26, 2011
78. Under Construction Exit status checks, ‘stack trace’ Helpers for login/su User and File Management Server-side testing (assert_script) Installs, config, deployments, etc. Tuesday, April 26, 2011
79. No need to wait! Tuesday, April 26, 2011
80. Linecook Recipes Packages Server Attributes Helpers Files Templates Tuesday, April 26, 2011
81. Learn More GitHub: pinnacol/linecook.git pinnacol/linebook.git Me: github.com/thinkerbot @thinkerbot Tuesday, April 26, 2011
82. Thanks Pinnacol! Tuesday, April 26, 2011
83. Thanks Derailed! Tuesday, April 26, 2011
84. Questions? Tuesday, April 26, 2011