Best tool to make config mgmt? ● cfengine ● puppet ● chef ● salt ● ansible -- Business decision-making. Make your choice -- Strengths/weakness. Tend to converge.
What is Chef? ● a company ○ Opscode Inc. ○ getchef.com ○ @chef ● a product (with paid support) ○ packages, not recommended install from gemfiles ● an open source project ● a community (open source but not only)
Initial problem Problem to be solved (in aprox. 2 weeks): 1. install chef-server 11 in a centos6 behind a VPN 2. install 2 web frontends + 2 backends a. (redis s/m + mysql m/s)
Real problem 1. refactor/migrate/write from scratch about 20 old-fashioned (2009) but very reliable (daily used, very difficult to extend for an outsider) ubuntu ruby18-webapp-related recipes to newer ones 2. proof of concept to migrate chef09 company’ s repository to anything better
Criteria to use community cookbooks? https://github.com/opscode-cookbooks/mysql When in doubt use popular opscode community cookbooks. So many content to follow!
1.- Hosting Environment (I) Open Source chef-server in a centos6. Modify and test undocumented chef-server config, with some “little” external problems: - shared server behind a VPN - unavailable ports: reassign 80/443 to 81/8443 - 8000 not opened in firewall (so no reports will be available)
2.- Cookbook Development (I) Opscode free(*) hosting ● Great help because Multiproject! ● http://learn.getchef.com ● Berkshelf helped a lot with dependencies (**) ● Good practices freezing versions (*) not open-source (limited to 10 nodes) (**) not so easy when not opscode server
2.- Cookbook Development (II) Chef-DK: ● ruby binaries in specific /opt folders ● not needed rvm/rbenv/ruby source code/... ● own path & pre-installed useful gems and tools
2.- Cookbook Development (III) ● http://jtimberman.housepub.org/blog/2014/04/30/chefdk- and-ruby/ ● Berkshelf 3.0. ● The Test Kitchen integration testing framework. ● ChefSpec, for unit testing cookbooks. ● Foodcritic, static code analysis on cookbooks. ● All of the Chef tools you're already familiar with: Chef Client, Knife, Ohai and Chef Zero.
2.- Cookbook Development (IV) Better practices? Any? ● No time for tests nor TDD (to learn about) ● Not enough RAM in laptop nor in AWS micro instances for making testing/CI with Vagrant ● Foodcritic and more, but later ● I hope to use them ASAP: just because I like it. The same with Puppet.
2.- Cookbook Development (V) First stages of development ● make risks, quick tests, quickwin, try and try ● think in advance, virtualhosts library ● ugly code you know it will be easy to change ● the community code is better than yours ● extend, not create from the ground ● parametrize cookbooks: redis
3.- Monolithic cookbook approach Thanks a lot to Mathias Lafeldt @mlafeldt “monolithic cookbook” idea, easy to refactor later (see Puppet Module Structure Redux). ● http://mlafeldt.github.io/practicing-ruby- cookbook ● https://github.com/elm-city- craftworks/practicing-ruby-cookbook
5.-ProductionCookbookDevelopment ● A new beginning. Almost from scratch. Develop new recipes and cookbooks against a new chef organization in opscode free hosting. ● install, fix, check-apply and repeat in frontend staging instances (Poor’s man plan–do–check–adjust) https://en.wikipedia.org/wiki/PDCA
6.-ProductionCookbookDeployment ● chef site cookbook install “cookbook” (new git branch in ~/chef-repo/cookbook/) ● chef site cookbook install chef-client
7.- Opscode packages (I) ● WARNING: chefdk.rpm installs binaries under /usr/bin and gems in its own gemsdir ● REMOVE any puppet or ruby preexistent package or binary, no interferences please! ● BEWARE: never use binaries without explicit PATH e.g. /opt/chef/bin/chef-client
7.- Opscode packages (II) NO RVM nor system ruby allowed ● chef-server.rpm ● chef.rpm ● chefdk.rpm every opscode rpm package installs one or more ruby binaries in its own path
8.- Developers isolated ruby environment (I) ● Developers need multiple rubies installed (1.9.3 & 2.0.X) ● unprivileged users will install its own gems, without interfering with chef ruby-binaries SOLUTION: compile explicit versions from source code, rewrite PATH in user environment and install “bundler gem” as root.
8.- Developers isolated ruby environment (II) Opscode chef.rpm package provides 2.1 ruby ● install passenger-apache library+binary compiled and installed as a gem under chef. rpm gemlib path (a community recipe is in charge of it) ● compatible to both 2.0.x and 1.9.x user rubies
8.- Developers isolated ruby environment(III) Explicit ruby version (1.9.3 or 2.0.X) in appropriate virtualhost apache files (from templates) NOTE: Foodcritic tool helped a lot when looking for errors (e.g. specially with chef templates)
9. Final Steps (I) After just 2 weeks, it almost worked ok. Traditional approach to development: logrotate and monit forked recipes with “999” suffix added to “semver” in metadata.rb ● not perfect ● Explicit installation of dependencies when in Opensource Opscode server
9. Final Steps (II) Pareto principle! 80/20 Rule. ● Two more weeks with “fringes” ● create new organization in opscode free account ● chef-solo-search for “local” databags ● BIG CRISIS: no chef-server available -> chef-solo deployment
9. Final Steps (III) Create git repo (first commit in 2 weeks) ● install explicit dependencies ● “chef cookbook site install” ● only needed “ancient” (fork and modify) approach in two cookbooks: monit and passenger
9. Final Steps (IV) “In extremis”: Refactor the “monolithic” cookbook Monolithic cookbook refactored as 3 cookbooks Three layers of attribute+template files: ● general cookbook (i.e. language, servers,...) ○ company cookbook (i.e. final customer specific) ■ project cookbook (i.e. virtualhost config)
10. Conclusions (I) To FIX: ● move logrotate recipes in railssites2 to another new bootstrapping-node-sysadmin cookbook ● not exactly applied but inspired in "everything is a cookbook"
10. Conclusions (II) ● You need to have “Luck” ○ no way to configure staging chef-server (only working through 127.0.0.1 and VPN) ○ no way to configure ubuntu from opscode deb packages ● You need big motivation to success
apenav
nttcom
yuj1osm
bengo4com
aoshun7
k1style
knsg16
smt7174
niftycorp
you
yoshiitaka
yuya4
nayuts
rocio
brad_frost
bryan
caitiem20
lara
samanthasiow
panda_program
wjessup
thoeni
reverentgeek
aki_iinuma
skipperchong