$30 off During Our Annual Pro Sale. View Details »

MadridDevops September 2014: "From chef09 to chef11, one approach to devops"

Antonio Peña
September 25, 2014

MadridDevops September 2014: "From chef09 to chef11, one approach to devops"

@MadridDevops September 2014: "From chef09 to chef11, one approach to devops"

Madrid, September 25, 2014

http://madrid.devops.es

An excuse to talk about devops, chef (and puppet as comparison), its community and a migration project I have been involved with.

Antonio Peña

September 25, 2014
Tweet

More Decks by Antonio Peña

Other Decks in Technology

Transcript

  1. From chef09 to chef11
    my approach to a real devops task
    Antonio Peña @apenav for @MadridDevops
    September 2014 http://about.me/apenav

    View Slide

  2. http://madrid.devops.es

    View Slide

  3. Motivation
    ● feedback
    ● sharing experience
    ● learning
    ● talking about devops

    View Slide

  4. View Slide

  5. View Slide

  6. Impostor Syndrome

    View Slide

  7. Three Parts
    ● Devops
    ● Chef
    ● Migration

    View Slide

  8. About
    DevOps

    View Slide

  9. Software is broken or will be

    View Slide

  10. Devops (I)

    View Slide

  11. Devops (II)

    View Slide

  12. Devops (III)

    View Slide

  13. So little time, so much to talk about

    View Slide

  14. Big Business. DevOps Myths (I)

    View Slide

  15. Big Business. DevOps Myths (II)

    View Slide

  16. My approach to a
    configuration management project
    Antonio Peña @apenav for @MadridDevops
    September 2014 http://about.me/apenav

    View Slide

  17. Best tool to make config mgmt?
    ● cfengine
    ● puppet
    ● chef
    ● salt
    ● ansible
    -- Business decision-making. Make your choice
    -- Strengths/weakness. Tend to converge.

    View Slide

  18. About
    Chef

    View Slide

  19. View Slide

  20. View Slide

  21. What is Chef?
    ● a company
    ○ Opscode Inc.
    ○ getchef.com
    ○ @chef
    ● a product (with paid support)
    ○ packages, not recommended install from gemfiles
    ● an open source project
    ● a community (open source but not only)

    View Slide

  22. Beginners, experts and middle class

    View Slide

  23. Berkshelf.I

    View Slide

  24. Berkshelf.II

    View Slide

  25. “Spanish” Way
    Master Chef RTVE

    View Slide

  26. Howto devops in Spain? (I)
    ● e.g. Demo in a Big Company

    View Slide

  27. Howto devops in Spain? (II)
    ● startup
    ● why not?
    ● sandbox

    View Slide

  28. Howto devops in Spain? (III)
    ● consulting

    View Slide

  29. Working in a company
    as a Developer.
    Daily tasks

    View Slide

  30. Initial problem
    Problem to be solved (in aprox. 2 weeks):
    1. install chef-server 11 in a centos6 behind a
    VPN
    2. install 2 web frontends + 2 backends
    a. (redis s/m + mysql m/s)

    View Slide

  31. Real problem
    1. refactor/migrate/write from scratch about 20
    old-fashioned (2009) but very reliable (daily
    used, very difficult to extend for an outsider)
    ubuntu ruby18-webapp-related recipes to
    newer ones
    2. proof of concept to migrate chef09 company’
    s repository to anything better

    View Slide

  32. From scratch
    http://devopsreactions.tumblr.com/post/97637875636/what-happens-when-i-am-allowed-to-rewrite-code-from

    View Slide

  33. Methodology or the like
    “Big bang” approach, but iterative:
    ● use chef11 community cookbooks
    ● almost no direct migration from chef09 ones

    View Slide

  34. Criteria to use community cookbooks?
    https://github.com/opscode-cookbooks/mysql
    When in doubt use popular
    opscode
    community
    cookbooks.
    So many content to
    follow!

    View Slide

  35. About
    Migration

    View Slide

  36. To the kitchen

    View Slide

  37. Let’s cook!
    1. Hosting Environment
    2. Cookbook Development
    3. Monolithic cookbook approach
    4. Everything is a cookbook
    5. Production Cookbook Deployment
    6. Production Cookbook Development
    7. Opscode packages
    8. Developers’ isolated ruby environment
    9. Final Steps
    10. Conclusions

    View Slide

  38. Chef Server
    Internals

    View Slide

  39. 1.- Hosting Environment (I)
    Open Source chef-server in a centos6.
    Modify and test undocumented chef-server
    config, with some “little” external problems:
    - shared server behind a VPN
    - unavailable ports: reassign 80/443 to 81/8443
    - 8000 not opened in firewall (so no reports will
    be available)

    View Slide

  40. Asking for opening ports in firewall
    http://devopsreactions.tumblr.com/post/41094252078/asking-the-security-team-for-a-firewall-exception

    View Slide

  41. 2.- Cookbook Development (I)
    Opscode free(*) hosting
    ● Great help because Multiproject!
    ● http://learn.getchef.com
    ● Berkshelf helped a lot with dependencies (**)
    ● Good practices freezing versions
    (*) not open-source (limited to 10 nodes)
    (**) not so easy when not opscode server

    View Slide

  42. 2.- Cookbook Development (II)
    Chef-DK:
    ● ruby binaries in specific /opt folders
    ● not needed rvm/rbenv/ruby source code/...
    ● own path & pre-installed useful gems and
    tools

    View Slide

  43. 2.- Cookbook Development (III)
    ● http://jtimberman.housepub.org/blog/2014/04/30/chefdk-
    and-ruby/
    ● Berkshelf 3.0.
    ● The Test Kitchen integration testing framework.
    ● ChefSpec, for unit testing cookbooks.
    ● Foodcritic, static code analysis on cookbooks.
    ● All of the Chef tools you're already familiar with: Chef
    Client, Knife, Ohai and Chef Zero.

    View Slide

  44. 2.- Cookbook Development (IV)
    Better practices? Any?
    ● No time for tests nor TDD (to learn about)
    ● Not enough RAM in laptop nor in AWS micro
    instances for making testing/CI with Vagrant
    ● Foodcritic and more, but later
    ● I hope to use them ASAP: just because I like
    it. The same with Puppet.

    View Slide

  45. 2.- Cookbook Development (V)
    First stages of development
    ● make risks, quick tests, quickwin, try and try
    ● think in advance, virtualhosts library
    ● ugly code you know it will be easy to change
    ● the community code is better than yours
    ● extend, not create from the ground
    ● parametrize cookbooks: redis

    View Slide

  46. 3.- Monolithic cookbook approach
    Thanks a lot to Mathias Lafeldt @mlafeldt
    “monolithic cookbook” idea, easy to refactor
    later (see Puppet Module Structure Redux).
    ● http://mlafeldt.github.io/practicing-ruby-
    cookbook
    ● https://github.com/elm-city-
    craftworks/practicing-ruby-cookbook

    View Slide

  47. 4.- Everything is a cookbook
    ● https://tomduffield.com/everything-as-a-
    cookbook-chefconf2014/
    ● https://speakerdeck.com/tduffield/everything-
    as-a-cookbook-1

    View Slide

  48. Cool guys don’t look back to explosions?

    View Slide

  49. 5.-ProductionCookbookDevelopment
    ● A new beginning. Almost from scratch.
    Develop new recipes and cookbooks against a
    new chef organization in opscode free hosting.
    ● install, fix, check-apply and repeat in
    frontend staging instances
    (Poor’s man plan–do–check–adjust)
    https://en.wikipedia.org/wiki/PDCA

    View Slide

  50. 6.-ProductionCookbookDeployment
    ● chef site cookbook install “cookbook” (new
    git branch in ~/chef-repo/cookbook/)
    ● chef site cookbook install chef-client

    View Slide

  51. 7.- Opscode packages (I)
    ● WARNING: chefdk.rpm installs binaries
    under /usr/bin and gems in its own gemsdir
    ● REMOVE any puppet or ruby preexistent
    package or binary, no interferences please!
    ● BEWARE: never use binaries without explicit
    PATH e.g. /opt/chef/bin/chef-client

    View Slide

  52. 7.- Opscode packages (II)
    NO RVM nor system ruby allowed
    ● chef-server.rpm
    ● chef.rpm
    ● chefdk.rpm
    every opscode rpm package installs one or
    more ruby binaries in its own path

    View Slide

  53. 8.- Developers isolated ruby environment (I)
    ● Developers need multiple rubies installed
    (1.9.3 & 2.0.X)
    ● unprivileged users will install its own gems,
    without interfering with chef ruby-binaries
    SOLUTION: compile explicit versions from
    source code, rewrite PATH in user environment
    and install “bundler gem” as root.

    View Slide

  54. 8.- Developers isolated ruby environment (II)
    Opscode chef.rpm package provides 2.1 ruby
    ● install passenger-apache library+binary
    compiled and installed as a gem under chef.
    rpm gemlib path (a community recipe is in
    charge of it)
    ● compatible to both 2.0.x and 1.9.x user
    rubies

    View Slide

  55. 8.- Developers isolated ruby environment(III)
    Explicit ruby version (1.9.3 or 2.0.X) in
    appropriate virtualhost apache files (from
    templates)
    NOTE: Foodcritic tool helped a lot when looking
    for errors (e.g. specially with chef templates)

    View Slide

  56. 9. Final Steps (I)
    After just 2 weeks, it almost worked ok.
    Traditional approach to development: logrotate
    and monit forked recipes with “999” suffix
    added to “semver” in metadata.rb
    ● not perfect
    ● Explicit installation of dependencies when in
    Opensource Opscode server

    View Slide

  57. 9. Final Steps (II)
    Pareto principle! 80/20 Rule.
    ● Two more weeks with “fringes”
    ● create new organization in opscode free
    account
    ● chef-solo-search for “local” databags
    ● BIG CRISIS: no chef-server available ->
    chef-solo deployment

    View Slide

  58. Angry chef?

    View Slide

  59. 9. Final Steps (III)
    Create git repo (first commit in 2 weeks)
    ● install explicit dependencies
    ● “chef cookbook site install”
    ● only needed “ancient” (fork and modify)
    approach in two cookbooks: monit and
    passenger

    View Slide

  60. 9. Final Steps (IV)
    “In extremis”: Refactor the “monolithic”
    cookbook
    Monolithic cookbook refactored as 3 cookbooks
    Three layers of attribute+template files:
    ● general cookbook (i.e. language, servers,...)
    ○ company cookbook (i.e. final customer specific)
    ■ project cookbook (i.e. virtualhost config)

    View Slide

  61. 9. Final Steps (V)
    ● only two “old fashioned” forked recipes:
    monit and passenger
    ● new recipes forces a different way to use
    databags

    View Slide

  62. 10. Conclusions (I)
    To FIX:
    ● move logrotate recipes in railssites2 to
    another new bootstrapping-node-sysadmin
    cookbook
    ● not exactly applied but inspired in
    "everything is a cookbook"

    View Slide

  63. 10. Conclusions (II)
    ● You need to have “Luck”
    ○ no way to configure staging chef-server (only
    working through 127.0.0.1 and VPN)
    ○ no way to configure ubuntu from opscode deb
    packages
    ● You need big motivation to success

    View Slide

  64. 10. Conclusions (III)
    To be improved:
    ● ubuntu/RHEL cookbooks
    ● tests
    ● mysql m/s cookbook
    ● monit recipe
    ● opensource chef-server hosting

    View Slide

  65. View Slide

  66. THANKS FOR YOUR
    PATIENCE

    View Slide