Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Joy of Cooking: Deploy MongoDB with Chef

The Joy of Cooking: Deploy MongoDB with Chef

A quick primer on Opscode Chef and an introduction to using it to deploy MongoDB.

Nathen Harvey

June 26, 2012
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. The Joy of Cooking
    Deploying MongoDB with Chef
    Nathen Harvey, CustomInk.com
    https://github.com/nathenharvey/deploying-
    mongo-with-chef
    @nathenharvey
    @nathenharvey

    View full-size slide

  2. Agenda
    Infrastructure as Code
    Introduction to Chef
    Building a project in Chef
    Deploying MongoDB with Chef
    Additional resources
    @nathenharvey

    View full-size slide

  3. Evolution of Server
    Provisioning
    Just build it
    Keep notes in server.txt
    Migrate notes to wiki
    Custom shell scripts (in git)
    Systems integration framework
    @nathenharvey

    View full-size slide

  4. Infrastructure as Code
    Enable the reconstruction of the business
    from nothing but
    a source code repository
    an application data backup
    and bare metal resources
    -Jesse Robins, Opscode
    @nathenharvey

    View full-size slide

  5. Chef
    Declarative - What, not how
    Idempotent - Only take action if required
    Convergent - Takes care of itself
    @nathenharvey

    View full-size slide

  6. Developer Happiness
    MongoDB makes developers happy
    MongoDB makes operations happy
    Chef makes developers & operations
    happy, too!
    @nathenharvey

    View full-size slide

  7. Building a Chef Project
    First, come up with your policy /
    specification
    Abstract the resources in your spec
    @nathenharvey

    View full-size slide

  8. Resources
    package "mongodb-10gen" do
    version node['mongodb']['version']
    action :install
    end
    user node["mongodb"]["user"] do
    comment "MongoDB Server"
    gid node["mongodb"]["group"]
    supports :manage_home => true
    home "/home/#{node["mongodb"]["user"]}"
    action [ :create, :manage ]
    end
    More resources... @nathenharvey

    View full-size slide

  9. Building a Chef Project
    First, come up with your policy /
    specification
    Abstract the resources in your spec
    Write recipes
    @nathenharvey

    View full-size slide

  10. Recipes
    include_recipe 'mongodb::users'
    service node['mongodb']['service_name'] do
    supports [:start, :stop, :restart]
    end
    template node['mongodb']['configfile'] do
    source "mongodb.conf.erb"
    cookbook "mongodb"
    variables(
    :dbpath => node['mongodb']['dbpath'],
    :logpath => node['mongodb']['logpath'],
    :port => node['mongodb']['port']
    )
    end @nathenharvey

    View full-size slide

  11. Building a Chef Project
    First, come up with your policy /
    specification
    Abstract the resources in your spec
    Write recipes
    Package recipes in cookbooks
    @nathenharvey

    View full-size slide

  12. Cookbooks
    cookbooks/mongodb
    |-- README.md
    |-- attributes
    | `-- default.rb
    |-- metadata.rb
    |-- recipes
    | |-- 10gen_repo.rb
    | |-- configure.rb
    | |-- install.rb
    | |-- mms.rb
    | |-- replica_set.rb
    | `-- users.rb
    `-- templates
    `-- default
    `-- mongodb.conf.erb @nathenharvey

    View full-size slide

  13. Building a Chef Project
    First, come up with your policy /
    specification
    Abstract the resources in your spec
    Write recipes
    Package recipes in cookbooks
    Apply recipes to nodes
    @nathenharvey

    View full-size slide

  14. Nodes
    Representation of a host
    runs the Chef client
    has attributes
    has a list of recipes to be applied
    @nathenharvey

    View full-size slide

  15. Building a Chef Project
    First, come up with your policy /
    specification
    Abstract the resources in your spec
    Write recipes
    Package recipes in cookbooks
    Apply recipes to nodes
    Group things into roles @nathenharvey

    View full-size slide

  16. Roles
    mechanism for easily composing sets of
    functionality
    have attributes and a list of recipes to be
    applied
    @nathenharvey

    View full-size slide

  17. Roles
    name "mongodc"
    description "Sample MongoDB Replica Set for Mo
    default_attributes(
    "mongodb" => {
    "replicaset" => "mongodc"
    }
    )
    run_list(
    "recipe[mongodb::install]",
    "recipe[mongodb::configure]",
    "recipe[mongodb::replica_set]"
    )
    @nathenharvey

    View full-size slide

  18. Building a Chef Project
    First, come up with your policy /
    specification
    Abstract the resources in your spec
    Write recipes
    Package recipes in cookbooks
    Apply recipes to nodes
    Group things into roles @nathenharvey

    View full-size slide

  19. What is Chef?
    Server - API, search, Web UI
    Client - chef-client
    Command line tool - knife
    @nathenharvey

    View full-size slide

  20. knife
    $ knife help list
    bootstrap
    client
    configure
    cookbook
    cookbook-site
    data-bag
    environment
    exec
    index
    knife
    node
    role
    search
    shef
    ssh
    status
    tag
    @nathenharvey

    View full-size slide

  21. What is Chef?
    Server - API, search, Web UI
    Client - chef-client
    Command line tool - knife
    Inspection library - ohai
    @nathenharvey

    View full-size slide

  22. ohai
    Collects detailed, extensible information about a host.
    {
    "uptime": "13 days 06 hours 16 minutes 02 se
    "platform": "ubuntu",
    "os_version": "2.6.32-38-generic",
    "cpu": {
    "total": 3,
    "real": 0,
    "2": {
    "cache_size": "4096 KB",
    "model": "2",
    "family": "6",
    ...
    @nathenharvey

    View full-size slide

  23. What is Chef?
    Server - API, search, Web UI
    Client - chef-client
    Command line tool - knife
    Inspection library - ohai
    REPL - shef
    @nathenharvey

    View full-size slide

  24. What is Chef?
    Server - API, search, Web UI
    Client - chef-client
    Command line tool - knife
    Inspection library - ohai
    REPL - shef
    Community
    @nathenharvey

    View full-size slide

  25. community.opscode.com
    @nathenharvey

    View full-size slide

  26. Community Site
    Publish and share cookbooks
    @nathenharvey

    View full-size slide

  27. Community Site
    Publish and share plugins for Chef, Knife,
    and Ohai
    @nathenharvey

    View full-size slide

  28. chef-repo directory
    chef-repo
    |-- .chef
    | |-- knife.rb
    | |-- rubynation-validator.pem
    | `-- rubynation.pem
    |-- README.md
    |-- Rakefile
    |-- certificates
    |-- config
    | `-- rake.rb
    |-- cookbooks
    |-- data_bags
    |-- environments
    `-- roles
    @nathenharvey

    View full-size slide

  29. Provision a server
    knife ec2 server create
    knife rackspace server create
    Vagrant
    @nathenharvey

    View full-size slide

  30. Vagrantfile
    Vagrant::Config.run do |config|
    config.vm.define :mongo01 do |mongo01|
    mongo01.vm.provision :chef_client do |chef
    chef.add_role("mongodc")
    chef.node_name = "mongo01"
    end
    end
    config.vm.define :mongo02 do |mongo02|
    mongo02.vm.provision :chef_client do |chef
    chef.add_role("mongodc")
    chef.node_name = "mongo02"
    end
    end
    end @nathenharvey

    View full-size slide

  31. Deploying MongoDB
    Install
    Configure
    Run
    Backup
    Monitor
    @nathenharvey

    View full-size slide

  32. Create a Cookbook
    $ knife cookbook create mongodb
    ** Creating cookbook mongodb
    ** Creating README for cookbook: mongodb
    ** Creating metadata for cookbook: mongodb
    @nathenharvey

    View full-size slide

  33. Write our recipes
    10gen_repo.rb
    install.rb
    configure.rb
    users.rb
    replica_set.rb
    @nathenharvey

    View full-size slide

  34. Get the code
    https://github.com/nathenharvey/deploying-
    mongo-with-chef
    @nathenharvey

    View full-size slide

  35. 10gen Repo Recipe
    include_recipe "apt"
    execute "apt-get update" do
    action :nothing
    end
    apt_repository "10gen" do
    keyserver "keyserver.ubuntu.com"
    key "7F0CEB10"
    uri "http://downloads-distro.mongodb.org/rep
    action :add
    notifies :run, "execute[apt-get update]",
    end
    @nathenharvey

    View full-size slide

  36. Install Recipe
    include_recipe "mongodb::10gen_repo"
    package node['mongodb']['package_name'] do
    version node['mongodb']['version']
    action :install
    end
    @nathenharvey

    View full-size slide

  37. Where does package_name
    come from?
    Attributes defined in the cookbook:
    case platform
    when 'debian', 'ubuntu'
    default['mongodb']['package_name'] = 'mongod
    when 'centos', 'redhat', 'fedora', 'amazon'
    default['mongodb']['package_name'] = 'mongo-
    end
    @nathenharvey

    View full-size slide

  38. Configure Recipe
    template node['mongodb']['configfile'] do
    variables(
    :dbpath => node['mongodb']['dbpath'],
    :logpath => node['mongodb']['logpath'],
    :port => node['mongodb']['port'],
    :journal => node['mongodb']['journal'],
    :replication_set => node['mongodb']['repli
    )
    owner "root"
    group "root"
    mode "0644"
    notifies :restart, "service[#{node['mongodb'
    end
    @nathenharvey

    View full-size slide

  39. Configure Recipe
    The configuration template
    dbpath = <%= @dbpath %>
    logpath = <%= @logpath %>/mongodb.log
    logappend = true
    port = <%= @port %>
    noauth = true
    <% if @journal %>
    journal=true
    <% else -%>
    nojournal=true
    <% end -%>
    @nathenharvey

    View full-size slide

  40. Replication Recipe
    ruby_block "configure-replica-set" do
    block do
    require "rubygems"
    require "mongo"
    10.times do |try|
    begin
    conn = Mongo::Connection.new(
    "localhost", node['mongodb']['port'],
    :slave_ok => true, :connect_timeout =>
    rescue
    delay = 2 ** (try + 1)
    Chef::Log.info("Failed to connect to mongo
    sleep(delay)
    end
    end @nathenharvey

    View full-size slide

  41. Upload cookbooks to Chef
    server
    $ knife cookbook upload -a
    @nathenharvey

    View full-size slide

  42. Run chef-client
    Automatically
    knife ssh
    vagrant provision
    @nathenharvey

    View full-size slide

  43. Review
    Servers provisioned and communicating
    with the Chef API
    MongoDB running in a replica set
    @nathenharvey

    View full-size slide

  44. But wait, there's more!
    Encrypted databags
    Environments
    Lightweight Resources and Providers
    (LWRP)
    Exception and report handlers
    @nathenharvey

    View full-size slide

  45. Deploying MongoDB
    Deployment Preparedness
    MongoDB on Amazon EC2
    Operations Best Practices
    Production Notes
    @nathenharvey

    View full-size slide

  46. MongoDB Cookbooks
    http://community.opscode.com/cookbooks/mong
    https://github.com/miketheman/fullstack
    @nathenharvey

    View full-size slide

  47. Want more?
    http://community.opscode.com
    http://wiki.opscode.com
    Opscode Training Materials
    #chef on irc.freenode.net
    @nathenharvey

    View full-size slide

  48. Want even more?
    http://foodfightshow.org
    Episode 5: Getting Started with Chef
    @nathenharvey

    View full-size slide

  49. Shameless Plugs
    DevOpsDC
    Washington DC MongoDB Users Group
    @nathenharvey

    View full-size slide

  50. Find Me
    @nathenharvey
    http://nathenharvey.com
    [email protected]
    @nathenharvey

    View full-size slide