Super MUG - Vagrant + MongoDB

Spinning up customized MongoDB instances quickly and easily.

Presented at London Super MUG: http://www.meetup.com/London-MongoDB-User-Group/events/152227472/.

Transcript

1. Vagrant + MongoDB Consulting Engineer, MongoDB Inc. James Tan #supermug
2. The Problem
7. vs Pets Cattle
8. •  Pets are given names like pussinboots.acme.com •  They are unique, lovingly hand raised and cared for •  When they get ill, you nurse them back to health •  Cattle are given numbers like vm001.acme.com •  Almost identical to other cattle •  When they get ill, you get another one
9. Infrastructure as Code •  Repeatable •  Scalable •  Maintainable
10. Solution?
12. vagrant-mongodb •  Optimized MongoDB instance in minutes –  RAID, file system, read-ahead, mount options, separate data, journal, and log partitions, etc. •  Fully customized –  Monitoring tools (dstat, htop, sysstat), Utils (tmux, mosh)
13. vagrant-mongodb •  Easy to customize and extend –  Fork it –  Change platform, OS, RAID config, disk settings, software versions, configs, etc. •  Easy to share and reproduce exact environment
14. Why Vagrant? What is Vagrant?
15. Providers Provisioners
16. Vagrantfile config.vm.provider :aws do |aws, override| aws.region = ENV['VAGRANT_AWS_REGION'] || 'eu-west-1' aws.keypair_name = ENV['VAGRANT_AWS_KEYPAIR_NAME'] || ENV['USER'] aws.instance_type = ENV['VAGRANT_AWS_INSTANCE_TYPE'] || 'm1.medium' # See http://aws.amazon.com/ec2/instance-types/#instance-details # the instance types that support this. aws.ebs_optimized = false # List of Amazon Linux AMIs (e.g. amzn-ami-pv-2013.09.0.x86_64-ebs) # Add the one for your region if it is missing. aws.region_config 'ap-southeast-1', :ami => 'ami-14f2b946' aws.region_config 'eu-west-1', :ami => 'ami-149f7863' aws.region_config 'us-east-1', :ami => 'ami-35792c5c' override.ssh.username = 'ec2-user' override.ssh.private_key_path = ENV['VAGRANT_SSH_PRIVATE_KEY_PATH'] || "~/aws/keys/#{region}/#{ENV['USER']}.pem” ... end
17. Why Chef?
18. vs
19. Vagrantfile config.omnibus.chef_version = :latest ... config.vm.provision :chef_solo do |chef| chef.add_recipe 'utils' chef.add_recipe 'mosh' chef.add_recipe 'ebs' chef.add_recipe 'mongodb::10gen_repo' chef.add_recipe 'mongodb' ... end
20. Berksfile site :opscode cookbook 'aws' cookbook 'delayed_evaluator' cookbook 'ebs', git: 'git://github.com/jamestyj/chef-ebs.git' cookbook 'mongodb', git: 'git://github.com/edelight/chef-mongodb.git' cookbook 'mosh', git: 'git://github.com/jtimberman/mosh-cookbook' cookbook 'yum'
21. Configuration
22. Vagrantfile config.vm.provision :chef_solo do |chef| ... chef.json = { :mongodb => { :dbpath => '/data', :smallfiles => true # Faster pre-allocation }, :ebs => { :access_key => ENV['AWS_ACCESS_KEY'], :secret_key => ENV['AWS_SECRET_KEY'], :fstype => 'ext4', # Or 'xfs' :md_read_ahead => 32, # No. of 512B sectors } } end
23. Vagrantfile (without RAID) config.vm.provision :chef_solo do |chef| ... chef.json[:ebs][:volumes] = { '/data' => { :size => 20, # Size in GB :fstype => chef.json[:ebs][:fstype], :mount_options => 'noatime,noexec', # :piops => 2000, # Provisioned IOPS # :uselvm => true } } end
24. Vagrantfile (with RAID) config.vm.provision :chef_solo do |chef| ... chef.json[:ebs][:raids] = { '/dev/md0' => { :num_disks => 4, :disk_size => 10, # Size in GB :raid_level => 10, :fstype => chef.json[:ebs][:fstype], :mount_point => '/data', :mount_options => 'noatime,noexec', # :piops => 2000, # Provisioned IOPS # :uselvm => true } } end
25. Customizations
26. my_cookbooks/utils/recipes/default.rb # Install monitoring tools package 'dstat' package 'htop' package 'sysstat' # Install other useful utils package 'tmux' package 'tree' ...
27. my_cookbooks/utils/recipes/default.rb ... execute 'install_dstat_with_mongodb_plugin' do command 'wget -P /usr/share/dstat/ https://raw.github.com/gianpaj/…' not_if { FileTest.directory?('/usr/share/dstat/') } end execute 'install_mongo_hacker' do command [ 'wget -P /tmp https://github.com/TylerBrock/mongo-hacker/…', 'unzip /tmp/master.zip -d /tmp/', 'cd /tmp/mongo-hacker-master', 'make', 'ln mongo_hacker.js /home/ec2-user/.mongorc.js', 'chown ec2-user: /home/ec2-user/.mongorc.js', 'rm -rf /tmp/{mongo-hacker-master,master.zip}' ].join(' && ') not_if { ::File.exists?('/home/ec2-user/.mongorc.js') } end ...
28. Demo
32. Use Cases e.g. https://github.com/jamestyj/benchmark
34. What’s next? Slides: http://speakerdeck.com/u/jamestyj Code: https://github.com/jamestyj/vagrant-mongodb
35. Thank You Consulting Engineer, MongoDB Inc. James Tan #supermug