Slide 1

Slide 1 text

Vagrant + MongoDB Consulting Engineer, MongoDB Inc. James Tan #supermug

Slide 2

Slide 2 text

The Problem

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

vs Pets Cattle

Slide 8

Slide 8 text

•  Pets are given names like pussinboots.acme.com •  They are unique, lovingly hand raised and cared for •  When they get ill, you nurse them back to health •  Cattle are given numbers like vm001.acme.com •  Almost identical to other cattle •  When they get ill, you get another one

Slide 9

Slide 9 text

Infrastructure as Code •  Repeatable •  Scalable •  Maintainable

Slide 10

Slide 10 text

Solution?

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

vagrant-mongodb •  Optimized MongoDB instance in minutes –  RAID, file system, read-ahead, mount options, separate data, journal, and log partitions, etc. •  Fully customized –  Monitoring tools (dstat, htop, sysstat), Utils (tmux, mosh)

Slide 13

Slide 13 text

vagrant-mongodb •  Easy to customize and extend –  Fork it –  Change platform, OS, RAID config, disk settings, software versions, configs, etc. •  Easy to share and reproduce exact environment

Slide 14

Slide 14 text

Why Vagrant? What is Vagrant?

Slide 15

Slide 15 text

Providers Provisioners

Slide 16

Slide 16 text

Vagrantfile config.vm.provider :aws do |aws, override| aws.region = ENV['VAGRANT_AWS_REGION'] || 'eu-west-1' aws.keypair_name = ENV['VAGRANT_AWS_KEYPAIR_NAME'] || ENV['USER'] aws.instance_type = ENV['VAGRANT_AWS_INSTANCE_TYPE'] || 'm1.medium' # See http://aws.amazon.com/ec2/instance-types/#instance-details # the instance types that support this. aws.ebs_optimized = false # List of Amazon Linux AMIs (e.g. amzn-ami-pv-2013.09.0.x86_64-ebs) # Add the one for your region if it is missing. aws.region_config 'ap-southeast-1', :ami => 'ami-14f2b946' aws.region_config 'eu-west-1', :ami => 'ami-149f7863' aws.region_config 'us-east-1', :ami => 'ami-35792c5c' override.ssh.username = 'ec2-user' override.ssh.private_key_path = ENV['VAGRANT_SSH_PRIVATE_KEY_PATH'] || "~/aws/keys/#{region}/#{ENV['USER']}.pem” ... end

Slide 17

Slide 17 text

Why Chef?

Slide 18

Slide 18 text

vs

Slide 19

Slide 19 text

Vagrantfile config.omnibus.chef_version = :latest ... config.vm.provision :chef_solo do |chef| chef.add_recipe 'utils' chef.add_recipe 'mosh' chef.add_recipe 'ebs' chef.add_recipe 'mongodb::10gen_repo' chef.add_recipe 'mongodb' ... end

Slide 20

Slide 20 text

Berksfile site :opscode cookbook 'aws' cookbook 'delayed_evaluator' cookbook 'ebs', git: 'git://github.com/jamestyj/chef-ebs.git' cookbook 'mongodb', git: 'git://github.com/edelight/chef-mongodb.git' cookbook 'mosh', git: 'git://github.com/jtimberman/mosh-cookbook' cookbook 'yum'

Slide 21

Slide 21 text

Configuration

Slide 22

Slide 22 text

Vagrantfile config.vm.provision :chef_solo do |chef| ... chef.json = { :mongodb => { :dbpath => '/data', :smallfiles => true # Faster pre-allocation }, :ebs => { :access_key => ENV['AWS_ACCESS_KEY'], :secret_key => ENV['AWS_SECRET_KEY'], :fstype => 'ext4', # Or 'xfs' :md_read_ahead => 32, # No. of 512B sectors } } end

Slide 23

Slide 23 text

Vagrantfile (without RAID) config.vm.provision :chef_solo do |chef| ... chef.json[:ebs][:volumes] = { '/data' => { :size => 20, # Size in GB :fstype => chef.json[:ebs][:fstype], :mount_options => 'noatime,noexec', # :piops => 2000, # Provisioned IOPS # :uselvm => true } } end

Slide 24

Slide 24 text

Vagrantfile (with RAID) config.vm.provision :chef_solo do |chef| ... chef.json[:ebs][:raids] = { '/dev/md0' => { :num_disks => 4, :disk_size => 10, # Size in GB :raid_level => 10, :fstype => chef.json[:ebs][:fstype], :mount_point => '/data', :mount_options => 'noatime,noexec', # :piops => 2000, # Provisioned IOPS # :uselvm => true } } end

Slide 25

Slide 25 text

Customizations

Slide 26

Slide 26 text

my_cookbooks/utils/recipes/default.rb # Install monitoring tools package 'dstat' package 'htop' package 'sysstat' # Install other useful utils package 'tmux' package 'tree' ...

Slide 27

Slide 27 text

my_cookbooks/utils/recipes/default.rb ... execute 'install_dstat_with_mongodb_plugin' do command 'wget -P /usr/share/dstat/ https://raw.github.com/gianpaj/…' not_if { FileTest.directory?('/usr/share/dstat/') } end execute 'install_mongo_hacker' do command [ 'wget -P /tmp https://github.com/TylerBrock/mongo-hacker/…', 'unzip /tmp/master.zip -d /tmp/', 'cd /tmp/mongo-hacker-master', 'make', 'ln mongo_hacker.js /home/ec2-user/.mongorc.js', 'chown ec2-user: /home/ec2-user/.mongorc.js', 'rm -rf /tmp/{mongo-hacker-master,master.zip}' ].join(' && ') not_if { ::File.exists?('/home/ec2-user/.mongorc.js') } end ...

Slide 28

Slide 28 text

Demo

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Use Cases e.g. https://github.com/jamestyj/benchmark

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

What’s next? Slides: http://speakerdeck.com/u/jamestyj Code: https://github.com/jamestyj/vagrant-mongodb

Slide 35

Slide 35 text

Thank You Consulting Engineer, MongoDB Inc. James Tan #supermug