Configuration Management and Deployment in the Cloud

Transcript

1. Configuration Management and Deployment in the Cloud example: Scala
2. Contents • Motivation • Overview • Puppet • Development ◦ Vagrant • Continuous Delivery ◦ AWS • Conclusion
3. Puppet
5. Motivation
6. Problems • hard to maintain / document • hard to change running system • hard to track changes • hard to prove that system is in the correct state • not really scalable for multiple machines
7. Puppet Destinations, Not Journeys Define 1 2 Simulate 3 Enforce 4 Reports
8. Puppet: Resources • Resource abstractions: platform independent ◦ Types ▪ users ▪ files ▪ services ▪ software packages ◦ Attributes ▪ ensure => present, running, stopped… ▪ home => ‘/home/peter’
9. Puppet: Manifests • Definitions are done in manifestes • Describe the expected state • Example: # /root/examples/file-1.pp file {'testfile': path => '/tmp/testfile', ensure => present, mode => 0640, content => "I'm a test file.", } • Run with puppet apply /root/examples/file-1.pp
10. Example $sbt_url = "http://repo.scala-sbt.org/scalasbt/sbt-native-packages/org/scala-sbt/sbt/0.13.0/sbt.deb" package { "openjdk-7-jdk": ensure => "installed", } package { "curl": ensure => "installed", } exec { "sbt.deb": command => "/usr/bin/curl -O $sbt_url", cwd => "/home/vagrant", creates => "/home/vagrant/sbt.deb", require => [ Package["curl"], Package["openjdk-7-jdk"], ], } package { "sbt.db": ensure => installed, provider => dpkg, source => "/home/vagrant/sbt.deb", require => Exec["sbt.deb"], }
12. Puppet Modularization class apache ($version = 'latest') { package {'httpd': ensure => $version, # Get version from the class declaration before => File['/etc/httpd.conf'], } file {'/etc/httpd.conf': ensure => file, owner => 'httpd', content => template('apache/httpd.conf.erb'), # Template from a module } service {'httpd': ensure => running, enable => true, subscribe => File['/etc/httpd.conf'], } }
13. Puppet Forge • repository of modules written by the community • common packages like mysql, apache, etc
14. Puppet Master Master DB Application Webserver
15. Puppet Master node 'db1.example.com' { include common include mysql } node ‘webserver.example.com' { include common include nginx } node /^www\d+.application.com$/ { include common include application }
16. Puppet: Conclusion • Purposes and application scenarios • Good things were mentioned, time for the bad ones ◦ Master / Agent ▪ Sometimes not possible for agents to reach the master (Firewalls, vendor politics, proxies, whatever) • extensive Documentation / Learning puppet
17. Development
18. Continuous Integration & Delivery Dev. Overview GIT-Repository Laptop Vagrant Dev. VM My favorite dev-tools App App GIT-Repository Vagrant Files ISO Images Fedora
19. Dev. - Motivation “It was running fine on my PC” • Goal: one development environment for all developers. • automated deployment, updates and install of this development environment
20. Dev. - Vagrant • Vagrant: Describe in Config-File • run: vagrant up ◦ Get required images ◦ Setup and run virtual machines ◦ Install required software (incl. described versions) ◦ make application code in VM available
21. Dev. - Vagrant • Boxes: • Vagrant uses special boxes for each VM, where changes are stored separate from the original image • Benefit: you can run several different VM’s without an OS-image for every VM
22. Dev. - Benefits • you are able to use your favorite environment, but the code runs in the VM • extremely fastened environment setup, e.g. for new employees. • Less space used when using several VM’s.
23. Dev. - Example # -*- mode: ruby -*- # vi: set ft=ruby : VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "precise32" config.vm.box_url = "http://files.vagrantup.com/precise32.box" config.vm.network :forwarded_port, guest: 9000, host: 9000 config.vm.provision :puppet do |puppet| puppet.manifests_path = "manifests" puppet.manifest_file = "site.pp" end end That’s it! The rest is Puppet
24. Dev. - Example $sbt_url = "http://repo.scala-sbt.org/scalasbt/sbt-native-packages/org/scala-sbt/sbt/0.13.0/sbt.deb" exec { "apt-update": command => "/usr/bin/apt-get update" } Exec["apt-update"] -> Package <| |> package { "openjdk-7-jdk": ensure => "installed",} package { "curl": ensure => "installed",} exec { "sbt.deb": command => "/usr/bin/curl -O $sbt_url", cwd => "/home/vagrant", creates => "/home/vagrant/sbt.deb", require => [ Package["curl"], Package["openjdk-7-jdk"], ],} package { "sbt.db": ensure => installed, provider => dpkg, source => "/home/vagrant/sbt.deb", require => Exec["sbt.deb"],} site.pp
25. Conclusion • Vagrant provides a very promising solution for handling development environments (at larger or often changing teams)
26. Configuration Management in the Cloud AWS
27. Brief Introduction to AWS • Elastic Compute Cloud (EC2) ◦ Instances with resizable compute capacity ◦ Auto-Scaling ◦ Tags | User Data ◦ EC2 is ephemeral (non-persistent) • Amazon Machine Image (AMI) • Tons of other services
28. What does Vagrant/Puppet offer? $ vagrant plugin install vagrant-aws ...configuration as before*… $ vagrant up --provider=aws *With some AWS specific additions...
29. Typical Cloud Application Architecture
30. So Vagrant/Puppet is not enough? • What if I need more than just an EC2 instance? Load Balancer, persistent Storage etc. • What if I want an exact copy of my stack?
31. Amazon CloudFormation CloudFormation is an infrastructure provisioning tool based on a declarative language and expressed in JSON. Example: https://s3.amazonaws.com/cloudformation-templates-us-east-1/Tracks_Single_Instance.template
32. CloudFormation
33. CloudFormer • CloudFormer takes your existing AWS resources and makes a template out of them • Once created, you can customize
34. CloudFormation and Puppet Puppet CloudFormation • overlap • CloudFormation is AWS only • Puppet is universal
35. Vagrant and Packer (packer.io) • Provisioning at runtime is too slow and makes the application unsecure • Solution: Pre-Bake AMIs • Packer is a tool for creating identical machine images from a single source configuration
36. Conclusion • Jenkins to mix it all together and deploy GIT BUILD CREATE ENV TEST PASS? DESTROY ENV PROMOTE
37. Questions?