Deploy tooling overview

Go over many of the best of breed tools for cloud deployment. Provided a high level overview of each one with some code snippets.

Web version with working links can be found at:
https://docs.google.com/presentation/d/1_55mi9v9o0lDT_j76-kZD3lkI_9XC3lUEQ2OAKGH6eQ/pub?start=false&loop=false&delayms=3000

Transcript

1. Let’s deploy code by Charlie Knudsen
2. Provisioning & Orchestration • Provisioning - Getting infrastructure setup in a secure and fault tolerant manner ◦ Getting necessary VMs in place ◦ Firewall setup ◦ DNS setup • Orchestration - Pushing or pulling an application onto the provisioned infrastructure ◦ Pushing the actual binaries/code ◦ Targeting the proper VMs ◦ Configuring the applications and load balancers ◦ Running migrations Cloud deployment will often blur the lines between these tasks
3. PaaS vs IaaS • PaaS - Platform as a service ◦ A platform where you give them code and they handle both provisioning and orchestration ◦ Heroku, GAE, Azure, OpenShift, CloudFoundry, Deis/Flynn/Kubernetes ◦ Often deploy using git push ◦ Often opinionated in how applications function. (i.e. 12 Factor App) • IaaS - Infrastructure as a service ◦ A platform where APIs are provided to allow you to provision infrastructure which the service provider will host ◦ AWS, GCE, Rackspace, OpenStack
4. • Very well documented and easy to get started ◦ Heroku Getting Started, Java Heroku Documentation, Ratpack documentation, Heroku addons, Architecting Applications for Heroku • Very opinionated on how your application should function (i.e. 12Factor) ◦ Stateless applications ◦ Configured with environment variables ◦ Deployed via git push • Expensive when past the free tier • Definitely worth trying out and using for fun/learning/low traffic projects Heroku
5. IaaS Commonalities • All have the concept of a base image • All require the users to determine how the application will be scaled and configured • Most have a cloud init concept on startup • All have unique product offerings and features
6. Burn base images • Image burning front loads the downloading of software • Reducing the bootstrap time and reliability of the bootstrap process • Packer is an excellent cross cloud tool for burning images. Use it. • If speed of startup is critical you could burn your whole application into an image ◦ Remember migrations should not be run when images are burned ◦ Last mile configuration should probably not be burned into the image Tips • You can use the same base boxes building process for vagrant and production • Bento project by chef has many great examples of burning vagrant images. • Start building base images early. They will save you a lot of time. • Help bootstrapping process by including install scripts and configuration management tool in base box
7. Configuration Management Main Contenders • Chef - Ruby, mostly declarative, large community • Puppet - Custom DSL, declarative, large community • SaltStack - YAML, jinja, python modules and extension points • Ansible - YAML, jinja, agent-less, python based extension points Tips • Prototype more than one system out. You will be stuck with it for a while. • Any one of these systems is better than nothing. Don’t rely on bash scripts for too long. • Get a test system and linting tool in place for whatever you choose from day one ◦ Make sure you treat your configuration building like code from day one
8. Provisioning Tools Full Infrastructure • CloudFormation - AWS specific, declarative, fragile, json based, very complete • HEAT - OpenStack version of CloudFormation • Terraform - Cross cloud, declarative, still buggy, from Hashicorp, much potential • Via cloud init bootstrapping config management systems these tools can do orchestration as well • Much overlap with the orchestration tools found on next page Tips • If you are in AWS you should probably leverage ASGs • If using cloudformation you’ll probably want a tool allow for comments and multiple files • Be careful with cloudformation. Lots of horror stories of getting in bad states.
9. Orchestration Tools Tools to help script SSH • Bash SSH loop - Please no, hard to catch failure, not parallel • Fabric - Python, task based, parallel execution • Capistrano - Ruby, task based, parallel execution More than SSH • Ansible - Python+yaml driven, ad-hoc shell, dynamic nodes • SaltStack - ZeroMQ based, Modules, Master/minion model • Rundeck - API-driven, Centralized, Workflows • MCollective - Probably worth looking into if you use puppet? • OpsWorks - Chef tightly integrated with AWS, also integrated with cloudformation More than just orchestration • Asgard - Grails application, netflix OSS, AWS specific, Opinionated • Elastic BeanStalk - AWS managed single app deployment, easier onboarding, bad docs
10. Other IaaS tips • If deploying the IaaS provider spend time learning linux ◦ Know how your init system works ◦ Have at least basic knowledge of emacs or vim ◦ Spend some time with unix tools (pipes / bash / grep / find / awk) • Building packages early will make a lot of things easier ◦ gradle-debian-plugin works to make deb packages • Watch Docker since it is changing the landscape in a big way ◦ Enables a polyglot stack much easier ◦ gradle-docker works well to build docker containers from gradle • Have a log aggregation system in place in all environments including vagrant ◦ Splunk, Logstash/Kibana, Graylog2, Sentry (open source) • Use vagrant so your dev environment is as close to production as possible
11. Questions?