Easy Ruby Development and Deployment with Otto

Ruby is amazing, but it isn't always the easiest tool to get set up for local development. Vagrant continues to be a great tool for automated and reproducible development environments, but it often falls short of real production scenarios. Otto, the successor to Vagrant, encompasses the entire workflow managing local development environments, infrastructure creation, and application deployment in just three easy commands. You can have the Heroku-like workflow while maintaining full control over the stack, without all the overhead. Come meet Otto!

More and more, organizations desire extended control over their full stack. From development to production, controlling the entire stack ensures parity, reduces bugs in production, and ultimately makes for happier developers :). With Otto, you can say goodbye to staging/qa/testing environments because any user can easily create all the required infrastructure in a single command. This saves on infrastructure costs, maintenance, and makes for an amazing disaster recovery story!

Following the story of a fictitious Ruby app, I will deploy their application to AWS in just a two commands in live-demo format. Otto has codified knowledge of today's best practices for deploying applications, so even with no operational experience, a developer can create and manage infrastructure using today's best practices. Otto brings the "Heroku-like" workflow to the command line, but you control the entire stack from top to bottom!

After each phase, we will "uncover the magic", digging into the commands Otto is running, the decisions Otto is making, and the various customizations that can be injected along the way.

Transcript

1. EASY DEVELOPMENT AND DEPLOYMENT Industry quality, self-service workflows with Otto
2. SETH VARGO @sethvargo
4. VERSUS
5. Otto and the HashiCorp Ecosystem
8. Why Otto?
9. Automate Development Environments
10. $ otto dev ===> otto: detected app: ruby
11. $ otto dev ===> otto: detected app: python
12. Focus on Application Development
13. $ otto dev ssh vagrant@precise64:/vagrant$
14. Codify Deployments
17. Today's Best Practices WEB DB MICRO SERVICES PRIVATE SUBNET  PUBLIC SUBNET  INTERNAL NETWORK LB BASTION NAT
18. WEB DB MICRO SERVICES PRIVATE SUBNET  PUBLIC SUBNET  INTERNAL NETWORK LB BASTION NAT Today's Best Practices COMPLEXITY
20. Simplify Microservices
21. Microservices Monolithic Microservices
22. Microservices
23. Microservices
24. application { name = "my-store" type = "ruby" dependency { source = "github.com/hashicorp/otto/examples/postgresql" } } customization "ruby" { ruby_version = "2.1" } Appfile
25. Doin' it Live1  1. Unless it breaks
26. BOOTSTRAP APP 1
27. $ vi Gemfile
28. source "https://rubygems.org" gem "pg", "~> 0.18" gem "shotgun", "~> 0.9" gem "sinatra", "~> 1.4" gem "sinatra-activerecord", "~> 2.0" Gemfile
29. CONFIGURE DEV 2
30. $ otto compile
31. $ otto compile ==> Loading Appfile... ==> No Appfile found! Detecting project information... No Appfile was found. If there is no Appfile, Otto will do its best to detect the type of application this is and set reasonable defaults. This is a good way to get started with Otto, but over time we recommend writing a real Appfile since this will allow more complex customizations, the ability to reference dependencies, versioning, and more. ==> Fetching all Appfile dependencies... Fetching dependency: git::github.com/hashicorp/otto.git//examples/postgresql ==> Compiling dependency 'postgresql'... ==> Compiling main application... ==> Detecting Ruby version to use... No desired Ruby version found! Will use the default: 2.2 ==> Compilation success!
32. $ otto compile ==> Loading Appfile... ==> No Appfile found! Detecting project information... No Appfile was found. If there is no Appfile, Otto will do its best to detect the type of application this is and set reasonable defaults. This is a good way to get started with Otto, but over time we recommend writing a real Appfile since this will allow more complex customizations, the ability to reference dependencies, versioning, and more. ==> Fetching all Appfile dependencies... Fetching dependency: git::github.com/hashicorp/otto.git//examples/postgresql ==> Compiling dependency 'postgresql'... ==> Compiling main application... ==> Detecting Ruby version to use... No desired Ruby version found! Will use the default: 2.2 ==> Compilation success!
33. $ otto dev ==> Creating development environment layers... Otto uses layers to speed up building development environments. Each layer only needs to be built once. We've detected that the layers below aren't created yet. These will be built this time. Future development environments will use the cached versions to be much, much faster.
34. $ otto dev ==> Creating layer: consul Bringing machine 'default' up with 'virtualbox' provider... ...
35. $ otto dev ==> Creating layer: ruby2.2 Bringing machine 'default' up with 'virtualbox' provider... ... ==> default: [otto] Installing supporting packages... ==> default: [otto] Installing Ruby 2.2. This can take a few minutes... ==> default: [otto] Installing Bundler...
36. $ otto dev ==> Creating local development environment with Vagrant ... ==> default: [otto] Bundling gem dependencies... ==> default: Fetching version metadata from https://rubygems.org/... ==> default: Resolving dependencies... ==> default: Installing rack 1.6.4 ==> default: Installing tilt 2.0.2 ==> default: Using bundler 1.11.2 ==> default: Installing rack-protection 1.5.3 ==> default: Installing shotgun 0.9.1 ==> default: Installing sinatra 1.4.7 ==> default: Bundle complete! 5 Gemfile dependencies, 24 gems now installed. ==> default: Use `bundle show [gemname]` to see where a bundled gem is installed.
37. $ otto dev ==> Development environment successfully created! IP address: 100.117.7.132 A development environment has been created for writing a generic Ruby-based app. Ruby is pre-installed. To work on your project, edit files locally on your own machine. The file changes will be synced to the development environment. When you're ready to build your project, run 'otto dev ssh' to enter the development environment. You'll be placed directly into the working directory where you can run 'bundle' and 'ruby' as you normally would. You can access any running web application using the IP above.
38. $ otto dev ==> Verifying created layer: consul ==> Verifying created layer: ruby2.2 ...
39. DEVELOP LOCALLY 3
40. $ vi app.rb
41. require "bundler" Bundler.require get "/" do "The app is running!" end app.rb
42. $ vi config.ru
43. require "./app" run Sinatra::Application config.ru
44. $ otto dev ssh ==> Executing SSH. This may take a few seconds... Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
45. vagrant@precise64:/vagrant$ shotgun --host 0.0.0.0 config.ru == Shotgun/WEBrick on http://0.0.0.0:9393/ [2020-01-01 20:49:36] INFO WEBrick 1.3.1 [2020-01-01 20:49:36] INFO ruby 2.2.4 (2015-12-16) [x86_64-linux] [2020-01-01 20:49:36] INFO WEBrick::HTTPServer#start: pid=2183 port=9393
46. $ curl http://100.117.7.132:9393/ The app is running!
47. $ curl http://$(otto dev address):9393/ The app is running!
48. $ curl -I http://$(otto dev address):9393/ HTTP/1.1 200 OK Content-Type: text/html;charset=utf-8 Content-Length: 19 X-Xss-Protection: 1; mode=block X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN Server: WEBrick/1.3.1 (Ruby/2.2.4/2015-12-16) Date: Mon, 01 Jan 2020 20:50:35 GMT Connection: Keep-Alive
49. $ psql ...?
50. $ otto dev ssh vagrant@precise64:/vagrant$ nslookup postgresql.service.consul Server: 127.0.0.1 Address: 127.0.0.1#53 Name: postgresql.service.consul Address: 10.0.2.15
51. vagrant@precise64:/vagrant$ ./scripts/createdb.sh
52. $ vi app.rb
53. require "bundler" Bundler.require configure do ActiveRecord::Base.establish_connection( adapter: "postgresql", host: "postgresql.service.consul", username: "postgres", database: "demo", encoding: "utf8", ) end class Post < ActiveRecord::Base; end app.rb
54. $ vi app.rb # ... get "/posts", provides: :json do Post.all.to_json end post "/posts", provides: :json do if post = Post.create(params) post.to_json else status 422 post.errors.to_json end end
55. vagrant@precise64:/vagrant$ rake db:migrate == 20160406140708 CreatePosts: migrating ====================================== -- create_table(:posts) -> 0.0782s == 20160406140708 CreatePosts: migrated (0.0788s) =============================
56. $ curl http://$(otto dev address):9393/posts []
57. $ curl -XPOST http://$(otto dev address):9393/posts -F "title=hello" -F "body=world" { "id": 1, "title": "hello", "body": "world", "created_at": "2016-04-06T19:34:01.998Z", "updated_at": "2016-04-06T19:34:01.998Z" }
58. $ curl http://$(otto dev address):9393/posts [ { "id": 1, "title": "hello", "body": "world", "created_at": "2016-04-06T19:34:01.998Z", "updated_at": "2016-04-06T19:34:01.998Z" }, ]
59. CONSTRUCT INFRA 4
60. $ otto infra ==> Detecting infrastructure credentials for: demo (aws) Existing infrastructure credentials were not found! Otto will now ask you for infrastructure credentials. These will be encrypted and saved on disk so this doesn't need to be repeated. IMPORTANT: If you're re-entering new credentials, make sure the credentials are for the same account, otherwise you may lose access to your existing infrastructure Otto set up.
61. $ otto infra AWS Access Key AWS access key used for API calls. Enter a value: ... AWS Secret Key AWS secret key used for API calls. Enter a value: ... SSH Public Key Path Path to an SSH public key that will be granted access to EC2 instances Default: ~/.ssh/id_rsa.pub Enter a value: ... Password for Encrypting Credentials This password will be used to encrypt and save the credentials so they don't need to be repeated multiple times. Enter a value: ...
62. $ otto infra ==> Building main infrastructure... ==> Executing Terraform to manage infrastructure... Raw Terraform output will begin streaming in below. Otto does not create this output. It is mirrored directly from Terraform while the infrastructure is being created. Terraform may ask for input. For infrastructure provider credentials, be sure to enter the same credentials consistently within the same Otto environment.
63. $ otto infra ... Apply complete! Resources: 6 added, 0 changed, 0 destroyed. Outputs: infra_id = efa4b38b key_name = otto-efa4b38b region = us-east-1 subnet_public = subnet-5435b57e vpc_cidr = 10.0.0.0/16 vpc_id = vpc-efa4b38b ==> Infrastructure successfully created! The infrastructure necessary to deploy this application is now available. You can now deploy using `otto deploy`.
64. $ otto status ==> App Info Application: demo (ruby) Project: demo Infrastructure: aws (simple) ==> Component Status Dev environment: CREATED Infra: READY Build: NOT BUILT Deploy: NOT DEPLOYED
65. BUILD APP 5
66. $ otto build ... ==> Building deployment archive... Error building app: error detecting VCS: no VCS found for path: /demo
67. $ git init && git add -A && git commit -m "Initial commit" Reinitialized existing Git repository in /demo/.git/ [master (root-commit) 5e57d71] Initial commit 63 files changed, 1744 insertions(+)
68. $ otto build ==> Building deployment archive... ==> Building deployment artifact with Packer... Raw Packer output will begin streaming in below. Otto does not create this output. It is mirrored directly from Packer while the build is being run. ==> otto: Prevalidating AMI Name... ==> otto: Inspecting the source AMI... ==> otto: Creating temporary keypair: packer 5702de4a-4f50-a189-6afd-25b0cd447ae2 ==> otto: Creating temporary security group for this instance... ==> otto: Authorizing access to port 22 the temporary security group... ==> otto: Launching a source AWS instance... otto: Instance ID: i-d3796849 ==> otto: Waiting for instance (i-d3796849) to become ready...
69. $ otto build ... ==> Builds finished. The artifacts of successful builds are: --> otto: AMIs were created: us-east-1: ami-64f1e50e ==> Storing build data in directory... ==> Build success! The build was completed successfully and stored within the directory service, meaning other members of your team don't need to rebuild this same version and can deploy it immediately.
70. $ otto status ==> App Info Application: demo (ruby) Project: demo Infrastructure: aws (simple) ==> Component Status Dev environment: CREATED Infra: READY Build: BUILD READY Deploy: NOT DEPLOYED
71. DEPLOY APP 6
72. $ otto deploy ==> Detecting infrastructure credentials for: demo (aws) Cached and encrypted infrastructure credentials found. Otto will now ask you for the password to decrypt these credentials. ... Apply complete! Resources: 2 added, 0 changed, 0 destroyed. Outputs: url = http://ec2-54-164-91-52.compute-1.amazonaws.com/
73. $ export URL="http://ec2-54-164-91-52.compute-1.amazonaws.com" $ curl $URL The app is running! $ curl $URL/posts [] $ curl -XPOST $URL/posts -F "title=hello" -F "body=world" { "id": 1, "title": "hello", "body": "world", "created_at": "2016-04-06T19:34:01.998Z", "updated_at": "2016-04-06T19:34:01.998Z" }
74. DESTROY IT ALL 7
75. $ otto dev destroy ==> Destroying the local development environment... $ otto deploy destroy Do you really want to destroy? Otto will delete all resources associated with the deploy. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes $ otto infra destroy Do you really want to destroy? Otto will delete all your managed infrastructure. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes
76. COMPLEX REQS ?
77. $ vi Appfile
78. application { name = "demo" type = "ruby" dependency { source = "github.com/hashicorp/otto/examples/postgresql" } } customization "ruby" { ruby_version = "2.2" } Appfile
79. $ otto compile ==> Loading Appfile... ==> Fetching all Appfile dependencies... Fetching dependency: git::github.com/hashicorp/otto.git/examples/postgresql ==> Compiling... Application: demo (ruby) Project: demo Infrastructure: aws (simple) Compiling infra... Compiling foundation: consul ==> Compiling dependency 'postgresql'... ==> Compiling main application... ==> Compilation success!
81. SETH VARGO @sethvargo QUESTIONS?