Application Automation & Containerization With Habitat Nathen Harvey | @nathenharvey 

Shifts in Modern Application Architecture Demand an application-centric operations model Infrastructure Centric Operations Application Centric Operations Cloud Containers Microservices UNIT OF VALUE is App UNIT OF WORK is Deployment UNIT OF VALUE is Server UNIT OF WORK is Configuration Infrastructure Automation Compliance Automation Application Automation EXISTING BUSINESS NEW BUSINESS 

Application automation that enables modern application teams to build, deploy, and manage any application in any environment - from traditional data- centers to containerized microservices. Habitat 

Lifecycle of any software Build Deploy Run 

Application-centric Operations with Chef Chef Automate and Habitat for simple, consistent management of apps BUILD DEPLOY MANAGE Supervisor 

SSH Into the Provided Workstation •  IP Address – on that little slip of paper we gave you •  Login: chef •  Password: habitat 

$ Log in to your remote workstation ssh [email protected] 

Using PuTTY on Windows 

$ hab 0.54.0/20180221022026 Habitat is installed hab --version 

Clean room environment Safe by default Installs dependences Habitat studio 

$ Habitat CLI Setup ================= Welcome to hab setup. Let's get started. Set up a default origin Every package in Habitat belongs to an origin, which indicates the person or organization responsible for maintaining that package. Each origin also has a key used to cryptographically sign packages in that origin. Selecting a default origin tells package building operations such as 'hab pkg build' what key should be used to sign the packages produced. If you do not set a default origin now, you will have to tell package building commands each time what origin to use. For more information on origins and how they are used in building packages, please consult the docs at https://www.habitat.sh/docs/create-packages-build/ Set up a default origin? [Yes/no/quit] Yes Set-up the Habitat Studio (1 of 7) hab setup 

$ Enter the name of your origin. If you plan to publish your packages publicly, we recommend that you select one that is not already in use on the Habitat build service found at https://bldr.habitat.sh/. Origins must begin with a lowercase letter or number. Allowed characters include lowercase letters, numbers, _, -. No more than 255 characters. Default origin name: [default: chef] chef Set-up the Habitat Studio (2 of 7) 

$ Create origin key pair It doesn't look like you have a signing key for the origin `chef'. Without it, you won't be able to build new packages successfully. You can either create a new signing key now, or, if you are building packages for an origin that already exists, ask the owner to give you the signing key. For more information on the use of origin keys, please consult the documentation at https://www.habitat.sh/docs/concepts-keys/#origin-keys Create an origin key for `chef'? [Yes/no/quit] yes Set-up the Habitat Studio (3 of 7) 

$ Create origin key pair It doesn't look like you have a signing key for the origin `chef'. Without it, you won't be able to build new packages successfully. You can either create a new signing key now, or, if you are building packages for an origin that already exists, ask the owner to give you the signing key. For more information on the use of origin keys, please consult the documentation at https://www.habitat.sh/docs/concepts-keys/#origin-keys Create an origin key for `chef'? [Yes/no/quit] yes Set-up the Habitat Studio (4 of 7) 

$ GitHub Access Token While you can build and run Habitat packages without sharing them on the public depot, doing so allows you to collaborate with the Habitat community. In addition, it is how you can perform continuous deployment with Habitat. The GitHub personal access token needs the user:email and read:org OAuth scopes. Habitat uses the information provided through these scopes for authentication and to determine features based on team membership. You can set this up at https://github.com/settings/tokens If you would like to share your packages on the depot, please enter your GitHub access token. Otherwise, just enter No. For more information on sharing packages on the depot, please read the documentation at https://www.habitat.sh/docs/share-packages-overview/ Set up a default GitHub access token? [Yes/no/quit] no Set-up the Habitat Studio (5 of 7) 

$ Analytics The `hab` command-line tool will optionally send anonymous usage data to Habitat's Google Analytics account. This is a strictly opt-in activity and no tracking will occur unless you respond affirmatively to the question below. We collect this data to help improve Habitat's user experience. For example, we would like to know the category of tasks users are performing, and which ones they are having trouble with (e.g. mistyping command line arguments). To see what kinds of data are sent and how they are anonymized, please read more about our analytics here: https://www.habitat.sh/docs/about-analytics/ Enable analytics? [Yes/no/quit] yes Set-up the Habitat Studio (6 of 7) 

$ CLI Setup Complete That's all for now. Thanks for using Habitat! Set-up the Habitat Studio (7 of 7) 

Starts with a plan 

$ » Attempting autodiscovery No scaffolding type was provided. Let's see if we can figure out what kind of application you're planning to package. ... For more information on any of the files: https://www.habitat.sh/docs/reference/plan-syntax/ ★ An abode for your code is initialized! Build a Sample Plan hab plan init --with-all --with-callbacks --with-docs sample-plan 

Plan ~/sample-node-app/habitat/plan.sh pkg_name=sample-node-app pkg_origin=your_origin pkg_origin=chef pkg_scaffolding="core/scaffolding-node" pkg_version="1.0.1" declare -A scaffolding_env # Define path to config file scaffolding_env[APP_CONFIG]="{{pkg.svc_config_path}}/config.json" - + 

Plan ~/sample-node-app/habitat/plan.sh pkg_name=sample-node-app pkg_origin=your_origin pkg_origin=chef pkg_scaffolding="core/scaffolding-node" pkg_version="1.0.1" declare -A scaffolding_env # Define path to config file scaffolding_env[APP_CONFIG]="{{pkg.svc_config_path}}/config.json" - + $ cp ~/new-plan.sh sample-node-app/habitat/plan.sh If you don’t like manually editing files… 

$ Move to the Application’s Directory cd ~/sample-node-app 

$ ∵ Missing package for core/hab-studio/0.54.0 » Installing core/hab-studio/0.54.0 from channel 'stable' ↓ Downloading core/hab-studio/0.54.0/20180221023422 3.17 MB / 3.17 MB - [===============================] 100.00 % 79.21 MB/s ☛ Verifying core/hab-studio/0.54.0/20180221023422 ✓ Installed core/hab-studio/0.54.0/20180221023422 ★ Install of core/hab-studio/0.54.0/20180221023422 complete with 1 new packages installed. hab-studio: Creating Studio at /hab/studios/home--chef--sample-node-app (default) ... --> To prevent a Supervisor from running automatically in your Studio, export 'HAB_STUDIO_SUP=false' before running 'hab studio enter'. Enter the Habitat Studio hab studio enter 

# : Loading /src/habitat/plan.sh sample-node-app: Plan loaded sample-node-app: Validating plan metadata ... sample-node-app: Installed Path: /hab/pkgs/chef/sample-node-app/1.0.1/20180221155144 sample-node-app: Artifact: /src/results/chef-sample-node-app-1.0.1-20180221155144- x86_64-linux.hart sample-node-app: Build Report: /src/results/last_build.env sample-node-app: SHA256 Checksum: 811247255d0b53319f29f5090a72e186247fb000b8f8b362282a3761422a3cd3 sample-node-app: Blake2b Checksum: bde6652ff19cd6ec8c336e466b200127aa7c60987bc3ba86917064237df0c421 sample-node-app: sample-node-app: I love it when a plan.sh comes together. sample-node-app: sample-node-app: Build time: 0m39s Build the Application build 

# hab-sup(MN): The chef/sample-node-app service was successfully loaded Run the Application in the Studio hab sup load chef/sample-node-app 

# --> Tailing the Habitat Supervisor's output (use 'Ctrl+c' to stop) hab-sup(MR): Starting chef/sample-node-app sample-node-app.default(UCW): Watching user.toml sample-node-app.default(HK): init, compiled to /hab/svc/sample-node-app/hooks/init sample-node-app.default(HK): Hooks compiled sample-node-app.default(SR): Hooks recompiled default(CF): Updated app_env.sh 9c88a9595def28f5914f08997b3cd3ba8d32f0e3eb2b4e4ea90264acd6acb66b default(CF): Updated config.json 3f22842e8d737bbb107d9ac19afba42642eccf68a06ddfbdba70507b23b8498a sample-node-app.default(SR): Configuration recompiled sample-node-app.default(SR): Initializing sample-node-app.default(SV): Starting service as user=hab, group=hab Follow the Supervisor Log sup-log 

http://YOUR_IP:8000 

Installs automatically Keeps things running Manages configuration Supervisor runs services 

# message = "Hello, World!" node_env = "production" [app] port = 8000 Configuration Options hab sup config chef/sample-node-app 

# Write Configuration Options to a File hab sup config chef/sample-node-app > config.toml 

Update the configuration config.toml message = "Hello, World!" node_env = "production" [app] port = 8000 port = 9000 - + 

Update the configuration config.toml message = "Hello, World!" node_env = "production" [app] port = 8000 port = 9000 - + # cp new-config.toml config.toml If you don’t like manually editing files… 

# Apply the New Configuration hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml 

Applying Configuration hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml •  peer - hostname or IP address of a running supervisor •  sample-node-app.default - the service group •  1 - the incarnation of this configuration •  config.toml - the file with the new configuration 

# ∵ Missing package for core/hab-butterfly/0.54.0 » Installing core/hab-butterfly/0.54.0 from channel 'stable' ↓ Downloading core/hab-butterfly/0.54.0/20180221020301 ... ★ Install of core/hab-butterfly/0.54.0/20180221020301 complete with 1 new packages installed. » Applying configuration for sample-node-app.default incarnation 1 Ω Creating service configuration ✓ Verified this configuration is valid TOML ↑ Applying to peer 127.0.0.1:9638 ★ Applied configuration Apply the New Configuration hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml 

http://YOUR_IP:9000 

Docker tar Kubernetes Post-process packages 

# ∵ Missing package for core/hab-pkg-export-docker/0.54.0 » Installing core/hab-pkg-export-docker/0.54.0 from channel 'stable' ↓ Downloading core/hab-pkg-export-docker/0.54.0/20180221020412 2.40 MB / 2.40 MB - [===============================] 100.00 % 89.49 MB/s ☛ Verifying core/hab-pkg-export-docker/0.54.0/20180221020412 ... Removing intermediate container 582430ddac0a ---> 0cb96cce166b Successfully built 0cb96cce166b Successfully tagged chef/sample-node-app:1.0.1-20180221155145 Successfully tagged chef/sample-node-app:1.0.1 Successfully tagged chef/sample-node-app:latest ☒ Deleting temporary files ★ Docker image 'chef/sample-node-app' created with tags: 1.0.1-20180221155145, 1.0.1, latest Ω Creating build report /src/results/last_docker_export.env Export to Docker hab pkg export docker results/chef-sample-node-app-*.hart 

# Leave the Studio exit 

$ REPOSITORY TAG IMAGE ID CREATED SIZE chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB Inspect the Local Docker Images docker images 

$ REPOSITORY TAG IMAGE ID CREATED SIZE chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB Run in a Docker Container docker run -it -p 8000:8000 chef/sample-node-app 

http://YOUR_IP:8000 

$ REPOSITORY TAG IMAGE ID CREATED SIZE chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB Run in a Docker Container with Updated Configuration docker run -e HAB_SAMPLE_NODE_APP="$(cat config.toml)" -p 9000:9000 -it chef/sample-node-app 

http://YOUR_IP:9000 

$ Copy the Origin Key to the Proper Location sudo cp ~/.hab/cache/keys/chef-*.pub /hab/cache/keys/ 

$ ∵ Missing package for core/hab-launcher » Installing core/hab-launcher from channel 'stable' ↓ Downloading core/hab-launcher/6925/20180202215752 4.78 MB / 4.78 MB / [===============================] 100.00 % 31.47 MB/s ... sample-node-app.default(SR): Configuration recompiled sample-node-app.default(SR): Initializing sample-node-app.default(SV): Starting service as user=hab, group=hab Start the Application Locally sudo hab sup start results/chef-sample-node-app-*.hart 

http://YOUR_IP:8000 

$ hab-sup(MR): Supervisor Member-ID 100b72aea4104c9ba52972622db4bda1 hab-sup(MR): Starting chef/sample-node-app/1.0.1/20180222023512 sample-node-app.default(UCW): Watching user.toml hab-sup(MR): Starting gossip-listener on 0.0.0.0:9638 hab-sup(MR): Starting http-gateway on 0.0.0.0:9631 sample-node-app.default(HK): Hooks compiled sample-node-app.default(SR): Initializing sample-node-app.default(SV): Starting service as user=hab, group=hab sample-node-app.default(HK): Hooks compiled Join My Supervisor Ring sudo hab sup run --peer 172.31.28.111 

http://YOUR_IP:9631/ butterfly 

Supervisors provide update strategies 

$ hab-sup(MN): Missing package for nathenharvey/sample-node-app » Installing nathenharvey/sample-node-app from channel 'stable' ... ✓ Installed nathenharvey/sample-node-app/1.0.8/20180221181902 ★ Install of nathenharvey/sample-node-app/1.0.8/20180221181902 complete with 1 new packages installed. hab-sup(MN): The nathenharvey/sample-node-app service was successfully loaded Launch My Sample App sudo hab sup load nathenharvey/sample-node-app -s rolling 

Builder Demo 

http://YOUR_IP:8000 

$ hab-sup(MN): Unloading "/hab/sup/default/specs/sample-node-app.spec" Unload the Application sudo hab sup unload nathenharvey/chef-sample-node-app 

$ Terminate the Supervisor sudo hab sup term 

Automation travels with the application Existing & Cloud Native Software Application Automation 

What’s Next? •  Join the Habitat Slack Team - http://slack.habitat.sh/ •  Work through the tutorial at https://www.habitat.sh/tutorials/ •  Explore Habitat packages on the depot - https://app.habitat.sh/ •  Explore the Habitat projects - https://github.com/habitat-sh •  Read Habitat Blog posts - https://www.habitat.sh/blog/ 

Bonus Slides Because you are awesome! 

SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR USER ARTIFACT How we do it Packaging Applications Running Applications PLAN DEPOT DEPOT ARTIFACT BARE METAL CONTAINERS AMI VM 

Habitat and Modern Architectures ●  Habitat + Better Containers ○  Application Centric Containers ●  Habitat + Cluster Management (e.g. k8s) ○  Build all your workloads with Habitat’s Builder ○  Stateless services run on k8s ○  Stateful services connect via cloud service broker ■  Export to Container, VMs, Bare Metal as fits the workload ○  Manage all your workloads with Habitat’s Supervisor & Runtime 

LEADER STAND ALONE Topologies Update Strategies Running Applications SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR How we do it LEADER ElECTION ALL AT ONCE ROLLING 

SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR How we do it Security PUB KEY SYMMETRIC ENCRYPTION LOAD BALANCER Build Service BUILD SERVIC E USER SECRET PAYLOADS SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR ARTIFACT PLAN DEPOT 

What the modern application team gets Developers, System Administrators, CIOs; Enterprise and Tech Innovators ▪  Runs the same way in any environment ▪  Management travels with the application; no drift ▪  Autonomous and self- organizing ▪  Legacy and Greenfield ▪  Lets the enterprise modernize without re- writing the world ▪  Faster to build, easier to deploy, safer to manage ▪  Easiest way to deploy applications, containers, and microservices in production ▪  Developers can focus on building great applications ▪  Systems Administrators can focus on how those applications should behave ▪  Gives both a language they can share, with clear boundaries Simplification Acceleration Empowerment 

Habitat Build Philosophy ●  Container formats recreate the traditional model of infrastructure and applications. ●  Poor at abstracting the Build + Run aspects of Applications Libraries Operating System Application Application & Libraries ●  Habitat builds from the application down ●  Small lightweight OS included ●  Embedded Supervisor for Application Management ●  Builds have strict dependency version control Application Libraries OS 

Application Centric Containers with Habitat SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR SERVICE SUPERVISOR CONTAINER ●  Provides a “DSL” for defining the build lifecycle of the application ●  Application run lifecycle is defined along side the application build lifecycle ●  Hab artifacts are based on application artifact, declared dependencies, and defined run lifecycle ●  Export hab artifacts to containers ●  Containers have built in supervisor for application operations concerns: ○  Configuration ○  Service Discovery ○  Clustering Topologies EXPORTS CONTAINER FORMATS 

Habitat + Container Orchestrators ●  Develop reusable application packages ●  Expose tunable settings ●  Export to your preferred Container Platform ●  Deploy your application in a scalable, topology aware format. BUILD DEPLOY MANAGE ●  Orchestrate and Automate your application config changes ●  Application lifecycle hooks respond to updates & events 

No content