Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Application Automation & Containerization With Habitat

Nathen Harvey
February 22, 2018

Application Automation & Containerization With Habitat

Habitat is a simple, flexible way to build, deploy, and manage cloud native and modern, distributed applications. Habitat centers application configuration, management, and behavior around the application itself, not the infrastructure that the application runs on. It provides automation that can programmatically and declaratively build, deploy, and manage applications and services, both stateful and stateless. Each Habitat application can be deployed and run on various infrastructure environments including bare metal, VM, containers, and PaaS.

This workshop provides a quick introduction to Habitat during which participants will build, deploy, and run a multi-tier application as a way to explore some of Habitat's application automation features.

Find more about Habitat at https://www.habitat.sh/.

This workshop was offered at DevOpsDays Charlotte, 2017.

Nathen Harvey

February 22, 2018
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. Application Automation &
    Containerization With Habitat
    Nathen Harvey | @nathenharvey

    View Slide

  2. Shifts in Modern Application Architecture
    Demand an application-centric operations model
    Infrastructure
    Centric
    Operations
    Application
    Centric
    Operations
    Cloud
    Containers
    Microservices
    UNIT OF VALUE is App
    UNIT OF WORK is Deployment
    UNIT OF VALUE is Server
    UNIT OF WORK is Configuration
    Infrastructure Automation Compliance Automation Application Automation
    EXISTING BUSINESS
    NEW BUSINESS

    View Slide

  3. Application automation that enables
    modern application teams to build,
    deploy, and manage any application in
    any environment - from traditional data-
    centers to containerized microservices.
    Habitat

    View Slide

  4. Lifecycle of any software
    Build Deploy Run

    View Slide

  5. Application-centric Operations with Chef
    Chef Automate and Habitat for simple, consistent management of apps
    BUILD DEPLOY MANAGE
    Supervisor

    View Slide

  6. SSH Into the Provided Workstation
    •  IP Address – on that little slip of paper we gave you
    •  Login: chef
    •  Password: habitat

    View Slide

  7. $
    Log in to your remote workstation
    ssh [email protected]

    View Slide

  8. Using PuTTY on Windows

    View Slide

  9. $
    hab 0.54.0/20180221022026
    Habitat is installed
    hab --version

    View Slide

  10. Clean room environment
    Safe by default
    Installs dependences
    Habitat studio

    View Slide

  11. $
    Habitat CLI Setup
    =================
    Welcome to hab setup. Let's get started.
    Set up a default origin
    Every package in Habitat belongs to an origin, which indicates the
    person or organization responsible for maintaining that package. Each
    origin also has a key used to cryptographically sign packages in that
    origin.
    Selecting a default origin tells package building operations such as
    'hab pkg build' what key should be used to sign the packages produced.
    If you do not set a default origin now, you will have to tell package
    building commands each time what origin to use.
    For more information on origins and how they are used in building
    packages, please consult the docs at
    https://www.habitat.sh/docs/create-packages-build/
    Set up a default origin? [Yes/no/quit] Yes
    Set-up the Habitat Studio (1 of 7)
    hab setup

    View Slide

  12. $
    Enter the name of your origin. If you plan to publish your packages
    publicly, we recommend that you select one that is not already in use on
    the Habitat build service found at https://bldr.habitat.sh/.
    Origins must begin with a lowercase letter or number. Allowed characters
    include lowercase letters, numbers, _, -. No more than 255 characters.
    Default origin name: [default: chef] chef
    Set-up the Habitat Studio (2 of 7)

    View Slide

  13. $
    Create origin key pair
    It doesn't look like you have a signing key for the origin `chef'.
    Without it, you won't be able to build new packages successfully.
    You can either create a new signing key now, or, if you are building
    packages for an origin that already exists, ask the owner to give you
    the signing key.
    For more information on the use of origin keys, please consult the
    documentation at https://www.habitat.sh/docs/concepts-keys/#origin-keys
    Create an origin key for `chef'? [Yes/no/quit] yes
    Set-up the Habitat Studio (3 of 7)

    View Slide

  14. $
    Create origin key pair
    It doesn't look like you have a signing key for the origin `chef'.
    Without it, you won't be able to build new packages successfully.
    You can either create a new signing key now, or, if you are building
    packages for an origin that already exists, ask the owner to give you
    the signing key.
    For more information on the use of origin keys, please consult the
    documentation at https://www.habitat.sh/docs/concepts-keys/#origin-keys
    Create an origin key for `chef'? [Yes/no/quit] yes
    Set-up the Habitat Studio (4 of 7)

    View Slide

  15. $
    GitHub Access Token
    While you can build and run Habitat packages without sharing them on the
    public depot, doing so allows you to collaborate with the Habitat
    community. In addition, it is how you can perform continuous deployment
    with Habitat.
    The GitHub personal access token needs the user:email and read:org OAuth
    scopes. Habitat uses the information provided through these scopes for
    authentication and to determine features based on team membership. You
    can set this up at https://github.com/settings/tokens
    If you would like to share your packages on the depot, please enter your
    GitHub access token. Otherwise, just enter No.
    For more information on sharing packages on the depot, please read the
    documentation at https://www.habitat.sh/docs/share-packages-overview/
    Set up a default GitHub access token? [Yes/no/quit] no
    Set-up the Habitat Studio (5 of 7)

    View Slide

  16. $
    Analytics
    The `hab` command-line tool will optionally send anonymous usage data to
    Habitat's Google Analytics account. This is a strictly opt-in activity
    and no tracking will occur unless you respond affirmatively to the
    question below.
    We collect this data to help improve Habitat's user experience. For
    example, we would like to know the category of tasks users are
    performing, and which ones they are having trouble with (e.g. mistyping
    command line arguments).
    To see what kinds of data are sent and how they are anonymized, please
    read more about our analytics here:
    https://www.habitat.sh/docs/about-analytics/
    Enable analytics? [Yes/no/quit] yes
    Set-up the Habitat Studio (6 of 7)

    View Slide

  17. $
    CLI Setup Complete
    That's all for now. Thanks for using Habitat!
    Set-up the Habitat Studio (7 of 7)

    View Slide

  18. Starts with a plan

    View Slide

  19. $
    » Attempting autodiscovery
    No scaffolding type was provided. Let's see if we can figure out what
    kind of application you're planning to package.
    ...
    For more information on any of the files:
    https://www.habitat.sh/docs/reference/plan-syntax/
    ★ An abode for your code is initialized!
    Build a Sample Plan
    hab plan init --with-all --with-callbacks --with-docs sample-plan

    View Slide

  20. Plan
    ~/sample-node-app/habitat/plan.sh
    pkg_name=sample-node-app
    pkg_origin=your_origin
    pkg_origin=chef
    pkg_scaffolding="core/scaffolding-node"
    pkg_version="1.0.1"
    declare -A scaffolding_env
    # Define path to config file
    scaffolding_env[APP_CONFIG]="{{pkg.svc_config_path}}/config.json"
    -
    +

    View Slide

  21. Plan
    ~/sample-node-app/habitat/plan.sh
    pkg_name=sample-node-app
    pkg_origin=your_origin
    pkg_origin=chef
    pkg_scaffolding="core/scaffolding-node"
    pkg_version="1.0.1"
    declare -A scaffolding_env
    # Define path to config file
    scaffolding_env[APP_CONFIG]="{{pkg.svc_config_path}}/config.json"
    -
    +
    $ cp ~/new-plan.sh sample-node-app/habitat/plan.sh
    If you don’t like manually editing files…

    View Slide

  22. $
    Move to the Application’s Directory
    cd ~/sample-node-app

    View Slide

  23. $
    ∵ Missing package for core/hab-studio/0.54.0
    » Installing core/hab-studio/0.54.0 from channel 'stable'
    ↓ Downloading core/hab-studio/0.54.0/20180221023422
    3.17 MB / 3.17 MB - [===============================] 100.00 % 79.21 MB/s
    ☛ Verifying core/hab-studio/0.54.0/20180221023422
    ✓ Installed core/hab-studio/0.54.0/20180221023422
    ★ Install of core/hab-studio/0.54.0/20180221023422 complete with 1 new packages installed.
    hab-studio: Creating Studio at /hab/studios/home--chef--sample-node-app (default)
    ...
    --> To prevent a Supervisor from running automatically in your
    Studio, export 'HAB_STUDIO_SUP=false' before running
    'hab studio enter'.
    Enter the Habitat Studio
    hab studio enter

    View Slide

  24. #
    : Loading /src/habitat/plan.sh
    sample-node-app: Plan loaded
    sample-node-app: Validating plan metadata
    ...
    sample-node-app: Installed Path: /hab/pkgs/chef/sample-node-app/1.0.1/20180221155144
    sample-node-app: Artifact: /src/results/chef-sample-node-app-1.0.1-20180221155144-
    x86_64-linux.hart
    sample-node-app: Build Report: /src/results/last_build.env
    sample-node-app: SHA256 Checksum:
    811247255d0b53319f29f5090a72e186247fb000b8f8b362282a3761422a3cd3
    sample-node-app: Blake2b Checksum:
    bde6652ff19cd6ec8c336e466b200127aa7c60987bc3ba86917064237df0c421
    sample-node-app:
    sample-node-app: I love it when a plan.sh comes together.
    sample-node-app:
    sample-node-app: Build time: 0m39s
    Build the Application
    build

    View Slide

  25. #
    hab-sup(MN): The chef/sample-node-app service was successfully loaded
    Run the Application in the Studio
    hab sup load chef/sample-node-app

    View Slide

  26. #
    --> Tailing the Habitat Supervisor's output (use 'Ctrl+c' to stop)
    hab-sup(MR): Starting chef/sample-node-app
    sample-node-app.default(UCW): Watching user.toml
    sample-node-app.default(HK): init, compiled to /hab/svc/sample-node-app/hooks/init
    sample-node-app.default(HK): Hooks compiled
    sample-node-app.default(SR): Hooks recompiled
    default(CF): Updated app_env.sh
    9c88a9595def28f5914f08997b3cd3ba8d32f0e3eb2b4e4ea90264acd6acb66b
    default(CF): Updated config.json
    3f22842e8d737bbb107d9ac19afba42642eccf68a06ddfbdba70507b23b8498a
    sample-node-app.default(SR): Configuration recompiled
    sample-node-app.default(SR): Initializing
    sample-node-app.default(SV): Starting service as user=hab, group=hab
    Follow the Supervisor Log
    sup-log

    View Slide

  27. http://YOUR_IP:8000

    View Slide

  28. Installs automatically
    Keeps things running
    Manages configuration
    Supervisor runs services

    View Slide

  29. #
    message = "Hello, World!"
    node_env = "production"
    [app]
    port = 8000
    Configuration Options
    hab sup config chef/sample-node-app

    View Slide

  30. #
    Write Configuration Options to a File
    hab sup config chef/sample-node-app > config.toml

    View Slide

  31. Update the configuration
    config.toml
    message = "Hello, World!"
    node_env = "production"
    [app]
    port = 8000
    port = 9000
    -
    +

    View Slide

  32. Update the configuration
    config.toml
    message = "Hello, World!"
    node_env = "production"
    [app]
    port = 8000
    port = 9000
    -
    +
    # cp new-config.toml config.toml
    If you don’t like manually editing files…

    View Slide

  33. #
    Apply the New Configuration
    hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml

    View Slide

  34. Applying Configuration
    hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml
    •  peer - hostname or IP address of a running supervisor
    •  sample-node-app.default - the service group
    •  1 - the incarnation of this configuration
    •  config.toml - the file with the new configuration

    View Slide

  35. #
    ∵ Missing package for core/hab-butterfly/0.54.0
    » Installing core/hab-butterfly/0.54.0 from channel 'stable'
    ↓ Downloading core/hab-butterfly/0.54.0/20180221020301
    ...
    ★ Install of core/hab-butterfly/0.54.0/20180221020301 complete with 1 new packages
    installed.
    » Applying configuration for sample-node-app.default incarnation 1
    Ω Creating service configuration
    ✓ Verified this configuration is valid TOML
    ↑ Applying to peer 127.0.0.1:9638
    ★ Applied configuration
    Apply the New Configuration
    hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml

    View Slide

  36. http://YOUR_IP:9000

    View Slide

  37. Docker
    tar
    Kubernetes
    Post-process packages

    View Slide

  38. #
    ∵ Missing package for core/hab-pkg-export-docker/0.54.0
    » Installing core/hab-pkg-export-docker/0.54.0 from channel 'stable'
    ↓ Downloading core/hab-pkg-export-docker/0.54.0/20180221020412
    2.40 MB / 2.40 MB - [===============================] 100.00 % 89.49 MB/s
    ☛ Verifying core/hab-pkg-export-docker/0.54.0/20180221020412
    ...
    Removing intermediate container 582430ddac0a
    ---> 0cb96cce166b
    Successfully built 0cb96cce166b
    Successfully tagged chef/sample-node-app:1.0.1-20180221155145
    Successfully tagged chef/sample-node-app:1.0.1
    Successfully tagged chef/sample-node-app:latest
    ☒ Deleting temporary files
    ★ Docker image 'chef/sample-node-app' created with tags: 1.0.1-20180221155145, 1.0.1,
    latest
    Ω Creating build report /src/results/last_docker_export.env
    Export to Docker
    hab pkg export docker results/chef-sample-node-app-*.hart

    View Slide

  39. #
    Leave the Studio
    exit

    View Slide

  40. $
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB
    Inspect the Local Docker Images
    docker images

    View Slide

  41. $
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB
    Run in a Docker Container
    docker run -it -p 8000:8000 chef/sample-node-app

    View Slide

  42. http://YOUR_IP:8000

    View Slide

  43. $
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB
    Run in a Docker Container with Updated Configuration
    docker run -e HAB_SAMPLE_NODE_APP="$(cat config.toml)" -p 9000:9000 -it chef/sample-node-app

    View Slide

  44. http://YOUR_IP:9000

    View Slide

  45. $
    Copy the Origin Key to the Proper Location
    sudo cp ~/.hab/cache/keys/chef-*.pub /hab/cache/keys/

    View Slide

  46. $
    ∵ Missing package for core/hab-launcher
    » Installing core/hab-launcher from channel 'stable'
    ↓ Downloading core/hab-launcher/6925/20180202215752
    4.78 MB / 4.78 MB / [===============================] 100.00 % 31.47 MB/s
    ...
    sample-node-app.default(SR): Configuration recompiled
    sample-node-app.default(SR): Initializing
    sample-node-app.default(SV): Starting service as user=hab, group=hab
    Start the Application Locally
    sudo hab sup start results/chef-sample-node-app-*.hart

    View Slide

  47. http://YOUR_IP:8000

    View Slide

  48. $
    hab-sup(MR): Supervisor Member-ID 100b72aea4104c9ba52972622db4bda1
    hab-sup(MR): Starting chef/sample-node-app/1.0.1/20180222023512
    sample-node-app.default(UCW): Watching user.toml
    hab-sup(MR): Starting gossip-listener on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    sample-node-app.default(HK): Hooks compiled
    sample-node-app.default(SR): Initializing
    sample-node-app.default(SV): Starting service as user=hab, group=hab
    sample-node-app.default(HK): Hooks compiled
    Join My Supervisor Ring
    sudo hab sup run --peer 172.31.28.111

    View Slide

  49. http://YOUR_IP:9631/
    butterfly

    View Slide

  50. Supervisors provide update strategies

    View Slide

  51. $
    hab-sup(MN): Missing package for nathenharvey/sample-node-app
    » Installing nathenharvey/sample-node-app from channel 'stable'
    ...
    ✓ Installed nathenharvey/sample-node-app/1.0.8/20180221181902
    ★ Install of nathenharvey/sample-node-app/1.0.8/20180221181902 complete with 1 new packages
    installed.
    hab-sup(MN): The nathenharvey/sample-node-app service was successfully loaded
    Launch My Sample App
    sudo hab sup load nathenharvey/sample-node-app -s rolling

    View Slide

  52. Builder Demo

    View Slide

  53. http://YOUR_IP:8000

    View Slide

  54. $
    hab-sup(MN): Unloading "/hab/sup/default/specs/sample-node-app.spec"
    Unload the Application
    sudo hab sup unload nathenharvey/chef-sample-node-app

    View Slide

  55. $
    Terminate the Supervisor
    sudo hab sup term

    View Slide

  56. Automation travels with the
    application

    Existing & Cloud Native Software
    Application Automation

    View Slide

  57. What’s Next?
    •  Join the Habitat Slack Team - http://slack.habitat.sh/
    •  Work through the tutorial at https://www.habitat.sh/tutorials/
    •  Explore Habitat packages on the depot - https://app.habitat.sh/
    •  Explore the Habitat projects - https://github.com/habitat-sh
    •  Read Habitat Blog posts - https://www.habitat.sh/blog/

    View Slide

  58. Bonus Slides
    Because you are awesome!

    View Slide

  59. SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    USER ARTIFACT
    How we do it
    Packaging Applications
    Running Applications
    PLAN DEPOT
    DEPOT ARTIFACT
    BARE METAL
    CONTAINERS
    AMI
    VM

    View Slide

  60. Habitat and Modern Architectures
    ●  Habitat + Better Containers
    ○  Application Centric Containers
    ●  Habitat + Cluster Management (e.g. k8s)
    ○  Build all your workloads with Habitat’s Builder
    ○  Stateless services run on k8s
    ○  Stateful services connect via cloud service broker
    ■  Export to Container, VMs, Bare Metal as fits the workload
    ○  Manage all your workloads with Habitat’s Supervisor & Runtime

    View Slide

  61. LEADER
    STAND ALONE
    Topologies Update Strategies
    Running Applications
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    How we do it
    LEADER
    ElECTION
    ALL AT
    ONCE
    ROLLING

    View Slide

  62. SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    How we do it
    Security
    PUB KEY
    SYMMETRIC ENCRYPTION
    LOAD BALANCER
    Build Service
    BUILD
    SERVIC
    E
    USER
    SECRET
    PAYLOADS
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    ARTIFACT
    PLAN DEPOT

    View Slide

  63. What the modern application team gets
    Developers, System Administrators, CIOs; Enterprise and Tech Innovators
    ▪  Runs the same way in any
    environment
    ▪  Management travels with
    the application; no drift
    ▪  Autonomous and self-
    organizing
    ▪  Legacy and Greenfield
    ▪  Lets the enterprise
    modernize without re-
    writing the world
    ▪  Faster to build, easier to
    deploy, safer to manage
    ▪  Easiest way to deploy
    applications, containers,
    and microservices in
    production
    ▪  Developers can focus on
    building great applications
    ▪  Systems Administrators can
    focus on how those
    applications should behave
    ▪  Gives both a language they
    can share, with clear
    boundaries
    Simplification Acceleration Empowerment

    View Slide

  64. Habitat Build Philosophy
    ● 
    Container formats recreate the
    traditional model of infrastructure and
    applications.
    ● 
    Poor at abstracting the Build + Run
    aspects of Applications
    Libraries
    Operating System
    Application
    Application &
    Libraries
    ● 
    Habitat builds from the application
    down
    ● 
    Small lightweight OS included
    ● 
    Embedded Supervisor for Application
    Management
    ● 
    Builds have strict dependency version
    control
    Application Libraries
    OS

    View Slide

  65. Application Centric Containers with Habitat
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    CONTAINER
    ●  Provides a “DSL” for defining the build
    lifecycle of the application
    ●  Application run lifecycle is defined along
    side the application build lifecycle
    ●  Hab artifacts are based on application
    artifact, declared dependencies, and
    defined run lifecycle
    ●  Export hab artifacts to containers
    ●  Containers have built in supervisor for
    application operations concerns:
    ○  Configuration
    ○  Service Discovery
    ○  Clustering Topologies
    EXPORTS
    CONTAINER
    FORMATS

    View Slide

  66. Habitat + Container Orchestrators
    ●  Develop reusable
    application packages
    ●  Expose tunable settings
    ●  Export to your preferred
    Container Platform
    ●  Deploy your application
    in a scalable, topology
    aware format.
    BUILD DEPLOY MANAGE
    ●  Orchestrate and
    Automate your
    application config
    changes
    ●  Application lifecycle
    hooks respond to
    updates & events

    View Slide

  67. View Slide