Application Automation & Containerization With Habitat

Habitat is a simple, flexible way to build, deploy, and manage cloud native and modern, distributed applications. Habitat centers application configuration, management, and behavior around the application itself, not the infrastructure that the application runs on. It provides automation that can programmatically and declaratively build, deploy, and manage applications and services, both stateful and stateless. Each Habitat application can be deployed and run on various infrastructure environments including bare metal, VM, containers, and PaaS.

This workshop provides a quick introduction to Habitat during which participants will build, deploy, and run a multi-tier application as a way to explore some of Habitat's application automation features.

Find more about Habitat at https://www.habitat.sh/.

This workshop was offered at DevOpsDays Charlotte, 2017.

Transcript

1. Application Automation &
   Containerization With Habitat
   Nathen Harvey | @nathenharvey
   

   View Slide

2. Shifts in Modern Application Architecture
   Demand an application-centric operations model
   Infrastructure
   Centric
   Operations
   Application
   Centric
   Operations
   Cloud
   Containers
   Microservices
   UNIT OF VALUE is App
   UNIT OF WORK is Deployment
   UNIT OF VALUE is Server
   UNIT OF WORK is Configuration
   Infrastructure Automation Compliance Automation Application Automation
   EXISTING BUSINESS
   NEW BUSINESS
   

   View Slide

3. Application automation that enables
   modern application teams to build,
   deploy, and manage any application in
   any environment - from traditional data-
   centers to containerized microservices.
   Habitat
   

   View Slide

4. Lifecycle of any software
   Build Deploy Run
   

   View Slide

5. Application-centric Operations with Chef
   Chef Automate and Habitat for simple, consistent management of apps
   BUILD DEPLOY MANAGE
   Supervisor
   

   View Slide

6. SSH Into the Provided Workstation
   •  IP Address – on that little slip of paper we gave you
   •  Login: chef
   •  Password: habitat
   

   View Slide

7. $
   Log in to your remote workstation
   ssh chef@12.34.56.78
   

   View Slide

8. Using PuTTY on Windows
   

   View Slide

9. $
   hab 0.54.0/20180221022026
   Habitat is installed
   hab --version
   

   View Slide

10. Clean room environment
    Safe by default
    Installs dependences
    Habitat studio
    

    View Slide

11. $
    Habitat CLI Setup
    =================
    Welcome to hab setup. Let's get started.
    Set up a default origin
    Every package in Habitat belongs to an origin, which indicates the
    person or organization responsible for maintaining that package. Each
    origin also has a key used to cryptographically sign packages in that
    origin.
    Selecting a default origin tells package building operations such as
    'hab pkg build' what key should be used to sign the packages produced.
    If you do not set a default origin now, you will have to tell package
    building commands each time what origin to use.
    For more information on origins and how they are used in building
    packages, please consult the docs at
    https://www.habitat.sh/docs/create-packages-build/
    Set up a default origin? [Yes/no/quit] Yes
    Set-up the Habitat Studio (1 of 7)
    hab setup
    

    View Slide

12. $
    Enter the name of your origin. If you plan to publish your packages
    publicly, we recommend that you select one that is not already in use on
    the Habitat build service found at https://bldr.habitat.sh/.
    Origins must begin with a lowercase letter or number. Allowed characters
    include lowercase letters, numbers, _, -. No more than 255 characters.
    Default origin name: [default: chef] chef
    Set-up the Habitat Studio (2 of 7)
    

    View Slide

13. $
    Create origin key pair
    It doesn't look like you have a signing key for the origin `chef'.
    Without it, you won't be able to build new packages successfully.
    You can either create a new signing key now, or, if you are building
    packages for an origin that already exists, ask the owner to give you
    the signing key.
    For more information on the use of origin keys, please consult the
    documentation at https://www.habitat.sh/docs/concepts-keys/#origin-keys
    Create an origin key for `chef'? [Yes/no/quit] yes
    Set-up the Habitat Studio (3 of 7)
    

    View Slide

14. $
    Create origin key pair
    It doesn't look like you have a signing key for the origin `chef'.
    Without it, you won't be able to build new packages successfully.
    You can either create a new signing key now, or, if you are building
    packages for an origin that already exists, ask the owner to give you
    the signing key.
    For more information on the use of origin keys, please consult the
    documentation at https://www.habitat.sh/docs/concepts-keys/#origin-keys
    Create an origin key for `chef'? [Yes/no/quit] yes
    Set-up the Habitat Studio (4 of 7)
    

    View Slide

15. $
    GitHub Access Token
    While you can build and run Habitat packages without sharing them on the
    public depot, doing so allows you to collaborate with the Habitat
    community. In addition, it is how you can perform continuous deployment
    with Habitat.
    The GitHub personal access token needs the user:email and read:org OAuth
    scopes. Habitat uses the information provided through these scopes for
    authentication and to determine features based on team membership. You
    can set this up at https://github.com/settings/tokens
    If you would like to share your packages on the depot, please enter your
    GitHub access token. Otherwise, just enter No.
    For more information on sharing packages on the depot, please read the
    documentation at https://www.habitat.sh/docs/share-packages-overview/
    Set up a default GitHub access token? [Yes/no/quit] no
    Set-up the Habitat Studio (5 of 7)
    

    View Slide

16. $
    Analytics
    The `hab` command-line tool will optionally send anonymous usage data to
    Habitat's Google Analytics account. This is a strictly opt-in activity
    and no tracking will occur unless you respond affirmatively to the
    question below.
    We collect this data to help improve Habitat's user experience. For
    example, we would like to know the category of tasks users are
    performing, and which ones they are having trouble with (e.g. mistyping
    command line arguments).
    To see what kinds of data are sent and how they are anonymized, please
    read more about our analytics here:
    https://www.habitat.sh/docs/about-analytics/
    Enable analytics? [Yes/no/quit] yes
    Set-up the Habitat Studio (6 of 7)
    

    View Slide

17. $
    CLI Setup Complete
    That's all for now. Thanks for using Habitat!
    Set-up the Habitat Studio (7 of 7)
    

    View Slide

18. Starts with a plan
    

    View Slide

19. $
    » Attempting autodiscovery
    No scaffolding type was provided. Let's see if we can figure out what
    kind of application you're planning to package.
    ...
    For more information on any of the files:
    https://www.habitat.sh/docs/reference/plan-syntax/
    ★ An abode for your code is initialized!
    Build a Sample Plan
    hab plan init --with-all --with-callbacks --with-docs sample-plan
    

    View Slide

20. Plan
    ~/sample-node-app/habitat/plan.sh
    pkg_name=sample-node-app
    pkg_origin=your_origin
    pkg_origin=chef
    pkg_scaffolding="core/scaffolding-node"
    pkg_version="1.0.1"
    declare -A scaffolding_env
    # Define path to config file
    scaffolding_env[APP_CONFIG]="{{pkg.svc_config_path}}/config.json"
    -
    +
    

    View Slide

21. Plan
    ~/sample-node-app/habitat/plan.sh
    pkg_name=sample-node-app
    pkg_origin=your_origin
    pkg_origin=chef
    pkg_scaffolding="core/scaffolding-node"
    pkg_version="1.0.1"
    declare -A scaffolding_env
    # Define path to config file
    scaffolding_env[APP_CONFIG]="{{pkg.svc_config_path}}/config.json"
    -
    +
    $ cp ~/new-plan.sh sample-node-app/habitat/plan.sh
    If you don’t like manually editing files…
    

    View Slide

22. $
    Move to the Application’s Directory
    cd ~/sample-node-app
    

    View Slide

23. $
    ∵ Missing package for core/hab-studio/0.54.0
    » Installing core/hab-studio/0.54.0 from channel 'stable'
    ↓ Downloading core/hab-studio/0.54.0/20180221023422
    3.17 MB / 3.17 MB - [===============================] 100.00 % 79.21 MB/s
    ☛ Verifying core/hab-studio/0.54.0/20180221023422
    ✓ Installed core/hab-studio/0.54.0/20180221023422
    ★ Install of core/hab-studio/0.54.0/20180221023422 complete with 1 new packages installed.
    hab-studio: Creating Studio at /hab/studios/home--chef--sample-node-app (default)
    ...
    --> To prevent a Supervisor from running automatically in your
    Studio, export 'HAB_STUDIO_SUP=false' before running
    'hab studio enter'.
    Enter the Habitat Studio
    hab studio enter
    

    View Slide

24. #
    : Loading /src/habitat/plan.sh
    sample-node-app: Plan loaded
    sample-node-app: Validating plan metadata
    ...
    sample-node-app: Installed Path: /hab/pkgs/chef/sample-node-app/1.0.1/20180221155144
    sample-node-app: Artifact: /src/results/chef-sample-node-app-1.0.1-20180221155144-
    x86_64-linux.hart
    sample-node-app: Build Report: /src/results/last_build.env
    sample-node-app: SHA256 Checksum:
    811247255d0b53319f29f5090a72e186247fb000b8f8b362282a3761422a3cd3
    sample-node-app: Blake2b Checksum:
    bde6652ff19cd6ec8c336e466b200127aa7c60987bc3ba86917064237df0c421
    sample-node-app:
    sample-node-app: I love it when a plan.sh comes together.
    sample-node-app:
    sample-node-app: Build time: 0m39s
    Build the Application
    build
    

    View Slide

25. #
    hab-sup(MN): The chef/sample-node-app service was successfully loaded
    Run the Application in the Studio
    hab sup load chef/sample-node-app
    

    View Slide

26. #
    --> Tailing the Habitat Supervisor's output (use 'Ctrl+c' to stop)
    hab-sup(MR): Starting chef/sample-node-app
    sample-node-app.default(UCW): Watching user.toml
    sample-node-app.default(HK): init, compiled to /hab/svc/sample-node-app/hooks/init
    sample-node-app.default(HK): Hooks compiled
    sample-node-app.default(SR): Hooks recompiled
    default(CF): Updated app_env.sh
    9c88a9595def28f5914f08997b3cd3ba8d32f0e3eb2b4e4ea90264acd6acb66b
    default(CF): Updated config.json
    3f22842e8d737bbb107d9ac19afba42642eccf68a06ddfbdba70507b23b8498a
    sample-node-app.default(SR): Configuration recompiled
    sample-node-app.default(SR): Initializing
    sample-node-app.default(SV): Starting service as user=hab, group=hab
    Follow the Supervisor Log
    sup-log
    

    View Slide

27. http://YOUR_IP:8000
    

    View Slide

28. Installs automatically
    Keeps things running
    Manages configuration
    Supervisor runs services
    

    View Slide

29. #
    message = "Hello, World!"
    node_env = "production"
    [app]
    port = 8000
    Configuration Options
    hab sup config chef/sample-node-app
    

    View Slide

30. #
    Write Configuration Options to a File
    hab sup config chef/sample-node-app > config.toml
    

    View Slide

31. Update the configuration
    config.toml
    message = "Hello, World!"
    node_env = "production"
    [app]
    port = 8000
    port = 9000
    -
    +
    

    View Slide

32. Update the configuration
    config.toml
    message = "Hello, World!"
    node_env = "production"
    [app]
    port = 8000
    port = 9000
    -
    +
    # cp new-config.toml config.toml
    If you don’t like manually editing files…
    

    View Slide

33. #
    Apply the New Configuration
    hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml
    

    View Slide

34. Applying Configuration
    hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml
    •  peer - hostname or IP address of a running supervisor
    •  sample-node-app.default - the service group
    •  1 - the incarnation of this configuration
    •  config.toml - the file with the new configuration
    

    View Slide

35. #
    ∵ Missing package for core/hab-butterfly/0.54.0
    » Installing core/hab-butterfly/0.54.0 from channel 'stable'
    ↓ Downloading core/hab-butterfly/0.54.0/20180221020301
    ...
    ★ Install of core/hab-butterfly/0.54.0/20180221020301 complete with 1 new packages
    installed.
    » Applying configuration for sample-node-app.default incarnation 1
    Ω Creating service configuration
    ✓ Verified this configuration is valid TOML
    ↑ Applying to peer 127.0.0.1:9638
    ★ Applied configuration
    Apply the New Configuration
    hab config apply --peer 127.0.0.1 sample-node-app.default 1 config.toml
    

    View Slide

36. http://YOUR_IP:9000
    

    View Slide

37. Docker
    tar
    Kubernetes
    Post-process packages
    

    View Slide

38. #
    ∵ Missing package for core/hab-pkg-export-docker/0.54.0
    » Installing core/hab-pkg-export-docker/0.54.0 from channel 'stable'
    ↓ Downloading core/hab-pkg-export-docker/0.54.0/20180221020412
    2.40 MB / 2.40 MB - [===============================] 100.00 % 89.49 MB/s
    ☛ Verifying core/hab-pkg-export-docker/0.54.0/20180221020412
    ...
    Removing intermediate container 582430ddac0a
    ---> 0cb96cce166b
    Successfully built 0cb96cce166b
    Successfully tagged chef/sample-node-app:1.0.1-20180221155145
    Successfully tagged chef/sample-node-app:1.0.1
    Successfully tagged chef/sample-node-app:latest
    ☒ Deleting temporary files
    ★ Docker image 'chef/sample-node-app' created with tags: 1.0.1-20180221155145, 1.0.1,
    latest
    Ω Creating build report /src/results/last_docker_export.env
    Export to Docker
    hab pkg export docker results/chef-sample-node-app-*.hart
    

    View Slide

39. #
    Leave the Studio
    exit
    

    View Slide

40. $
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB
    Inspect the Local Docker Images
    docker images
    

    View Slide

41. $
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB
    Run in a Docker Container
    docker run -it -p 8000:8000 chef/sample-node-app
    

    View Slide

42. http://YOUR_IP:8000
    

    View Slide

43. $
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chef/sample-node-app 1.0.1 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app 1.0.1-20180221155145 0cb96cce166b 4 minutes ago 207MB
    chef/sample-node-app latest 0cb96cce166b 4 minutes ago 207MB
    Run in a Docker Container with Updated Configuration
    docker run -e HAB_SAMPLE_NODE_APP="$(cat config.toml)" -p 9000:9000 -it chef/sample-node-app
    

    View Slide

44. http://YOUR_IP:9000
    

    View Slide

45. $
    Copy the Origin Key to the Proper Location
    sudo cp ~/.hab/cache/keys/chef-*.pub /hab/cache/keys/
    

    View Slide

46. $
    ∵ Missing package for core/hab-launcher
    » Installing core/hab-launcher from channel 'stable'
    ↓ Downloading core/hab-launcher/6925/20180202215752
    4.78 MB / 4.78 MB / [===============================] 100.00 % 31.47 MB/s
    ...
    sample-node-app.default(SR): Configuration recompiled
    sample-node-app.default(SR): Initializing
    sample-node-app.default(SV): Starting service as user=hab, group=hab
    Start the Application Locally
    sudo hab sup start results/chef-sample-node-app-*.hart
    

    View Slide

47. http://YOUR_IP:8000
    

    View Slide

48. $
    hab-sup(MR): Supervisor Member-ID 100b72aea4104c9ba52972622db4bda1
    hab-sup(MR): Starting chef/sample-node-app/1.0.1/20180222023512
    sample-node-app.default(UCW): Watching user.toml
    hab-sup(MR): Starting gossip-listener on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    sample-node-app.default(HK): Hooks compiled
    sample-node-app.default(SR): Initializing
    sample-node-app.default(SV): Starting service as user=hab, group=hab
    sample-node-app.default(HK): Hooks compiled
    Join My Supervisor Ring
    sudo hab sup run --peer 172.31.28.111
    

    View Slide

49. http://YOUR_IP:9631/
    butterfly
    

    View Slide

50. Supervisors provide update strategies
    

    View Slide

51. $
    hab-sup(MN): Missing package for nathenharvey/sample-node-app
    » Installing nathenharvey/sample-node-app from channel 'stable'
    ...
    ✓ Installed nathenharvey/sample-node-app/1.0.8/20180221181902
    ★ Install of nathenharvey/sample-node-app/1.0.8/20180221181902 complete with 1 new packages
    installed.
    hab-sup(MN): The nathenharvey/sample-node-app service was successfully loaded
    Launch My Sample App
    sudo hab sup load nathenharvey/sample-node-app -s rolling
    

    View Slide

52. Builder Demo
    

    View Slide

53. http://YOUR_IP:8000
    

    View Slide

54. $
    hab-sup(MN): Unloading "/hab/sup/default/specs/sample-node-app.spec"
    Unload the Application
    sudo hab sup unload nathenharvey/chef-sample-node-app
    

    View Slide

55. $
    Terminate the Supervisor
    sudo hab sup term
    

    View Slide

56. Automation travels with the
    application
    
    Existing & Cloud Native Software
    Application Automation
    

    View Slide

57. What’s Next?
    •  Join the Habitat Slack Team - http://slack.habitat.sh/
    •  Work through the tutorial at https://www.habitat.sh/tutorials/
    •  Explore Habitat packages on the depot - https://app.habitat.sh/
    •  Explore the Habitat projects - https://github.com/habitat-sh
    •  Read Habitat Blog posts - https://www.habitat.sh/blog/
    

    View Slide

58. Bonus Slides
    Because you are awesome!
    

    View Slide

59. SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    USER ARTIFACT
    How we do it
    Packaging Applications
    Running Applications
    PLAN DEPOT
    DEPOT ARTIFACT
    BARE METAL
    CONTAINERS
    AMI
    VM
    

    View Slide

60. Habitat and Modern Architectures
    ●  Habitat + Better Containers
    ○  Application Centric Containers
    ●  Habitat + Cluster Management (e.g. k8s)
    ○  Build all your workloads with Habitat’s Builder
    ○  Stateless services run on k8s
    ○  Stateful services connect via cloud service broker
    ■  Export to Container, VMs, Bare Metal as fits the workload
    ○  Manage all your workloads with Habitat’s Supervisor & Runtime
    

    View Slide

61. LEADER
    STAND ALONE
    Topologies Update Strategies
    Running Applications
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    How we do it
    LEADER
    ElECTION
    ALL AT
    ONCE
    ROLLING
    

    View Slide

62. SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    How we do it
    Security
    PUB KEY
    SYMMETRIC ENCRYPTION
    LOAD BALANCER
    Build Service
    BUILD
    SERVIC
    E
    USER
    SECRET
    PAYLOADS
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    ARTIFACT
    PLAN DEPOT
    

    View Slide

63. What the modern application team gets
    Developers, System Administrators, CIOs; Enterprise and Tech Innovators
    ▪  Runs the same way in any
    environment
    ▪  Management travels with
    the application; no drift
    ▪  Autonomous and self-
    organizing
    ▪  Legacy and Greenfield
    ▪  Lets the enterprise
    modernize without re-
    writing the world
    ▪  Faster to build, easier to
    deploy, safer to manage
    ▪  Easiest way to deploy
    applications, containers,
    and microservices in
    production
    ▪  Developers can focus on
    building great applications
    ▪  Systems Administrators can
    focus on how those
    applications should behave
    ▪  Gives both a language they
    can share, with clear
    boundaries
    Simplification Acceleration Empowerment
    

    View Slide

64. Habitat Build Philosophy
    ● 
    Container formats recreate the
    traditional model of infrastructure and
    applications.
    ● 
    Poor at abstracting the Build + Run
    aspects of Applications
    Libraries
    Operating System
    Application
    Application &
    Libraries
    ● 
    Habitat builds from the application
    down
    ● 
    Small lightweight OS included
    ● 
    Embedded Supervisor for Application
    Management
    ● 
    Builds have strict dependency version
    control
    Application Libraries
    OS
    

    View Slide

65. Application Centric Containers with Habitat
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    CONTAINER
    ●  Provides a “DSL” for defining the build
    lifecycle of the application
    ●  Application run lifecycle is defined along
    side the application build lifecycle
    ●  Hab artifacts are based on application
    artifact, declared dependencies, and
    defined run lifecycle
    ●  Export hab artifacts to containers
    ●  Containers have built in supervisor for
    application operations concerns:
    ○  Configuration
    ○  Service Discovery
    ○  Clustering Topologies
    EXPORTS
    CONTAINER
    FORMATS
    

    View Slide

66. Habitat + Container Orchestrators
    ●  Develop reusable
    application packages
    ●  Expose tunable settings
    ●  Export to your preferred
    Container Platform
    ●  Deploy your application
    in a scalable, topology
    aware format.
    BUILD DEPLOY MANAGE
    ●  Orchestrate and
    Automate your
    application config
    changes
    ●  Application lifecycle
    hooks respond to
    updates & events
    

    View Slide

67. View Slide