$30 off During Our Annual Pro Sale. View Details »

Hands on with Habitat

Nathen Harvey
February 08, 2017

Hands on with Habitat

Chef recently announced a new open source framework for application automation, Habitat. We are embarking on a tour of cities around the world to provide you with hands-on experience with the project. The workshops are free to attend - we’d love for you to join us.

Habitat is Chef’s open-source project for application automation. It simplifies container management by packaging applications in a compact, atomic, and easily auditable format that makes it easier to deploy your application on various container runtimes. It also helps you run stateful application services (think Redis, Mongo, etc) on cloud instances, VMs, or bare metal. Find out more at habitat.sh.

This will be a hands-on workshop aimed at anyone involved in building, deploying, or managing applications. You may be a developer or may come from an infrastructure background. You may currently be working with container technology or a microservices architecture (or planning to).

Nathen Harvey

February 08, 2017
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. • 10:00 – Welcome!
    • 10:15 – Introduction to Habitat
    • 11:00 – Hacking!
    • 12:40 – Demos
    • 13:00 – Lunch
    • 14:00 – Hacking!
    • 15:00 - Demos

    View Slide

  2. Habitat Code of Conduct
    https://github.com/habitat-sh/habitat/blob/master/CODE_OF_CONDUCT.md
    We value the participation of each member of the community and want all
    attendees to have an enjoyable and fulfilling experience. Accordingly, all
    attendees are expected to show respect and courtesy to other attendees
    throughout all Meetup events.

    To make clear what is expected, all delegates/attendees, speakers,
    exhibitors, organizers, and volunteers at any Habitat event are required to
    conform to our Code of Conduct. Organizers will enforce this code
    throughout events.

    View Slide

  3. • Join the Habitat Community
    • Build, manage, deploy an app with
    Habitat
    • Share your experience
    • Win beer!

    View Slide

  4. http://slack.habitat.sh

    View Slide

  5. View Slide

  6. https://goo.gl/T5ptFm

    View Slide

  7. Modern Applications
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    SERVICE
    SUPERVISOR
    DEPOT ARTIFACT
    Bare Metal
    Container
    Cloud Instance
    Virtual Machine

    View Slide

  8. Let’s Run an Application!
    •  Run the National Parks application
    •  MongoDB
    •  Java Application running in Tomcat
    •  Service Discovery
    •  Injecting configuration parameters

    View Slide

  9. View Slide

  10. https://goo.gl/T5ptFm

    View Slide

  11. Let’s Run an Application!
    •  Run the National Parks application
    •  MongoDB
    •  Java Application running in Tomcat
    •  Service Discovery
    •  Injecting configuration parameters

    View Slide

  12. Start MongoDB
    hab-sup(MN): Starting nathenharvey/mongodb
    hab-sup(MR): Butterfly Member ID 7b0aff5fcb7a428a91c20ef09ec74d49
    mongodb.default(SR): Process will run as user=hab, group=hab
    hab-sup(MR): Starting butterfly on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    hab-sup(SC): Updated mongod.conf
    f9886a510442a32e91d38a79b0baad2a4bb589ba36fed74e2ea3493d517d8bd5
    mongodb.default(SR): Initializing
    mongodb.default(SV): Starting
    mongodb.default(O): note: noprealloc may hurt performance in many applications
    sudo hab start nathenharvey/mongodb

    View Slide

  13. OK, so…
    •  Where did nathenharvey/mongodb come
    from?
    •  Where is it stored?
    •  How is it running?

    View Slide

  14. Central location
    Publicly hosted
    Lightweight views
    Stored in a depot

    View Slide

  15. Binaries and Config
    Metadata
    Side-by-side
    Packages on disk

    View Slide

  16. Installs automatically
    Keeps things running

    Supervisor runs services

    View Slide

  17. Start the Java Application
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID 83689b4180e64832a0f1c2b2e6810eac
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9638
    hab-sup(ER)[src/error.rs:330:8]: Butterfly error: Cannot bind to port:
    Error { repr: Os { code: 98, message: "Address already in use" } }
    sudo hab start nathenharvey/national-parks

    View Slide

  18. Supervisors self-organize
    Peers

    Service Groups

    Gossip

    Availability
    increases with
    scale

    View Slide

  19. Supervisors provide a REST API
    External Actors

    Health and
    Status

    Supervisor
    Debugging

    View Slide

  20. Specify supervisor’s ports
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID 5b006ec73fad42a9a4611f573139b4ee
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9639
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9632
    hab-sup(SC): Updated mongoimport-opts.conf 4f4d6fa7d8a6bfd193b347d76ab74e7d0e7a3b4b21255711bd8f49d4119d8d86
    hab-sup(SC): Updated catalina-opts.conf 4f4d6fa7d8a6bfd193b347d76ab74e7d0e7a3b4b21255711bd8f49d4119d8d86
    national-parks.default hook[init]:(PH): Seeding Mongo Collection
    national-parks.default hook[init]:(PH): $MONGOIMPORT_OPTS=
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.015+0000 connected to: localhost
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.015+0000 dropping: demo.nationalparks
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.019+0000 Failed: not authorized on demo to execute command { drop:
    "nationalparks" }
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.019+0000 imported 0 documents
    national-parks.default(SR): Initialization failed: hab-sup(PH)[src/package/hooks.rs:89:16]: init hook failed to run with exit code 1
    sudo hab start nathenharvey/national-parks --listen-gossip 0.0.0.0:9639 --listen-http 0.0.0.0:9632

    View Slide

  21. What’s wrong?
    •  Application is unable to connect to
    MongoDB on localhost.
    •  Application assumes MongoDB is available
    on localhost

    View Slide

  22. Service Discovery!
    •  Supervisors are organized in a gossip mesh
    •  Connect to any other supervisor as a peer
    •  Bind to the database

    View Slide

  23. Bind to MongoDB
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID aa0ff2d192d24e4cb8fe6de589f1fbb1
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9639
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9632
    national-parks.default hook[init]:(PH): Seeding Mongo Collection
    national-parks.default hook[init]:(PH): $MONGOIMPORT_OPTS=
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.256+0000 connected to: localhost
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.257+0000 dropping: demo.nationalparks
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.260+0000 Failed: not authorized on demo to execute
    command { drop: "nationalparks" }
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.260+0000 imported 0 documents
    national-parks.default(SR): Initialization failed: hab-sup(PH)[src/package/hooks.rs:89:16]: init hook failed to run
    with exit code 1
    sudo hab start nathenharvey/national-parks --listen-gossip 0.0.0.0:9639 --listen-http 0.0.0.0:9632
    --peer 127.0.0.1 --bind database:mongodb.default

    View Slide

  24. Still cannot connect to MongoDB?!
    •  OK, we have to know a little more about
    MongoDB
    •  It needs to bind to the proper address
    •  We’re going to disable security – YOLO!

    View Slide

  25. Manages configuration
    Installs automatically
    Keeps things running
    Supervisor runs services

    View Slide

  26. What are the tunables for mongodb?
    [mongod]
    ...
    [mongod.net]
    port = 27017
    bind_ip = "127.0.0.1"
    ...
    [mongod.security]
    key_file = ""
    cluster_auth_mode = "keyFile"
    ...
    hab sup config nathenharvey/mongodb

    View Slide

  27. Restart MongoDB, override default configuration
    hab-sup(MN): Starting nathenharvey/mongodb
    hab-sup(MR): Butterfly Member ID a12737657e7943efb772f0d53a576aa7
    mongodb.default(SR): Process will run as user=hab, group=hab
    hab-sup(MR): Starting butterfly on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    mongodb.default(SR): Initializing
    mongodb.default(SV): Starting
    mongodb.default(O): note: noprealloc may hurt performance in many
    applications
    HAB_MONGODB="[mongod.net]
    bind_ip = '0.0.0.0'
    [mongod.security]
    cluster_auth_mode = ''" sudo -E hab start nathenharvey/mongodb

    View Slide

  28. Restart the Application
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID 434519a93f4a4146bcd5061337640cd3
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9639
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9632
    national-parks.default hook[init]:(PH): Seeding Mongo Collection
    national-parks.default hook[init]:(PH): $MONGOIMPORT_OPTS=
    national-parks.default hook[init]:(PH): 2017-02-08T00:42:14.474+0000 connected to: localhost
    national-parks.default hook[init]:(PH): 2017-02-08T00:42:14.474+0000 dropping: demo.nationalparks
    national-parks.default hook[init]:(PH): 2017-02-08T00:42:14.510+0000 imported 359 documents
    national-parks.default(SR): Initializing
    national-parks.default(SV): Starting
    national-parks.default(O): Starting Apache Tomcat
    national-parks.default(O): $CATALINA_OPTS=
    national-parks.default(O): 08-Feb-2017 00:42:15.060 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.5.9
    ...
    national-parks.default(O): 08-Feb-2017 00:42:19.398 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [http-nio-8080]
    national-parks.default(O): 08-Feb-2017 00:42:19.403 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [ajp-nio-8009]
    national-parks.default(O): 08-Feb-2017 00:42:19.404 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 1156 ms
    sudo hab start nathenharvey/national-parks --listen-gossip 0.0.0.0:9639 --listen-http 0.0.0.0:9632
    --peer 127.0.0.1 --bind database:mongodb.default

    View Slide

  29. http://YOUR_IP:8080/national-parks

    View Slide

  30. Supervisors provide a REST API
    External Actors

    Health and
    Status

    Supervisor
    Debugging

    View Slide

  31. REST API
    •  http://YOUR_IP:9631/services
    •  http://YOUR_IP:9631/services/mongodb/default/config

    •  http://YOUR_IP:9632/services
    •  http://YOUR_IP:9632/services/national-parks/default/config

    View Slide

  32. HTTP API
    • /census
    • /services
    • /services/{name}/{group}/config
    • /services/{name}/{group}/{organization}/config
    • /services/{name}/{group}/health
    • /services/{name}/{group}/{organization}/health
    • /butterfly

    View Slide

  33. Operable Applications
    • Isolated
    • Immutable
    • Atomic
    • Configurable
    • Common interface for monitoring health
    • Rebuild from source
    • Common packaging
    • Runtime Independence

    View Slide

  34. Build
    Deploy
    Manage

    View Slide

  35. Starts with a plan

    View Slide

  36. ©2016 Chef Software Inc.
    Simple, native, and declaritive
    pkg_name=redis
    pkg_origin=core
    pkg_version=3.0.7
    pkg_license=('BSD')
    pkg_maintainer="The Habitat Maintainers "
    pkg_source=http://download.redis.io/releases/${pkg_name}-${pkg_version}.tar.gz
    pkg_shasum=b2a791c4ea3bb7268795c45c6321ea5abcc24457178373e6a6e3be6372737f23
    pkg_bin_dirs=(bin)
    pkg_build_deps=(core/make core/gcc)
    pkg_deps=(core/glibc)
    pkg_svc_run="bin/redis-server $pkg_svc_config_path/redis.config"
    pkg_expose=(6379)
    do_build() {
    make
    }
    plan.sh
    Low
    abstraction

    Complete
    dependencies

    Declare
    services

    Simple
    functions

    View Slide

  37. Clean room environment
    Safe by default
    Installs dependencies
    Built in a studio

    View Slide

  38. Central location
    Publicly hosted
    Lightweight views
    Stored in a depot

    View Slide

  39. Easy pattern
    Shortcuts
    Increasing specificity
    Installed by hab

    View Slide

  40. Binaries and Config
    Metadata
    Side-by-side
    Packages on disk

    View Slide

  41. Docker
    ACI
    Mesosphere
    Ye Olde Tarball
    Post-process packages

    View Slide

  42. Installs automatically
    Keeps things running
    Manages configuration
    Supervisor runs services

    View Slide

  43. Supervisors self-organize
    Peers

    Service Groups

    Gossip

    Availability
    increases with
    scale

    View Slide

  44. Supervisors provide a REST API
    External Actors

    Health and
    Status

    Supervisor
    Debugging

    View Slide

  45. Supervisors understand topologies
    The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
    Dynamic
    configuration

    Service group
    level

    Uses the ring

    View Slide

  46. Supervisors provide update strategies
    Topology
    aware

    Tracks views in
    the depot

    Ideal for
    Continuous
    Delivery

    View Slide

  47. Automation travels with the
    application

    Existing & Cloud Native Software
    Application Automation

    View Slide

  48. It’s all open source

    Apache License

    View Slide

  49. https://www.habitat.sh/community/

    View Slide

  50. What’s Next?
    •  Use a provided Habitat development environment, or install Habitat
    locally
    •  Join the Habitat Slack Team - http://slack.habitat.sh/
    •  Bind your Java application to your neighbor’s MongoDB
    •  Work through the tutorial at https://www.habitat.sh/tutorials/
    •  Explore Habitat packages on the depot - https://app.habitat.sh/
    •  Explore the Habitat projects - https://github.com/habitat-sh
    •  Read Habitat Blog posts - https://blog.chef.io/category/habitat/
    •  Package one of your own applications
    •  Demo your work throughout the day
    •  Win Beer!

    View Slide