Hands on with Habitat

Chef recently announced a new open source framework for application automation, Habitat. We are embarking on a tour of cities around the world to provide you with hands-on experience with the project. The workshops are free to attend - we’d love for you to join us.

Habitat is Chef’s open-source project for application automation. It simplifies container management by packaging applications in a compact, atomic, and easily auditable format that makes it easier to deploy your application on various container runtimes. It also helps you run stateful application services (think Redis, Mongo, etc) on cloud instances, VMs, or bare metal. Find out more at habitat.sh.

This will be a hands-on workshop aimed at anyone involved in building, deploying, or managing applications. You may be a developer or may come from an infrastructure background. You may currently be working with container technology or a microservices architecture (or planning to).

Transcript

1. • 10:00 – Welcome!
   • 10:15 – Introduction to Habitat
   • 11:00 – Hacking!
   • 12:40 – Demos
   • 13:00 – Lunch
   • 14:00 – Hacking!
   • 15:00 - Demos
   

   View Slide

2. Habitat Code of Conduct
   https://github.com/habitat-sh/habitat/blob/master/CODE_OF_CONDUCT.md
   We value the participation of each member of the community and want all
   attendees to have an enjoyable and fulfilling experience. Accordingly, all
   attendees are expected to show respect and courtesy to other attendees
   throughout all Meetup events.
   
   To make clear what is expected, all delegates/attendees, speakers,
   exhibitors, organizers, and volunteers at any Habitat event are required to
   conform to our Code of Conduct. Organizers will enforce this code
   throughout events.
   
   

   View Slide

3. • Join the Habitat Community
   • Build, manage, deploy an app with
   Habitat
   • Share your experience
   • Win beer!
   

   View Slide

4. http://slack.habitat.sh
   

   View Slide

5. View Slide

6. https://goo.gl/T5ptFm
   

   View Slide

7. Modern Applications
   SERVICE
   SUPERVISOR
   SERVICE
   SUPERVISOR
   SERVICE
   SUPERVISOR
   SERVICE
   SUPERVISOR
   DEPOT ARTIFACT
   Bare Metal
   Container
   Cloud Instance
   Virtual Machine
   

   View Slide

8. Let’s Run an Application!
   •  Run the National Parks application
   •  MongoDB
   •  Java Application running in Tomcat
   •  Service Discovery
   •  Injecting configuration parameters
   

   View Slide

9. View Slide

10. https://goo.gl/T5ptFm
    

    View Slide

11. Let’s Run an Application!
    •  Run the National Parks application
    •  MongoDB
    •  Java Application running in Tomcat
    •  Service Discovery
    •  Injecting configuration parameters
    

    View Slide

12. Start MongoDB
    hab-sup(MN): Starting nathenharvey/mongodb
    hab-sup(MR): Butterfly Member ID 7b0aff5fcb7a428a91c20ef09ec74d49
    mongodb.default(SR): Process will run as user=hab, group=hab
    hab-sup(MR): Starting butterfly on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    hab-sup(SC): Updated mongod.conf
    f9886a510442a32e91d38a79b0baad2a4bb589ba36fed74e2ea3493d517d8bd5
    mongodb.default(SR): Initializing
    mongodb.default(SV): Starting
    mongodb.default(O): note: noprealloc may hurt performance in many applications
    sudo hab start nathenharvey/mongodb
    

    View Slide

13. OK, so…
    •  Where did nathenharvey/mongodb come
    from?
    •  Where is it stored?
    •  How is it running?
    

    View Slide

14. Central location
    Publicly hosted
    Lightweight views
    Stored in a depot
    

    View Slide

15. Binaries and Config
    Metadata
    Side-by-side
    Packages on disk
    

    View Slide

16. Installs automatically
    Keeps things running
    
    Supervisor runs services
    

    View Slide

17. Start the Java Application
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID 83689b4180e64832a0f1c2b2e6810eac
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9638
    hab-sup(ER)[src/error.rs:330:8]: Butterfly error: Cannot bind to port:
    Error { repr: Os { code: 98, message: "Address already in use" } }
    sudo hab start nathenharvey/national-parks
    

    View Slide

18. Supervisors self-organize
    Peers
    
    Service Groups
    
    Gossip
    
    Availability
    increases with
    scale
    

    View Slide

19. Supervisors provide a REST API
    External Actors
    
    Health and
    Status
    
    Supervisor
    Debugging
    

    View Slide

20. Specify supervisor’s ports
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID 5b006ec73fad42a9a4611f573139b4ee
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9639
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9632
    hab-sup(SC): Updated mongoimport-opts.conf 4f4d6fa7d8a6bfd193b347d76ab74e7d0e7a3b4b21255711bd8f49d4119d8d86
    hab-sup(SC): Updated catalina-opts.conf 4f4d6fa7d8a6bfd193b347d76ab74e7d0e7a3b4b21255711bd8f49d4119d8d86
    national-parks.default hook[init]:(PH): Seeding Mongo Collection
    national-parks.default hook[init]:(PH): $MONGOIMPORT_OPTS=
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.015+0000 connected to: localhost
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.015+0000 dropping: demo.nationalparks
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.019+0000 Failed: not authorized on demo to execute command { drop:
    "nationalparks" }
    national-parks.default hook[init]:(PH): 2017-02-08T00:24:51.019+0000 imported 0 documents
    national-parks.default(SR): Initialization failed: hab-sup(PH)[src/package/hooks.rs:89:16]: init hook failed to run with exit code 1
    sudo hab start nathenharvey/national-parks --listen-gossip 0.0.0.0:9639 --listen-http 0.0.0.0:9632
    

    View Slide

21. What’s wrong?
    •  Application is unable to connect to
    MongoDB on localhost.
    •  Application assumes MongoDB is available
    on localhost
    

    View Slide

22. Service Discovery!
    •  Supervisors are organized in a gossip mesh
    •  Connect to any other supervisor as a peer
    •  Bind to the database
    

    View Slide

23. Bind to MongoDB
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID aa0ff2d192d24e4cb8fe6de589f1fbb1
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9639
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9632
    national-parks.default hook[init]:(PH): Seeding Mongo Collection
    national-parks.default hook[init]:(PH): $MONGOIMPORT_OPTS=
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.256+0000 connected to: localhost
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.257+0000 dropping: demo.nationalparks
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.260+0000 Failed: not authorized on demo to execute
    command { drop: "nationalparks" }
    national-parks.default hook[init]:(PH): 2017-02-08T00:32:15.260+0000 imported 0 documents
    national-parks.default(SR): Initialization failed: hab-sup(PH)[src/package/hooks.rs:89:16]: init hook failed to run
    with exit code 1
    sudo hab start nathenharvey/national-parks --listen-gossip 0.0.0.0:9639 --listen-http 0.0.0.0:9632
    --peer 127.0.0.1 --bind database:mongodb.default
    

    View Slide

24. Still cannot connect to MongoDB?!
    •  OK, we have to know a little more about
    MongoDB
    •  It needs to bind to the proper address
    •  We’re going to disable security – YOLO!
    

    View Slide

25. Manages configuration
    Installs automatically
    Keeps things running
    Supervisor runs services
    

    View Slide

26. What are the tunables for mongodb?
    [mongod]
    ...
    [mongod.net]
    port = 27017
    bind_ip = "127.0.0.1"
    ...
    [mongod.security]
    key_file = ""
    cluster_auth_mode = "keyFile"
    ...
    hab sup config nathenharvey/mongodb
    

    View Slide

27. Restart MongoDB, override default configuration
    hab-sup(MN): Starting nathenharvey/mongodb
    hab-sup(MR): Butterfly Member ID a12737657e7943efb772f0d53a576aa7
    mongodb.default(SR): Process will run as user=hab, group=hab
    hab-sup(MR): Starting butterfly on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    mongodb.default(SR): Initializing
    mongodb.default(SV): Starting
    mongodb.default(O): note: noprealloc may hurt performance in many
    applications
    HAB_MONGODB="[mongod.net]
    bind_ip = '0.0.0.0'
    [mongod.security]
    cluster_auth_mode = ''" sudo -E hab start nathenharvey/mongodb
    

    View Slide

28. Restart the Application
    hab-sup(MN): Starting nathenharvey/national-parks
    hab-sup(MR): Butterfly Member ID 434519a93f4a4146bcd5061337640cd3
    national-parks.default(SR): Process will run as user=root, group=root
    hab-sup(MR): Starting butterfly on 0.0.0.0:9639
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9632
    national-parks.default hook[init]:(PH): Seeding Mongo Collection
    national-parks.default hook[init]:(PH): $MONGOIMPORT_OPTS=
    national-parks.default hook[init]:(PH): 2017-02-08T00:42:14.474+0000 connected to: localhost
    national-parks.default hook[init]:(PH): 2017-02-08T00:42:14.474+0000 dropping: demo.nationalparks
    national-parks.default hook[init]:(PH): 2017-02-08T00:42:14.510+0000 imported 359 documents
    national-parks.default(SR): Initializing
    national-parks.default(SV): Starting
    national-parks.default(O): Starting Apache Tomcat
    national-parks.default(O): $CATALINA_OPTS=
    national-parks.default(O): 08-Feb-2017 00:42:15.060 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.5.9
    ...
    national-parks.default(O): 08-Feb-2017 00:42:19.398 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [http-nio-8080]
    national-parks.default(O): 08-Feb-2017 00:42:19.403 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [ajp-nio-8009]
    national-parks.default(O): 08-Feb-2017 00:42:19.404 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 1156 ms
    sudo hab start nathenharvey/national-parks --listen-gossip 0.0.0.0:9639 --listen-http 0.0.0.0:9632
    --peer 127.0.0.1 --bind database:mongodb.default
    

    View Slide

29. http://YOUR_IP:8080/national-parks
    

    View Slide

30. Supervisors provide a REST API
    External Actors
    
    Health and
    Status
    
    Supervisor
    Debugging
    

    View Slide

31. REST API
    •  http://YOUR_IP:9631/services
    •  http://YOUR_IP:9631/services/mongodb/default/config
    
    •  http://YOUR_IP:9632/services
    •  http://YOUR_IP:9632/services/national-parks/default/config
    
    

    View Slide

32. HTTP API
    • /census
    • /services
    • /services/{name}/{group}/config
    • /services/{name}/{group}/{organization}/config
    • /services/{name}/{group}/health
    • /services/{name}/{group}/{organization}/health
    • /butterfly
    

    View Slide

33. Operable Applications
    • Isolated
    • Immutable
    • Atomic
    • Configurable
    • Common interface for monitoring health
    • Rebuild from source
    • Common packaging
    • Runtime Independence
    

    View Slide

34. Build
    Deploy
    Manage
    

    View Slide

35. Starts with a plan
    

    View Slide

36. ©2016 Chef Software Inc.
    Simple, native, and declaritive
    pkg_name=redis
    pkg_origin=core
    pkg_version=3.0.7
    pkg_license=('BSD')
    pkg_maintainer="The Habitat Maintainers "
    pkg_source=http://download.redis.io/releases/${pkg_name}-${pkg_version}.tar.gz
    pkg_shasum=b2a791c4ea3bb7268795c45c6321ea5abcc24457178373e6a6e3be6372737f23
    pkg_bin_dirs=(bin)
    pkg_build_deps=(core/make core/gcc)
    pkg_deps=(core/glibc)
    pkg_svc_run="bin/redis-server $pkg_svc_config_path/redis.config"
    pkg_expose=(6379)
    do_build() {
    make
    }
    plan.sh
    Low
    abstraction
    
    Complete
    dependencies
    
    Declare
    services
    
    Simple
    functions
    

    View Slide

37. Clean room environment
    Safe by default
    Installs dependencies
    Built in a studio
    

    View Slide

38. Central location
    Publicly hosted
    Lightweight views
    Stored in a depot
    

    View Slide

39. Easy pattern
    Shortcuts
    Increasing specificity
    Installed by hab
    

    View Slide

40. Binaries and Config
    Metadata
    Side-by-side
    Packages on disk
    

    View Slide

41. Docker
    ACI
    Mesosphere
    Ye Olde Tarball
    Post-process packages
    

    View Slide

42. Installs automatically
    Keeps things running
    Manages configuration
    Supervisor runs services
    

    View Slide

43. Supervisors self-organize
    Peers
    
    Service Groups
    
    Gossip
    
    Availability
    increases with
    scale
    

    View Slide

44. Supervisors provide a REST API
    External Actors
    
    Health and
    Status
    
    Supervisor
    Debugging
    

    View Slide

45. Supervisors understand topologies
    The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
    Dynamic
    configuration
    
    Service group
    level
    
    Uses the ring
    

    View Slide

46. Supervisors provide update strategies
    Topology
    aware
    
    Tracks views in
    the depot
    
    Ideal for
    Continuous
    Delivery
    

    View Slide

47. Automation travels with the
    application
    
    Existing & Cloud Native Software
    Application Automation
    

    View Slide

48. It’s all open source
    
    Apache License
    

    View Slide

49. https://www.habitat.sh/community/
    

    View Slide

50. What’s Next?
    •  Use a provided Habitat development environment, or install Habitat
    locally
    •  Join the Habitat Slack Team - http://slack.habitat.sh/
    •  Bind your Java application to your neighbor’s MongoDB
    •  Work through the tutorial at https://www.habitat.sh/tutorials/
    •  Explore Habitat packages on the depot - https://app.habitat.sh/
    •  Explore the Habitat projects - https://github.com/habitat-sh
    •  Read Habitat Blog posts - https://blog.chef.io/category/habitat/
    •  Package one of your own applications
    •  Demo your work throughout the day
    •  Win Beer!
    

    View Slide