Application Automation & Containerization With Habitat

Habitat builds and manages modern applications following the practices and principles of modern application teams. Build applications of any flavor, from microservices to traditional applications. Deploy applications in any operational environment from bare metal to containers. Habitat provides consistent, repeatable, auditable applications that lower operational complexity and simplify development workflows.

This hands-on workshop is for anyone involved in building, deploying, or managing applications. The workshop will give you experience running and building applications with Habitat and describe the application-first approach that Habitat provides. Find out more at habitat.sh.

This workshop was offered at DevOpsDays Minneapolis 2017.

Transcript

1. Application Automation
   with Habitat
   Nathen Harvey – nharvey@chef.io - @nathenharvey
   Nicole Johnson – nj@chef.io - @_ncjohnson
   

   View Slide

2. • 1515 – Welcome!
   • 1530 - Modern applications, modern teams
   • 1545 - Running a Habitat Application
   • 1630 - Next steps and wrap-up
   Agenda
   

   View Slide

3. • Describe Habitat
   • Run an application with Habitat
   • Join the Habitat Community
   • Share your experience
   Goals for today
   

   View Slide

4. View Slide

5. View Slide

6. We Run Infrastructure
   To Run Applications
   That the Business Requires
   

   View Slide

7. View Slide

8. Monolith
   Application
   Application
   Services
   

   View Slide

9. Monolith Microservices
   Application
   Application
   Services
   

   View Slide

10. Monolith Serverless
    Microservices
    Application
    Application
    Services
    

    View Slide

11. Monolith Serverless
    Microservices
    Application
    Application
    Services
    Centralized Highly Distributed
    

    View Slide

12. Monolith Serverless
    Microservices
    Application
    Application
    Services
    Few Large Artifacts Thousands of
    Small Artifacts
    

    View Slide

13. Application Services
    What Developers Need
    Application Runtimes
    

    View Slide

14. Application Services
    What Developers Need Want
    Application Runtimes
    

    View Slide

15. https://twitter.com/codinghorror/status/347070841059692545
    What Developers Need Want Build
    

    View Slide

16. Heterogeneity is the #1 stumbling block
    to velocity.
    Sprawl of management tools is the
    biggest challenge facing modern IT
    teams.
    Current approaches start upside-down:
    with the OS and not the app.
    The core of the problem: application management
    

    View Slide

17. Lifecycle of any software
    Build Deploy Run
    

    View Slide

18. Runtime Concerns
    •  Application supervision
    •  Service discovery
    •  Application configuration
    •  Topology
    •  Updates
    •  Orchestration
    

    View Slide

19. Habitat is used to build, deploy, and
    manage applications…
    

    View Slide

20. …in any environment
    from traditional datacenters
    to containerized microservices
    

    View Slide

21. National Parks
    •  National Parks Application
     
    Java Application running in Tomcat
     
    MongoDB
    

    View Slide

22. View Slide

23. Let’s login!
    u: chef
    p: chef
    

    View Slide

24. Modern Applications
    Source
    Code
    Repo
    

    View Slide

25. Modern Applications
    Source
    Code
    Repo
    Artifact
    

    View Slide

26. Modern Applications
    Source
    Code
    Repo
    Artifact
    Artifact
    Repo
    

    View Slide

27. Modern Applications
    Source
    Code
    Repo
    Artifact
    Bare Metal
    Container
    Cloud Instance
    VM
    Artifact
    Repo
    

    View Slide

28. Modern Applications
    Source
    Code
    Repo
    Artifact
    Bare Metal
    Container
    Cloud Instance
    VM
    Artifact
    Repo
    

    View Slide

29. Modern Applications
    

    View Slide

30. National Parks
    •  National Parks Application
     
    Java Application running in Tomcat
     
    MongoDB
    Start the Supervisor first
    

    View Slide

31. Installs automatically
    Keeps things running
    Supervisor runs services
    

    View Slide

32. $
    Start the Supervisor
    ∵ Missing package for core/hab-sup/0.26.1
    » Installing core/hab-sup/0.26.1
    ↓ Downloading core/hab-sup/0.26.1/20170715031552
    2.18 MB / 2.18 MB - [=======================] 100.00 % 70.60 MB/s
    ☛ Verifying core/hab-sup/0.26.1/20170715031552
    ...
    ★ Install of core/hab-launcher/4435/20170715015238 complete with 1 new packages installed.
    hab-sup(MR): Supervisor Member-ID 0d15afc1907e42afb3c09bcb155e8129
    hab-sup(MR): Starting gossip-listener on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    sudo hab sup run
    

    View Slide

33. $
    Start Java Application
    sudo hab sup load learn/national-parks
    hab-sup(PK): learn/national-parks not found in local package cache, installing
    from https://willem.habitat.sh/v1/depot
    » Installing learn/national-parks
    ↓ Downloading learn/national-parks/0.1.5/20170407081316
    5.75 MB / 5.75 MB | [======================] 100.00 % 100.34 MB/s
    ☛ Verifying learn/national-parks/0.1.5/20170407081316
    ...
    

    View Slide

34. $
    Meanwhile, in the supervisor output
    hab-sup(MR): Supervisor Member-ID 0d15afc1907e42afb3c09bcb155e8129
    hab-sup(MR): Starting gossip-listener on 0.0.0.0:9638
    hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
    hab-sup(MR): Starting learn/national-parks
    hab-sup(MR): Unable to start learn/national-parks, hab-sup(SS)[src/manager/
    service/spec.rs:188:23]: Missing required bind(s), database
    

    View Slide

35. Pitfall!
    Missing required bind(s), database
    

    View Slide

36. Load MongoDB Service
    The Supervisor is designed to supervise one or more services concurrently
    

    View Slide

37. $
    Load MongoDB into the Supervisor
    sudo hab sup load learn/mongodb
    hab-sup(MN): The learn/mongodb service was successfully loaded
    

    View Slide

38. $
    Meanwhile, in the supervisor output
    hab-sup(MR): Starting learn/mongodb
    mongodb.default(SR): Hooks recompiled
    default(CF): Updated mongod.conf
    f9886a510442a32e91d38a79b0baad2a4bb589ba36fed74e2ea3493d517d8bd5
    default(CF): Updated mongos.conf
    8c3e4cd2f3cdd7cd922ce3b529f93672f3af8274e5f6d462a5f8824c25bbbe18
    mongodb.default(SR): Configuration recompiled
    mongodb.default(SR): Initializing
    mongodb.default(SV): Starting service as user=hab, group=hab
    mongodb.default(O): note: noprealloc may hurt performance in many applications
    

    View Slide

39. This raises a question
    •  Where are these packages coming from?
    

    View Slide

40. Modern Applications
    Source
    Code
    Repo
    Artifact
    Bare Metal
    Container
    Cloud Instance
    VM
    Artifact
    Repo
    

    View Slide

41. Central location
    Publicly hosted
    Lightweight views
    Stored in a depot
    

    View Slide

42. https://app.habitat.sh/#/pkgs/learn
    

    View Slide

43. $
    Unload Java Application
    sudo hab sup unload learn/national-parks
    

    View Slide

44. $
    Reload Java Application
    sudo hab sup load learn/national-parks --bind database:mongodb.default
    hab-sup(MR): Starting learn/national-parks
    ...
    national-parks.default(SR): Initializing
    national-parks.default hook[init]:(HK): Seeding Mongo Collection
    ...
    national-parks.default hook[init]:(HK): 2017-07-25T21:16:09.524+0000 Failed:
    error connecting to db server: no reachable servers
    national-parks.default hook[init]:(HK): 2017-07-25T21:16:09.524+0000 imported 0
    documents
    national-parks.default(HK): Initialization failed! 'init' exited with status
    code 1
    

    View Slide

45. Pitfall!
    Cannot connect to the database
    Probably a configuration issue with the database
    

    View Slide

46. What are the tunables?
    [mongod.net]
    bind_ip = "127.0.0.1"
    ipv6 = false
    ...
    [mongod.security]
    authorization = "disabled"
    cluster_auth_mode = "keyFile"
    javascript_enabled = true
    key_file = ""
    ...
    $ sudo hab sup config learn/mongodb
    

    View Slide

47. Create a configuration file
    $ sudo hab sup config learn/mongodb > mongodb.toml
    

    View Slide

48. [mongod.net]
    bind_ip = "127.0.0.1"
    bind_ip = "0.0.0.0"
    ipv6 = false
    ...
    [mongod.security]
    authorization = "disabled"
    cluster_auth_mode = "keyFile"
    cluster_auth_mode = ""
    javascript_enabled = true
    ...
    Edit the mongodb.toml
    -
    +
    -
    +
    

    View Slide

49. Never do this at home!
    •  We are about disable some MongoDB security
    

    View Slide

50. Manages configuration
    Installs automatically
    Keeps things running
    Supervisor runs services
    

    View Slide

51. $
    Change the configuration for MongoDB
    sudo hab config apply --peer 127.0.0.1 mongodb.default \
    1 mongodb.toml
    

    View Slide

52. Applying Configuration
    sudo hab config apply --peer 127.0.0.1 mongodb.default 1 mongodb.toml
    •  peer - hostname or IP address of a running supervisor
    •  mongodb.default - the service group
    •  1 - the incarnation of this configuration
    •  mongodb.toml - the file with the new configuration
    

    View Slide

53. $
    Change the configuration for MongoDB
    ∵ Missing package for core/hab-butterfly/0.26.1
    » Installing core/hab-butterfly/0.26.1
    ↓ Downloading core/hab-butterfly/0.26.1/20170715032942
    1.14 MB / 1.14 MB / [=======================] 100.00 % 65.20 MB/s
    ☛ Verifying core/hab-butterfly/0.26.1/20170715032942
    ...
    » Applying configuration for mongodb.default incarnation 1
    Ω Creating service configuration
    ✓ Verified this configuration is valid TOML
    ↑ Applying to peer 127.0.0.1:9638
    ★ Applied configuration
    sudo hab config apply --peer 127.0.0.1 mongodb.default \
    1 mongodb.toml
    

    View Slide

54. $
    Meanwhile, in the supervisor output
    mongodb.default(SR): Hooks recompiled
    default(CF): Updated mongod.conf
    12a29a9de79166f30aa87640bc4ba4a88b6dc4d4641cf0d6ed00e179229d576f
    mongodb.default(SR): Configuration recompiled
    

    View Slide

55. $
    Meanwhile, in the supervisor output
    national-parks.default(SR): Hooks recompiled
    ...
    Deployment of web application directory /hab/pkgs/core/
    tomcat8/8.5.9/20170215223900/tc/webapps/ROOT has finished in 30 ms
    national-parks.default(O): 25-Jul-2017 21:21:13.399 INFO [main]
    org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [http-
    nio-8080]
    national-parks.default(O): 25-Jul-2017 21:21:13.404 INFO [main]
    org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [ajp-nio-8009]
    national-parks.default(O): 25-Jul-2017 21:21:13.405 INFO [main]
    org.apache.catalina.startup.Catalina.start Server startup in 6850 ms
    

    View Slide

56. Supervisors provide a REST API
    External Actors
    
    Health and
    Status
    
    Supervisor
    Debugging
    

    View Slide

57. Supervisors self-organize
    Service
    Discovery
    
    Service Groups
    

    View Slide

58. http://YOUR_IP:8080/national-parks
    

    View Slide

59. •  http://YOUR_IP:9631/services
    •  http://YOUR_IP:9631/services/mongodb/default/config
    •  http://YOUR_IP:9631/services/national-parks/default/config
    REST API
    

    View Slide

60. •  http://YOUR_IP:9631/services/mongodb/default/config
    REST API
    

    View Slide

61. •  http://YOUR_IP:9631/services/mongodb/default/config
    REST API
    With JSONView
    Chrome Plugin
    

    View Slide

62. Runtime Concerns
    •  Application supervision
    •  Service discovery
    •  Application configuration
    •  Topology
    •  Updates
    •  Orchestration
    

    View Slide

63. It’s all open source
    
    Apache License
    

    View Slide

64. View Slide

65. Unload the services
    hab-sup(SV): national-parks.default - Shutdown method: Graceful Termination
    hab-sup(SV): national-parks.default - process 2457 died with exit code 143
    hab-sup(SV): national-parks.default - Service exited
    $ sudo hab sup unload learn/national-parks
    

    View Slide

66. Unload the services
    hab-sup(SV): mongodb.default - Shutdown method: Graceful Termination
    hab-sup(SV): mongodb.default - process 2572 died with exit code 0
    hab-sup(SV): mongodb.default - Service exited
    $ sudo hab sup unload learn/mongodb
    

    View Slide

67. What’s Next?
    •  Use a provided Habitat development environment, or install Habitat locally
    •  Join the Habitat Slack Team - http://slack.habitat.sh/
    •  Work through the tutorial at https://www.habitat.sh/tutorials/
    •  Explore Habitat packages on the depot - https://app.habitat.sh/
    •  Explore the Habitat projects - https://github.com/habitat-sh
    •  Read Habitat Blog posts - https://www.habitat.sh/blog/
    

    View Slide

68. View Slide