Introduction to Glu

JAXJUG presentation 10/09/13 on Glu deployment automation platform.

Transcript

1. Introduction to Glu (just a sniff) Gary Hale Jax JUG 10/09/14
2. Agenda • Introduction • Basic Concepts • Demo • Comparison • APIs/Tools
3. Introduction Does this sound familiar? Admin: “Ok the new test environment is ready - you’re good to go…” Developer/QA: “Uhh, I’m getting a 500 error” Admin: “Oops - I forgot something - hold on…” . . . Admin: “Ok - It’s fixed - you’re good to go…” Developer/QA: “Uhh, this data looks all wrong” Admin: “Oops - I misconfigured something - hold on…” . . . Admin: “Ok - It’s fixed - you’re good to go…” Developer/QA: “Uhh, it’s only working some of the time” Admin: “Oops - I missed server X - hold on…” . . . Rinse and Repeat for Two Weeks
4. Introduction Glu ◦ Written by Yan Pujante for LinkedIn ◦ Declarative ◦ Efficient ◦ Secure ◦ Addresses operations needs
5. Introduction History • Started in July 2009 • Initial Prod Rollout March 2010 • Open Sourced in Nov 2010 • Circa 2011, LinkedIn deploys to over 2650 nodes • Also used by Orbitz and Outbrain • V5.3.1 released 10/03/13
6. Introduction Availity’s Legacy System Jenkins Datacenter Deployment Server Application Server Application Server Application Server Application Server . . . Sequential Cron Packages
7. Introduction Problems • Glorified SSH Loop • Low visibility for Operations Staff • Sequential nature makes for long deployments • No way to customize deployment without creating a new package • No way to redeploy selectively • No way to evaluate current state
8. Introduction Glu • Tracks state of the system • Sequential or Parallel deployments (or both) • Increases visibility for Operations staff • Submit deployments by declaring intention rather than procedure • Easy to create different views of the environment for different audiences • Allows arbitrarily selective deployments • Highly customizable • Deployment of anything - not just applications
9. Introduction Availity’s Glu System Jenkins Datacenter Glu Server Application Server Application Server Application Server Application Server . . . Repository Packages REST
10. Basic Concepts • Fabric - Defines a domain of servers/applications (e.g. prod fabric vs test fabric) • Static Model - The intended state of the environment (where, what, and how to deploy) • Live Model - The actual state of the environment • Agent - A “server” where an application can be deployed • Script - The set of instructions for how to stop, start, install and configure an application • Plan - The set of steps required to move the model from one state to another
11. Basic Concepts
12. Basic Concepts Live Model Static Model Glu Engine Delta Plan
13. Basic Concepts Model - Describes where, what and how to deploy { "mountPoint": "/app1", "agent": "agent-1", "initParameters": { "port": 9002, "skeleton": "http://localhost:8080/glu/repository/tgzs/jetty-distribution-7.2.2.v20101205.tar.gz", "war": "http://localhost:8080/glu/repository/wars/org.linkedin.glu.samples.sample-webapp-4.6.1.war" }, "script": "http://localhost:8080/glu/repository/scripts/org.linkedin.glu.script-jetty-4.6.1/JettyGluScript.groovy", "tags": [ "app1", "step002", "tst" ] }
14. Basic Concepts Possible uses of tags • application • OS • frontend/backend • phase of deployment • type of deployment • batch/realtime • line of business • stewardship Tags allow you to slice/dice the model any way you like!
15. Basic Concepts State Engine:
16. Basic Concepts Script - definition for how to move between states
17. Basic Concepts State Machine Codified: You can create your own state machine!
18. Basic Concepts Glu Script:
19. Basic Concepts Script Capabilities: ◦ log - access to the deployment log ◦ shell - access to various OS-level operations (ls, mv, tar, untar, fetch, etc) ◦ shell.env - access to environment variables ◦ stateManager - access to the stateManager ◦ params - access to the init parameters ◦ timers - allows you to configure timers (useful for monitoring)
20. Basic Concepts Script Packaging • Flat File: "script": "http://host:port/x/c/v/MyGluScript.groovy" "script": "file:///x/c/v/MyGluScript.groovy" • Bundled: "script": "class:/com.acme.MyGluScript?cp=http%3A%2F% 2Facme.com%2Fjars%2Fscript.jar&cp=http%3A%2F%2Facme. com%2Fjars%2Fdependency.jar"
21. Basic Concepts Security • Support for LDAP • Roles ◦ User - Read access only ◦ Release - Ability to deploy, start, stop, etc ◦ Admin - Full access to everything ◦ Restricted - No access to anything • All changes are audited You can modify what capabilities are available to what roles.
22. Basic Concepts Glu is not just a deployment application, it is a deployment platform that is highly customizable. • Customization points: ◦ Deployment scripts ◦ State Machine ◦ Look/feel ◦ Security ◦ Dashboards ◦ Saved Filters
23. Demo
24. Comparison Disclaimer I’m not very familiar with other tools. Some of this information may be uninformed, misrepresented and/or incorrect.
25. Comparison First of all: Using any configuration management tool is a win! They are all better than doing it manually. Stop treating servers like pets, and start treating them like livestock. If it’s sick, shoot it! --Jeffrey Hulten
26. Comparison Second of all: This brief (and possibly irresponsible) comparison is based on the open source versions of these products. Some of them have commercial versions which have support, better documentation, etc, etc.
27. Comparison Model-Based Glu Puppet Salt Recipe-Based Chef Ansible
28. Comparison Language • Glu - Groovy/Json • Puppet - Ruby • Chef - Ruby (sort of) • Salt - Python/Yaml • Ansible - Yaml
29. Comparison Complexity of Installation: LOW HIGH Ansible Salt Puppet Chef Glu
30. Comparison Ability to Customize: HIGH LOW Glu Salt Chef Puppet Ansible
31. Comparison Amount/Quality of Documentation: HIGH LOW Salt Glu Chef * Puppet Ansible
32. Comparison Where Glu shines • Dynamic applications that change frequently • Multi-node orchestration Many of the other tools are focused on the steps to install/configure a node (users, groups, packages, etc). Stuff that doesn’t change frequently. Glu is more focused on maintaining a system of nodes and managing frequent releases of multiple components.
33. Comparison Operating System Infrastructure Applications OS-specific bootstrapping tools Other CM Tools Glu Rate of Change
34. APIs/Tools REST APIs: • Agents • Plans • Deployments • Model • Commands
35. APIs/Tools Json Groovy DSL • Allows programmatic definition of the model • Loads just like a json model and is expanded at load time [ [instance: 'i001', webapps: ['cp1', 'cp2'], port: 9000, cluster: 'c1', product: 'product1', tags: ['frontend', 'webapp']], [instance: 'i002', webapps: ['cp1'], port: 9001, cluster: 'c1', product: 'product1', tags: ['frontend', 'webapp']], [instance: 'i003', webapps: ['cp4'], port: 9002, cluster: 'c2', product: 'product1', tags: ['backend', 'webapp']], ].each { m -> entries << [ agent: agent, mountPoint: "/sample/${m.instance}", script: script, initParameters: [ skeleton: skeleton, port: m.port, … ] ] }
36. APIs/Tools gradle-glu-plugin • Manage glu from gradle • Combine glu configuration/deployment with build scripts • Allows programmatic generation of the model • Allows templating of “applications” for generating hundreds of nearly identical entries • Allows one to break up a large fabric into smaller scripts that generate different parts of the model • Allows programmatic submission of deployments (start, stop, deploy, bounce, etc) • Allows management of multiple fabrics • Documentation: https://github.com/ghale/gradle-glu-plugin/wiki
37. References • Glu Documentation: http://pongasoft.github. io/glu/docs/latest/html/contents.html • Glu Source Code: https://github.com/pongasoft/glu • Monitoring with Glu: http://www.pongasoft. com/blog/yan/glu/2011/03/18/building-monitoring- solution-with-glu/ • Case Study at Outbrain: http://prettyprint.me/prettyprint. me/2011/01/24/continuous-deployment-at- outbrain/index.html
38. Questions ?