MCollective as an Integration Layer

At PuppetCamp in April 2014, @jtopper spoke about using MCollective as an integration layer, covering our two projects ZCollective (http://www.scalefactory.com/resources/github/zcollective) and sf-deploy (https://github.com/scalefactory/sf-deploy).

The talk was recorded, and this recording can be seen at https://www.youtube.com/watch?v=usemi8WdGHo

Transcript

1. MCollective as an integration layer Jon Topper The Scale Factory
2. MCollective a framework to build server orchestration or parallel job execution systems. “ ”
3. MCollective not simply a fancy SSH “for loop” “ ”
4. MCollective Servers Middleware Client
5. MCollective Servers Middleware Client
6. Finding Servers
7. Check Service Status
8. Check Package Versions
9. Still looks a bit like a fancy SSH “for loop”
10. a framework to build server orchestration or parallel job execution systems. “ ”
12. Zabbix • Distributed monitoring system • Fetches time series data - can alert on anomaly • Uses MySQL • Multiple data sources: • Zabbix agent / sender • SNMP poll / traps • JMX
13. Zabbix
14. Zabbix
15. ZCollective zcollective —zabbix-api-url \! http://localhost/api_jsonrpc.php
16. ZCollective include MCollective::RPC! ! mc = rpcclient("rpcutil", :debug => true)! ! mc.discover.sort.each do |host|! ! inventory = mc.custom_request( ! "inventory", {}, host,! { "identity" => host }! ).first! ! do_some_zabbix_magic(inventory)! ! end
17. ZCollective def do_some_zabbix_magic(inventory)! ! # If the host isn’t added to Zabbix! # Add the host using the API! # For each Puppet class on this host! # If there’s a Zabbix template by this name! # Assign that template to the host.! # end! # end! # end! ! # If the host is already in Zabbix! # Make sure all templates correctly assigned! # end ! ! end
18. ZCollective
19. ZCollective
20. ZCollective https://github.com/scalefactory/zcollective
21. Traditional Deployment
22. sf-deploy Latest: Deployed: aaa1 aaa1
23. sf-deploy Latest: Deployed: aaa1 aaa1
24. sf-deploy Latest: Deployed: aaa1 aaa2
25. sf-deploy Latest: Deployed: aaa2 aaa2
26. Code class MCollective::Application::Sfdeploy < MCollective::Application! ! def main! ! action = configuration[:action]! ! mc = rpcclient("sfdeploy")! ! printrpc mc.send(! action,! :application => configuration[:application],! :branch => configuration[:branch],! :tag => configuration[:tag],! :groups => configuration[:groups],! :options => options! )! ! printrpcstats! end
27. Code module MCollective! module Agent! class Sfdeploy < RPC::Agent! ! all_actions.each do |act|! action act do! ! sf_deploy = config.pluginconf["sfdeploy.binary"] || "sf-deploy"! deploy_user = config.pluginconf["sfdeploy.user"]|| nil! ! command = "#{sf_deploy} -a #{request[:application]} -v "! command << "-b #{request[:branch]} " if request[:branch]! command << "-t #{request[:tag]} " if request[:tag]! command << "-g #{request[:groups]} " if request[:groups]! command << act! ! if deploy_user! to_run = "su #{deploy_user} -c \"#{command}\""! else! to_run = command! end
28. ZCollective https://github.com/scalefactory/sf-deploy
29. Resources http://docs.puppetlabs.com/mcollective/
30. http://www.scalefactory.com/ jon@scalefactory.com @jtopper scalefactory