VMware and OpenNMS in Real World

Talk about OpenNMS in VMware environments given at OUCE 2014 at University in Southampton.

Talk: http://cfp.opennms.eu/en/ouce2014/public/events/19

Transcript

1. and Real World™ experience
2. Why is it hard Introdocution Real World™ Roadmap
3. What is it doing for you? • Synchronize vCenter inventory in OpenNMS • Control workflow with filter • Longterm performance data collection • Service assurance for VMs and ESX health status • Visualize VMware topology Introduction
4. A little bit more detailed … • Authentication is handled by the vCenter • Detection of the management interface for
 Host-System • VMware related information, e.g. VMware topology, is
 stored in asset fields Introduction
5. Introduction vCenter Virtual Machine OpenNMS
6. Introduction vCenter Virtual Machine OpenNMS https://vcenter-ip:8443/sdk 3rd party ViJava VMware internal
7. Introduction vCenter Virtual Machine OpenNMS VMware-ManagedEntity [VmwareCollector]! KPI’s for VM from vCenter ! VMware-ManagedEntity [VmwareMonitor]! Test: PowerState
8. Introduction Node: Virtual Machine IP-Interface VMware-VirtualMachine VMware-ManagedEntity
9. Introduction Node: Virtual Machine IP-Interface VMware-VirtualMachine VMware-ManagedEntity VMware Tools required!
10. Introduction Node: Virtual Machine IP-Interface VMware-VirtualMachine VMware-ManagedEntity Pollerd Collectd
11. Introduction Node: Virtual Machine IP-Interface VMware-VirtualMachine VMware-ManagedEntity Pollerd Collectd VMware-ManagedEntity [VmwareMonitor]! Test: PowerState VMware-ManagedEntity [VmwareCollector]! KPI’s for VM from vCenter
12. Virtual Machine Introduction vCenter Host System OpenNMS
13. Introduction vCenter Host System OpenNMS Virtual Machine vCenter - Management Network vCenter - OpenNMS needs access directly to ESX Host
14. Introduction vCenter Host System OpenNMS Virtual Machine Pollerd: VMwareMonitor! Collectd: VmwareCollector
15. Introduction vCenter Host System OpenNMS Virtual Machine getAuthenticationTicket()
16. Introduction vCenter Host System OpenNMS Virtual Machine getAuthenticationTicket() Using AuthenticationTicket Pollerd: VMwareCimMonitor! Collectd: VmwareCimCollector Pollerd: VMwareMonitor! Collectd: VmwareCollector
17. Introduction Node: Virtual Machine IP-Interface VMware-VirtualMachine VMware-ManagedEntity Pollerd Collectd Node: Host System IP-Interface VMware-HostSystem VMwareCim-HostSystem VMware-ManagedEntity
18. Introduction Node: Virtual Machine IP-Interface VMware-VirtualMachine VMware-ManagedEntity Pollerd Collectd Node: Host System IP-Interface VMware-HostSystem VMwareCim-HostSystem VMware-ManagedEntity Pollerd Collectd
19. Introduction Node: Virtual Machine IP-Interface VMware-VirtualMachine VMware-ManagedEntity Pollerd Collectd Node: Host System IP-Interface VMware-HostSystem VMwareCim-HostSystem VMware-ManagedEntity Pollerd Collectd VMware-HostSystem [VmwareCollector]! KPI’s from vCenter ! VMwareCim-HostSystem
 [VmwareCimCollector]! KPI’s from ESX VMwareCim-HostSystem
 [VmwareCimMonitor]! Test: ESX Health State ! VMware-ManagedEntity [VmwareMonitor]! Test: PowerState
20. Introduction
21. Real World™
22. Real World™ • Detecting services on Host-Systems with multiple
 unreachable interfaces • Authentication with local vCenter user vs. ADS integration
23. Workflow for a Host-System
 during the import Real World™ requisition = null;! for (hostSystem : allHostSystems) {! ! for (ipadd : allExceptLoopback) {! !! detectService(“VMwareCim-HostSystem”);! !}! ! generateTopologyAssetField();! ! addAssetFields(“CPU”,”RAM”);! ! addCategory(“VMware5”);! ! addNodeToRequisition();! ! return requisitiion;! }
24. Workflow for a Host-System
 during the import Real World™ requisition = null;! for (hostSystem : allHostSystems) {! ! for (ipadd : allExceptLoopback) {! !! detectService(“VMwareCim-HostSystem”);! !}! ! generateTopologyAssetField();! ! addAssetFields(“CPU”,”RAM”);! ! addCategory(“VMware5”);! ! addNodeToRequisition();! ! return requisitiion;! }
25. vCenter Host System OpenNMS Virtual Machine Real World™ vCenter Microsoft Active Directory Services getAuthenticationTicket() Authenticate against LDAP
26. vCenter Host System OpenNMS Virtual Machine Real World™ vCenter Microsoft Active Directory Services getAuthenticationTicket() Authenticate against LDAP LDAP vs. local user
27. • Data collection is configured by default to get *ALL* collectable KPI’s • Measurements from sensors needs additional calculation
 Metric called: Base Unit 2 which-> value / 10^2) Real World™
28. http://pubs.vmware.com/vsphere-55/index.jsp#com.vmware.cimsdk.smashref.doc/class_OMC_NumericSensor.html Real World™
29. uint16 BaseUnit = Volts(5)! sint32 UnitModifier = -6! ! sensorReading * 10^-6 => Microvolts Real World™
30. Why is it hard?
31. Version combinations! ! • ViJava Library • vCenter 3/4/5.0/5.1/5.5 • Underlying OS for vCenter • Authentication methods • ESX Host operating system versions • API calls which require additional communication for authentication • Side effects from non-perfect functional ADS setups Why is it hard?
32. What do we have under control?! ! • Implementation of the 3rd party libraries ViJava and Sblime • Algorithms for the import • Algorithms for performance data collection • Workflows to model the synchronization behavior Why is it hard?
33. Roadmap
34. • Calculate sensor readings during data collection
 https://github.com/OpenNMS/opennms.git
 branch: feature-vmware-metrics • Move import to “Inventory Integration Service”
 https://github.com/OpenNMS/opennms-provisioning-integration-server.git
 branch: feature-vmware-source • Improve diagnostic tools like vmwareConDiag
 https://github.com/opennms-forge/vmwareConDiag.git • Improve graphing
 https://github.com/OpenNMS/opennms/tree/vmware-graph-improvement/opennms-base-assembly/src/main/ filtered/etc/snmp-graph.properties.d • Create detector for Host-System service • Bamboo has to build Yum/Debian packages
 http://www.opennms.org/wiki/OpenNMS_and_Vagrant_with_VirtualBox
 release -> “branches/topology-HUD-display” Roadmap
35. $OPENNMS_HOME/bin
 ./vmwarereqtool vmware://user:pass@vcenter Roadmap • Run provisioning run on cli • Using filter like ?importHostOnly=true
36. Roadmap $OPENNMS_HOME/bin
 ./vmwarecimquery vcenter user pass
37. https://github.com/OpenNMS/opennms-provisioning-integration-server.git
 branch: feature-vmware-source VMware Connection Diagnostics Roadmap • It has to build from source • Java JDK + Maven required
 mvn clean install! • java -jar …jar-with-dependencies.jar! • config.properties
38. Roadmap
39. Roadmap
40. • Using just standardized XML model for import • More flexibility manipulating the requisition before import with Groovy scripts • Free provisiond from 3rd party dependencies • Simplify code for JUnit test environment around Provisioned • Allow faster iteration for fixing import issues Move from integrated URL requisition in Provisiond to to “Inventory Integration Service” Roadmap
41. Roadmap OpenNMS Provisiond Pollerd Notifd Eventd Alarmd VMware VMware vCenter - External dependencies in OpenNMS! - Changes require a full release cycle of the whole OpenNMS project! - Hard to write JUnit tests, mocking vCenter with versions is quite impossible
42. Roadmap OpenNMS Provisiond Pollerd Notifd Eventd Alarmd VMware VMware vCenter Start loosing control cause of depending on 3rd party libs
43. Roadmap OpenNMS Integration Layer OpenNMS Provisiond VMware Pollerd Notifd Eventd Alarmd VMware vCenter
44. Roadmap OpenNMS Integration Layer OpenNMS Provisiond VMware Pollerd Notifd Eventd Alarmd VMware vCenter Moving technology specific implementation to smaller independent code base
45. Roadmap OpenNMS Integration Layer OpenNMS Provisiond VMware Pollerd Notifd Eventd Alarmd VMware vCenter - Standardized API! - Provides public integration interface via ReST and XML data model! - Hardening and Unit-Testing is easier - ReST API calls! - XML requisition data model
46. Roadmap OpenNMS Integration Layer OpenNMS Provisiond VMware Pollerd Notifd Eventd Alarmd VMware vCenter - Standardized API! - Provides public integration interface via ReST and XML data model! - Hardening and Unit-Testing is easier - Changes don’t affect core project —> faster iteration! - Less complex code surrounded easier to test! - Fast development cycle - ReST API calls! - XML requisition data model Poor mans adaption of ETL
 Extract, Transform, Load