Operational Intelligence

Enterprise application development, testing, staging and ultimately pushing out to production involves number of operational activities. Moreover operational activities will also expand to the runtimes of each environment. Today the strength of a devops engineers is measured by the weight of his tool belt. That showcases his/her ability to manage the operational process and the ability to drill down the root causes and optimization possibilities at the runtime. Operational duties also expands to system maintenance, system health check and guaranteed availability. The ability to know the system behavior is the key to implement a proper operational process. During the talk Nuwan will be elaborating operational intelligence with regard to above processors and planning to discuss about the right level of information at each platform environment and runtime.

Transcript

1. Operational Intelligence Nuwan Bandara Solutions Architect - WSO2 @nuwanbando
2. Data, Information & Knowledge http://spreadgoodpractice.blogspot.com/2010/12/model-4-data-information-knowledge.html
3. Producers of data movement temp power institute people storage servers applications POS router camera vibration
4. Produces of information
5. Knowledge o Deriving meaning from information o Co-relation o Historical analysis o Mathematical analysis o Model driven analysis o Graphs & charts based comparison
6. Wisdom o Act upon knowledge o Business decisions (when to invest / when to withdraw) o When to compete o Tactics and strategies o Automated decision making o Triggering alerts / email / SMS o Device control (Temperature control / traffic control etc.) o Failover decision making o Elastic scaling decision making
7. So what is intelligence o Collective correlated knowledge which enables informed decision (wisdom) making https://dmaiph.wordpress.com/2015/01/28/time-consuming-tasks-your-small-business-should- outsource/
8. Looking back at data collection o Collecting right (?) data is the key o Collection method is important o Storing the un-sorted, need a proper thought process
9. Operational data o The type of data that is core to the system of operation resource usage of the cloud container useage app usage App Cloud Echosystem raw material product manufature material inventory demand statistics product inventory Manufacturing Plant
10. Conversion of data to information o Data correlation o Data graphs and trees o Application of dimensions o Summarization o Data dictionary reference
11. When should we posses information o At all stages of the operation o i.e.: An application pushed through development, testing, staging and production lifecycle o At each stage the knowledge we gather is different, hence information has a different shape at each step
12. An example o Deploying an elastically scaling middleware platform for application development & integration app server bus api gateway elastic nodes / containers data nodes
13. The application o Class / package dependencies & associations o Exception / stack trace logs o Memory / load statistics o Database connection statistics o JMX channel data on garbage collection / deadlocks / heap usage etc. Package (p0) Class (cp0) Package (p1) Class (cp1) Package (p1) Class (cp1) data bridge / event receiver exception / error logs memory / load stats JMX application
14. Application server & server container o Server logs o JMX channel data of the server o Server health check o Memory / processor / network usage o Response times o Container health o Container load o Container storage space stats application (a1) service (s2) service (s1) server logs server JMX container stats server response time container app server data bridge / event receiver
15. Integration platform o Mediation statistics o Workflow statistics o Data services query history / audit o MQ statistics o Container statistics o Integration platform latency o Request-response log containers c1 c2 c3 c4 cn bps log esb log kafka log data bridge / event receiver
16. API platform o Load balancer logs – container routing / fail-over stats o API subscriber statistics o API usage statistics o API lifecycle statistics containers c1 c2 c3 cn load balancer data bridge / event receiver api gateway api store api publisher API subscriber stats LB logs container stats API usage stats API lifecycle stats
17. Information factory data bridge / event receiver realtime near realtime / batch realtime info feed summarized / co-related feed
18. Knowledge factory data bridge / event receiver realtime near realtime / batch realtime info feed summarized / co-related feed Machine learning lambda loop back loop back intelegence ML feed back ML feed back
19. Decision enablers data bridge / event receiver realtime near realtime / batch realtime info feed summarized / co-related feed Machine learning lambda loop back loop back intelegence ML feed back ML feed back integration engine APIs dashboards topics
20. Operational intelligence reference model application (a1) service (s2) service (s1) server logs server JMX container stats server response time app server data bridge / event receiver containers c1 c2 c3 c4 cn bps log esb log kafka log data bridge / event receiver containers c1 c2 c3 cn load balancer api gateway api store api publisher API subscriber stats LB logs container stats API usage stats API lifecycle stats realtime near realtime / batch realtime info feed summarized / co-related feed Machine learning lambda loop back loop back intelegence ML feed back ML feed back integration engine APIs dashboards topics
21. Knowledge is power "Technology is so much fun but we can drown in our technology. The fog of information can drive out knowledge." -- Daniel J. Boorstin
22. //Questions ? Thank You