Context-Based Analytics - Establishing Explicit Links between Runtime Traces and Source Code

Joint work with IBM Research on improving cloud analytics presented at the International Conference on Software Engineering (ICSE'17) in Buenos Aires, Argentina

Transcript

1. Context-Based Analytics - Establishing Explicit 
 Links between Runtime Traces and Source Code ICSE’17, Software Engineering in Practice, Buenos Aires, Argentina Source: Jesus Alexander Reyes Sánchez https://flic.kr/p/6T8rNY
2. 2 Distributed Architectures Multiple Technologies Infrastructure at Scale Scattered Information Figure taken from Chen et al. A provider-side viewof web search response time. 2013. Modern Software Systems 2
3. State-of-Practice Log Management, Dashboard 3
4. Solving Problems in Modern Software Systems Metrics Dashboard ....... [26/06/2015:21205.0], responseTime, "CustomerS", 204 [26/06/2015:21215.0], responseTime, "CustomerS", 169 [26/06/2015:21216.0], connection, "VM2", 0.73 [26/06/2015:21216.0], connection, "VM1", 0.69 [26/06/2015:21216.1], vmBilled, “CSVM2”, 0.35 [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ Hint#1 Distributed Logs & Configuration ....... [26/06/2015:21205.0], environment, “DB”, test [26/06/2015:21215.0], environment, “USR”, admin [26/06/2015:21219.4], ids, "ids", [1,16,32,189,216] ........ Hint#2 Version Control History Hint#3 Search Code Files Hint#4 Hint#5 4
5. Developers build a mental context-model
 
 They establish implicit links between fragments 5
6. Idea: Establish Explicit Links between 
 Runtime Traces and Source Code [26/06/2015:21216.0], connection, "VM2", 0.73 [26/06/2015:21216.0], connection, "VM1", 0.69 Connection [26/06/2015:21205.0], environment, “DB”, test [26/06/2015:21215.0], environment, “USR”,admin Environment Relevant Code Fragment 6
7. Conceptual Framework 7 Meta-Model Application Model Initial Graph Construction Context Expansion Offline Online
8. Conceptual Framework
 Meta-Model Time Series Entity Application State Set Entity Unit Entity Program Fragment Program File Configuration File Script File n 8
9. Conceptual Framework
 Application-Model Network VM Instance Memory CPU Environment Variables Heat Template Bash Script Java File Java Method Access Rate Swap VCPU Disk IO Request Packetloss Adaptation Bitrate Jitter Frames Exception .ini-config Network Network Telecommunications Application Application Model Process Memory CPU Environment Variables Dockerfile Bash Script Python File Python Method API Endpoint Response Time Status Code Online Processing EV ‘DB’ Process ‘python’ Graph Construction Method ‘get_papers’ Bash start_db.sh Context Expansion Feature Memory {74,78, 75…} Time Series Entity Runtime Entity Set Entity Unit Entity Code Fragment Program File n Cloud Middleware Service 9
10. Conceptual Framework Online Processing Variables Bash Script File Python Method API Endpoint Response Time Status Code Online Processing EV ‘DB’ Process ‘python’ Graph Construction Method ‘get_papers’ Bash start_db.sh Context Expansion Feature Visualization Memory {74,78, 75…} Context Graph Starting Nodes Context Inference on Node Similarity Measure 10
11. Context-Based Analytics Prototype 11
12. Industrial Case Study Application & Tasks CASE STUDY APPLICATION active-deploy in IBM Bluemix 8 typical problem diagnosis tasks from issue tracker 12
13. Industrial Case Study Effort Metrics # Steps: Number of steps taken - Data Query - Creating Visualizations/Plots - Software History Analysis # Traces: Number of traces inspected - Log statements - Graphical Representations - Datastore Entries 13
14. Industrial Case Study Baseline vs Company Baseline (Kibana) CBA Prototype 14
15. Industrial Case Study Preliminary Results Effort reduction of CBA compared to a company baseline, Kibana 48% reduction in steps taken to solve the tasks 40% reduction in traces inspected to solve the tasks 15
16. Discussion & Future Work Dynamic Context Models [Adapting application models on-the-fly through queries] Learning context inference models [Instrumenting queries and interaction with log data to learn models] Integrating Business Metrics [Establishing explicit links between business metrics and source code] 16
17. Jürgen Cito <cito@ifi.uzh.ch> Context-Based Analytics establishes explicit links between runtime traces and source code to form a context-graph that developers can navigate to diagnose problems originating at runtime. Distributed Runtime Information Context-Based Analytics: Conceptual Framework & Prototype ON ix Case Study: active-deploy 48% reduction in steps taken 40% reduction in traces inspected Reduced effort in case study Meta-Model Application Model Process Memory CPU Environment Variables Dockerfile Bash Script Python File Python Method API Endpoint Response Time Status Code Online Processing EV ‘DB’ Process ‘python’ Graph Construction Method ‘get_papers’ Bash start_db.sh Context Expansion Feature Visualization Memory {74,78, 75…} Time Series Entity Runtime Entity Set Entity Unit Entity Code Fragment Program File Configuration File Script File n ICSE’17, Software Engineering in Practice, Buenos Aires, Argentina @citostyle