Dynamic Dependence Summaries

Modern applications make heavy use of third-party libraries and components, which poses new challenges for efficient dynamic dataflow analysis. To perform such analyses, transitive dependent components at all layers of the call stack must be monitored and analyzed, and as such may be prohibitively expensive for systems with large libraries and components. As an approach to address such expenses, we record, summarize, and reuse dynamic dataflows between inputs and outputs of components, based on dynamic control and data traces. These summarized dataflows are computed at a fine-grained instruction level; the result of which, we call “dynamic dependence summaries.” Although static summaries have been proposed, to the best of our knowledge, this work presents the first technique for dynamic dependence summaries. The benefits to efficiency of such summarization may be afforded with losses of accuracy. As such, we evaluate the degree of accuracy loss and the degree of efficiency gain when using dynamic dependence summaries of library methods. On five large programs from the DaCapo benchmark (for which no existing whole-program dynamic dependence analyses have been shown to scale) and 21 versions of NANOXML, the summarized dependence analysis provided 90% accuracy and a speedup of 100% (i.e., ×2), on average, when compared to traditional exhaustive dynamic dependence analysis.

Transcript

1. Improving Efficiency of Dynamic Analysis with Dynamic Dependence Summaries Vijay Krishna Palepu Guoqing Xu James A. Jones University of California, Irvine, USA 28th IEEE/ACM International Conference on Automated Software Engineering, 2013 Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (1/24) ASE 2013
2. Introduction Basic Problem & Background The Hut and The Mountain Image Credit:“Eagle’s Eye KHMR”, Doug Zwick@Flickr Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (2/24) ASE 2013
3. Introduction Basic Problem & Background The Hut and The Mountain Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (3/24) ASE 2013
4. Introduction Basic Problem & Background The Hut and The Mountain Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (4/24) ASE 2013
5. Introduction Basic Problem & Background The Hut and The Mountain Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (5/24) ASE 2013
6. Introduction Basic Problem & Background Method Summaries Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (6/24) ASE 2013
7. Introduction Basic Problem & Background Method Summaries Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (6/24) ASE 2013
8. Introduction Basic Problem & Background Method Summaries Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (6/24) ASE 2013
9. Introduction Basic Problem & Background Method Summaries Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (6/24) ASE 2013
10. Introduction Basic Problem & Background Example. void main() { IntList k = new IntList(); int num = 1; k.add(num); ... ... ... } Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (7/24) ASE 2013
11. Introduction Basic Problem & Background Example. void main() { IntList k = new IntList(); int num = 1; k.add(num); ... ... ... } void add(int i) { int t = this.size; int[] a = this.arr; a[t] = i; t = t + 1; this.size = t; } Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (7/24) ASE 2013
12. Introduction Basic Problem & Background Example. void main() { IntList k = new IntList(); int num = 1; k.add(num); ... ... ... } void main() { IntList k = new IntList(); int num = 1; ... ... ... } k.arr[k.size] = num; k.size = k.size + 1; Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (7/24) ASE 2013
13. Introduction Background Background. • Sharir and Pnueli. Two Approaches to Interprocedural Data Flow Analysis. Program Flow Analysis: Theory and Applications, 1981. • Horwitz, Reps and Binkley. Interprocedural Slicing using Dependence Graphs. TOPLAS, 1990. • Rountev, Sharp and Xu. IDE Dataflow Analysis in the Presence of Large Object-Oriented Libraries. CC, 2008. • Yorsh, Yahav and Chandra. Generating Precise and Concise Procedure Summaries. POPL, 2008. • Xu, Rountev and Sridharan. Scaling CFL-reachability-based Points-to Analysis using Context-sensitive Must-not-alias Analysis. ECOOP , 2009. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (8/24) ASE 2013
14. Our Approach Overview Typical Dynamic Analysis Instrument Program & Libraries. Monitor Execution (Program & Libraries). After Some Waiting Extract Dataflow (Program & Libraries). Client Analysis. e.g., Find Bug! Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (9/24) ASE 2013
15. Our Approach Overview Approach Overview. Instrument Program & Libraries. Monitor Execution (Program & Libraries). After Some Waiting Extract Dataflow (Program & Libraries). Client Analysis. e.g., Find Bug! Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (10/24) ASE 2013
16. Our Approach Overview Approach Overview. Instrument Program & Libraries. Monitor Execution (Program & Libraries). After Some Waiting Extract Dataflow (Program & Libraries). Client Analysis. e.g., Find Bug! Library Method’s Dataflows. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (10/24) ASE 2013
17. Our Approach Overview Approach Overview. Instrument Program & Libraries. Monitor Execution (Program & Libraries). After Some Waiting Extract Dataflow (Program & Libraries). Client Analysis. e.g., Find Bug! Library Method’s Dataflows. Recognize Method Inputs/Outputs. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (10/24) ASE 2013
18. Our Approach Overview Approach Overview. Instrument Program & Libraries. Monitor Execution (Program & Libraries). After Some Waiting Extract Dataflow (Program & Libraries). Client Analysis. e.g., Find Bug! Library Method’s Dataflows. Recognize Method Inputs/Outputs. Summarize Dataflows. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (10/24) ASE 2013
19. Our Approach Overview Approach Overview. Instrument Program & Libraries. Monitor Execution (Program & Libraries). After Some Waiting Extract Dataflow (Program & Libraries). Client Analysis. e.g., Find Bug! Library Method’s Dataflows. Recognize Method Inputs/Outputs. Summarize Dataflows. Abstract & Store. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (10/24) ASE 2013
20. Our Approach Overview Approach Overview. Instrument Program. Monitor Execution (Program). ' Reuse Stored Library Method Summaries. Extract Dataflow (Program & Libraries). Client Analysis. e.g., Find Bug! Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (11/24) ASE 2013
21. Our Approach Analysis & Implementation Dynamic Dependence Summaries. • Summarize • Abstract • Reuse Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (12/24) ASE 2013
22. Our Approach Analysis & Implementation Summarize. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (13/24) ASE 2013
23. Our Approach Analysis & Implementation Summarize. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (13/24) ASE 2013
24. Our Approach Analysis & Implementation Summarize. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (13/24) ASE 2013
25. Our Approach Analysis & Implementation Summarize. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (13/24) ASE 2013
26. Our Approach Analysis & Implementation Summarize. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (13/24) ASE 2013
27. Our Approach Analysis & Implementation Summarize. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (13/24) ASE 2013
28. Our Approach Analysis & Implementation Summarize. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (13/24) ASE 2013
29. Our Approach Analysis & Implementation Abstract. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (14/24) ASE 2013
30. Our Approach Analysis & Implementation Reuse. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (15/24) ASE 2013
31. Our Approach Analysis & Implementation Reuse. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (15/24) ASE 2013 Key Technical Challenges Addressed. • Summary Abstraction and Reuse. • Precise modeling of Array element accesses. • Accounting for Varying Method Behavior due to polymorphism. • Handling object-graph mismatch. • Object sensitivity.
32. Our Approach Analysis & Implementation Implementation. Key Parts Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (16/24) ASE 2013
33. Our Approach Analysis & Implementation Implementation. Key Parts • Includes: Java Bytecode Instrumenter uses:Java; ASM (asm.org) Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (16/24) ASE 2013
34. Our Approach Analysis & Implementation Implementation. Key Parts • Includes: Java Bytecode Instrumenter uses:Java; ASM (asm.org) Trace Analyzer (Dataflow or Dependencies) uses:Java Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (16/24) ASE 2013
35. Our Approach Analysis & Implementation Implementation. Key Parts • Includes: Java Bytecode Instrumenter uses:Java; ASM (asm.org) Trace Analyzer (Dataflow or Dependencies) uses:Java Dependence Summarizer uses:Java Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (16/24) ASE 2013
36. Our Approach Analysis & Implementation Implementation. Key Parts • Includes: Java Bytecode Instrumenter uses:Java; ASM (asm.org) Trace Analyzer (Dataflow or Dependencies) uses:Java Dependence Summarizer uses:Java • Built using: Java; ASM (asm.org) Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (16/24) ASE 2013
37. Empirical Study Experiment One Experiment One: Performance. RQ1 How does the reuse of dynamic dependence summaries affect the costs of dynamic analysis? Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (17/24) ASE 2013
38. Empirical Study Experiment One Experiment One: Performance. RQ1 How does the reuse of dynamic dependence summaries affect the costs of dynamic analysis? Metrics Execution Trace Size. Execution Running Time. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (17/24) ASE 2013
39. Empirical Study Experiment One Experiment One: Performance. RQ1 How does the reuse of dynamic dependence summaries affect the costs of dynamic analysis? Metrics Execution Trace Size. Execution Running Time. Treatments Exhaustive. Summary-based. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (17/24) ASE 2013
40. Empirical Study Experiment One Experiment One: Performance. RQ1 How does the reuse of dynamic dependence summaries affect the costs of dynamic analysis? Metrics Execution Trace Size. Execution Running Time. Treatments Exhaustive. Summary-based. Client Subjects ANTLR (35KLOCs) BLOAT (41KLOCS) FOP (102KLOCs) JYTHON (245KLOCs) PMD (60KLOCs) Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (17/24) ASE 2013
41. Empirical Study Experiment One Experiment One: Performance. RQ1 How does the reuse of dynamic dependence summaries affect the costs of dynamic analysis? Metrics Execution Trace Size. Execution Running Time. Treatments Exhaustive. Summary-based. Client Subjects ANTLR (35KLOCs) BLOAT (41KLOCS) FOP (102KLOCs) JYTHON (245KLOCs) PMD (60KLOCs) Library Subject JAVA DEVELOPMENT KIT (rt.jar) Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (17/24) ASE 2013
42. Empirical Study Experiment One Results: Runtime (RQ1). 0.00 500.00 1,000.00 1,500.00 2,000.00 2,500.00 3,000.00 Run Time (s) summary exhaustive • 1.5× — 3.6× speedup in execution runtimes. • Exhaustive: 112× runtime overhead • Summary Times: 43× runtime overhead Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (18/24) ASE 2013
43. Empirical Study Experiment One Results: Trace Size (RQ1). 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 Trace Size (# of instructions in billion) summary exhaustive • 44% smaller traces with summary usage. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (19/24) ASE 2013
44. Empirical Study Experiment Two Experiment Two: Accuracy. RQ2 How does the reuse of dynamic dependence summaries affect the accuracy of dynamic analysis? Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (20/24) ASE 2013
45. Empirical Study Experiment Two Experiment Two: Accuracy. RQ2 How does the reuse of dynamic dependence summaries affect the accuracy of dynamic analysis? Metrics Found Bugs. Runtime Overhead. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (20/24) ASE 2013
46. Empirical Study Experiment Two Experiment Two: Accuracy. RQ2 How does the reuse of dynamic dependence summaries affect the accuracy of dynamic analysis? Metrics Found Bugs. Runtime Overhead. Treatment Exhaustive. Summary-based. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (20/24) ASE 2013
47. Empirical Study Experiment Two Experiment Two: Accuracy. RQ2 How does the reuse of dynamic dependence summaries affect the accuracy of dynamic analysis? Metrics Found Bugs. Runtime Overhead. Treatment Exhaustive. Summary-based. Client Subject NANOXML (7KLOC) Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (20/24) ASE 2013
48. Empirical Study Experiment Two Experiment Two: Accuracy. RQ2 How does the reuse of dynamic dependence summaries affect the accuracy of dynamic analysis? Metrics Found Bugs. Runtime Overhead. Treatment Exhaustive. Summary-based. Client Subject NANOXML (7KLOC) Library Subject JAVA DEVELOPMENT KIT (rt.jar) Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (20/24) ASE 2013
49. Empirical Study Experiment Two Results: Found Bugs (RQ2). NanoXML: Exhaustive vs. Summary 18 20 0 2 4 6 8 10 12 14 16 18 20 Bugs Found (out of 20) summary exhaustive Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (21/24) ASE 2013
50. Empirical Study Experiment Two Results: Runtime Overhead (RQ2). NanoXML: Exhaustive vs. Summary 3,838 33 1 10 100 1,000 10,000 Runtime Overhead (ratio) summary exhaustive Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (22/24) ASE 2013
51. Takeaways Open Issues. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (23/24) ASE 2013
52. Takeaways Open Issues. • Assess suitability for summarization; adequacy criteria. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (23/24) ASE 2013
53. Takeaways Open Issues. • Assess suitability for summarization; adequacy criteria. • Comparison with static summaries. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (23/24) ASE 2013
54. Takeaways Open Issues. • Assess suitability for summarization; adequacy criteria. • Comparison with static summaries. • Accuracy analysis with multiple test subjects and client analyses. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (23/24) ASE 2013
55. Takeaways Takeaways. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (24/24) ASE 2013
56. Takeaways Takeaways. • Theory, Models, Implementation for construction and reuse of Dynamic Summaries. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (24/24) ASE 2013
57. Takeaways Takeaways. • Theory, Models, Implementation for construction and reuse of Dynamic Summaries. • 2× performance gains (best case: 3.6×) while analyzing large software benchmarks. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (24/24) ASE 2013
58. Takeaways Takeaways. • Theory, Models, Implementation for construction and reuse of Dynamic Summaries. • 2× performance gains (best case: 3.6×) while analyzing large software benchmarks. • Empirical study indicates cost savings with modest accuracy losses. Palepu, Xu, Jones (UCI) Dynamic Dependence Summaries (24/24) ASE 2013