Automatic program instrumentation to the rescue!

Interested in learning more about this topic? Visit this web site to see related presentations: https://www.gregorykapfhammer.com/research/presentations/

Transcript

1. Automatic Program Instrumentation to the Rescue! Gregory M. Kapfhammer Department of Computer Science Allegheny College Mary Lou Soffa Department of Computer Science University of Virginia Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 1/15
2. What is My Program Doing? Contribution: An instrumentation framework to support testing, analysis, debugging, and understanding Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 2/15
3. Potential Probe Locations Program and Test Suite Java Virtual Machine Operating System Database Manager Java Virtual Machine Operating System JDBC Driver Instrumentation probes can be placed in many locations How can we “best” capture the behavior of a program? Is it possible to automatically introduce the probes? What tools already exist? What approach is the most efficient? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 3/15
4. Understanding Static Instrumentation Insert the probes into the source code or bytecode Instrumentation occurs before program execution Less flexible if a program regularly changes What is the impact on space overhead? Aspect-oriented programming versus bytecode instrumentation Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 4/15
5. Dynamic Instrumentation Program Stack Native Code Cache Fast? Interpreter? Machine Virtual JIT? Adaptive? Class Loader Heap methodA testOne Input Output Byte Code P Perform dynamic instrumentation at established interface(s) Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 5/15
6. Constructing Dynamic Call Trees B B G G H I H A C F D E Number of Nodes = 13, Number of Edges = 12 Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 6/15
7. Using Calling Context Trees B G H I A C F D E Number of Nodes = 9, Number of Edges = 10 Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 7/15
8. Static Instrumentation Time FF PI RM ST TM GB All Application 2 4 6 8 10 Static Instrumentation Time sec FF PI RM ST TM GB All 4.391 4.404 4.396 4.394 5.169 5.583 8.687 Instrumentation never takes longer than nine seconds Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 8/15
9. Space Overhead of the Instrumentation ZIP GZIP PACK Compression Technique TM 10000 20000 30000 40000 50000 Application Size bytes ZIP GZIP PACK 14582 11424 6058 48887 45730 45650 A As Increase in the number of bytecodes is substantial Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 9/15
10. Size of the Instrumented Applications Compr Tech Before Instr (bytes) After Instr (bytes) None 29275 887609 Zip 15623 41351 Gzip 10624 35594 Pack 5699 34497 Average static size across all case study applications Compressed the bytecodes with general purpose techniques Specialized compressor nicely reduces space overhead Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 10/15
11. Size of the Instrumentation Probes Compression Technique Probe Size (bytes) None 119205 Zip 40017 Gzip 34982 Pack 35277 420% average increase in space overhead! Why? Reflection vs. extra bytecode instructions Is this increase in space overhead acceptable? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 11/15
12. Static and Dynamic Time Overhead Norm Sta CCT Sta DCT Dyn CCT Dyn DCT Instrumentation Technique Tree Type TM 2 4 6 8 10 12 14 TCM Time sec Norm Sta CCT Sta DCT Dyn CCT Dyn DCT 6.933 8.02 8.796 11.245 11.916 What trends can you find in this graph? Which tree and instrumentation technique would you pick? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 12/15
13. Test Execution Time Overhead Instr Tech Tree Type TCM Time (sec) Percent Increase (%) Static CCT 7.44 12.5 Static DCT 8.35 26.1 Dynamic CCT 10.17 53.0 Dynamic DCT 11.0 66.0 Normal average testing time of 6 . 62 seconds Which tree and instrumentation technique is most efficient? Which configuration would you select? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 13/15
14. Average Tree Storage Time Tree Type Tree Representation Tree Storage Time (msec) CCT Binary 144.9 DCT Binary 1011.72 CCT XML 408.17 DCT XML 2569.22 Strengths and weaknesses of tree representations Is it ever better to store the tree in XML? Which configuration would you use? Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 14/15
15. Conclusions and Future Work Automatic program instrumentation can save the day! A complete framework for recording call trees Useful applications: test coverage monitoring, performance analysis, regression test suite reduction Characterizing the DCT and CCT for object-oriented programs Automatic visualization of method input and output What are your suggestions? I value your comments and participation! Automatic Program Instrumentation to the Rescue!, RICSS, February 9, 2007 – p. 15/15