4/14/11 Enhancements to MC/DC Test Data Generation Tool Anuj Thhakar(ID 072099) Dharmesh Kakadia(ID 072033) Internal Guide: Prof. C K Bhensdadia DDU, Nadiad External Guide: Ulka Shrotri & R Venkatesh Software R&D Lab, TRDDC, Pune 

4/14/11 The Holy Grail of Automated Testing •Software testing is very effort-intensive and expensive  ~ 50% of the cost of software development •As the size and complexity of software increases, the testing remains the bottleneck of software development. This calls for the Automation of testing. •Significant reduction in cost of software development if Automated •AutoGen, Automated Test-Data Generation Tool  Developed at TRDDC, Pune  Generates Test-data from given C source code  Supports multiple Code Coverage Criteria ✔ MC/DC, Boundary Value Analysis 

4/14/11 Issues with AutoGen •Effectiveness  No Standard C Library implementation. ✔ Thus could not effectively generate test data for the programs with Standard C Library function calls •Scalability  Scalability issues with larger programs ✔ Bottleneck : Scalability in Model Checking  Multiple passes over the code to analyse and annotate the C Program. 

4/14/11 Enhancements to AutoGen •Effectiveness  Implementation of Standard C Library suitable for Model Checking. ✔ Exact vs Model Implementation •Scalability  Development of Intermediate Representation(IR) model to C Unparser  Development and Integration of Program Slicer ✔ Integration of IRUnparser with IR Slicer 

4/14/11 Definitions •Coverage  Extent to which a given verification activity has satisfied its objectives. •MC/DC Coverage Criterion  Every point of entry and exit in the program has been invoked at least once,  Every condition in a decision in the program has taken all possible outcomes at least once, • Every decision in the program has taken all possible outcomes at least once, and • Each condition in a decision has been shown to independently affect that decision’s outcome. 

4/14/11 Definitions •Model Checking The process of converting one form( e.g., C Program ) to a model( e.g, Boolean Equation ), and then checking if the model satisfies certain properties( e.g., Satisfiable? ) •Model Checker  Tool that perform model checking  May even find the parameters such that model invalidate the specified properties  Example, CBMC ( assert-based C Bounded Model Checker ) 

4/14/11 •AutoGen Current Implementation 

4/14/11 Solution : Effectiveness •What we need ?  Implementation of the standard C library functions, which exhibits the same behaviour (for model-checking).  Implementation need to be efficient for model-checking.  To model the functions that takes values from user or environment, generate values. ✔ Provide the generated values to the end-user for testing. •What we don’t need ?  No need for actual implementation of the function. ✔ Except in few cases, exact implementation is required (e.g., for ctype, Math library). 

4/14/11 Solution : Effectiveness Strategy •Model Implementation  Global Array for every function ✔ Also considered as Input variable ✔ Model Checker generates value for it  which can be provided to End-User  Access through incrementing index ✔ Thus different values for each call •Exact implementation.  Functions from stringh, ctype.h, math.h 

4/14/11 Solution : Effectiveness • Supported Headers  math.h  ctype.h  string.h  pthread.h  unistd.h  fcntl.h  stdio.h  termios.h  ... 

4/14/11 Solution : Scalability •What we can do ?  Reduce number of passes.  Reduce the program size. •Improvements for better scalability  Add IR Unparser and Annotator to AutoGen and combine passes -- Reduces Passes  Add Program Slicer to slice the given C program in such way to produce the same behaviour as the original C program – Reduces Program size 

4/14/11 Solution : Scalability 

4/14/11 IRUnparser •Converts Intermediate Representation(IR) Model to C code  Code should be Semantically Correct and Syntactically Valid •IR Model •ST, AST and Application Model •Access through PRISM APIs •Challenges  Basically Tree Traversal  Ordering of Datatype Definitions non-deterministic in IR ✔ Solution : Topological Sort for Datatype dependencies  Special Cases ✔ Function Pointers ✔ Typedefs, etc. 

4/14/11 Solution : Scalability 

4/14/11 Program Slicer •Slicer  Removes Non-affecting statements w.r.t. slice points  Smaller code – Easier to understand and model-check  Slice points on Expressions on some line no. ✔ Resulting program contains statements which affect execution till that line •Integration of IR Slicer with IRUnparser  IR Slicer returns Set of Statements which should be present in Sliced Program  Challenges  Cases like removal of then statement from if..then construct  All Labels should be preserved @ proper locations  Line number information 

4/14/11 Demo if(j==5) { j=i; j=1; } else i=2; for(cnt=0;cnt<5;cnt++) { j++; i=j; j+=5; } foo(); i++; i++; return 1; } int i,j,cnt; int foo() { int local=5; local*=3; return local; } int bar(){ i++; return 1; } int main() { foo(); bar(); j=++i; 

4/14/11 Demo if(j==5) { j=1; } else i=2; for(cnt=0 ; cnt<5 ; cnt++) { j++; i=j; j+=5; } i++; return 1; } int main(); int foo(); int bar(); int i; int j; int cnt; int bar() { i++; return 1; } int main() { bar(); j=++i; 

4/14/11 Conclusions and Future Extensions •This Project provides three enhancements to AutoGen, namely  Implementation of common library functions for efficient Model- checking.  Development of IR to C Unparser for code annotation, and  Integration of Unparser with IR Slicer resulting in Program Slicer. •With these changes AutoGen is more scalable and effective test-data generation tool. •What Next ?  Extension of unparser to support unparsing of C++.  Slicing of Input C Source Code ✔ Before any other processing 

4/14/11 •“Automatic Test Data Generation for C Programs”, TRDDC, Pune. •C Bounded Model Checker Manual. •"The Standard C Library” by P. J. Plauger ( Prentice Hall Publication, 1992 ). •C99 Standard •TCS PRISM API manuals References 

4/14/11 Thank You