Enhancements to MC/DC Test Data Generation Tool

Transcript

1. 4/14/11
   Enhancements to
   MC/DC Test Data Generation Tool
   Anuj Thhakar(ID 072099)
   Dharmesh Kakadia(ID 072033)
   Internal Guide: Prof. C K Bhensdadia
   DDU, Nadiad
   External Guide: Ulka Shrotri &
   R Venkatesh
   Software R&D Lab,
   TRDDC, Pune
   

   View Slide

2. 4/14/11
   The Holy Grail of Automated Testing
   •Software testing is very effort-intensive and expensive
   
   ~ 50% of the cost of software development
   •As the size and complexity of software increases, the testing
   remains the bottleneck of software development. This calls for
   the Automation of testing.
   •Significant reduction in cost of software development if
   Automated
   •AutoGen, Automated Test-Data Generation Tool
   
   Developed at TRDDC, Pune
   
   Generates Test-data from given C source code
   
   Supports multiple Code Coverage Criteria
   ✔
   MC/DC, Boundary Value Analysis
   

   View Slide

3. 4/14/11
   Issues with AutoGen
   •Effectiveness
   
   No Standard C Library implementation.
   ✔
   Thus could not effectively generate test data for the
   programs with Standard C Library function calls
   •Scalability
   
   Scalability issues with larger programs
   ✔
   Bottleneck : Scalability in Model Checking
   
   Multiple passes over the code to analyse and annotate
   the C Program.
   

   View Slide

4. 4/14/11
   Enhancements to AutoGen
   •Effectiveness
   
   Implementation of Standard C Library suitable for Model
   Checking.
   ✔
   Exact vs Model Implementation
   •Scalability
   
   Development of Intermediate Representation(IR) model
   to C Unparser
   
   Development and Integration of Program Slicer
   ✔
   Integration of IRUnparser with IR Slicer
   

   View Slide

5. 4/14/11
   Definitions
   •Coverage
    Extent to which a given verification activity has satisfied its
   objectives.
   •MC/DC Coverage Criterion
   
   Every point of entry and exit in the program has been invoked
   at least once,
   
   Every condition in a decision in the program has taken all
   possible outcomes at least once,
   • Every decision in the program has taken all possible outcomes
   at least once, and
   • Each condition in a decision has been shown to independently
   affect that decision’s outcome.
   

   View Slide

6. 4/14/11
   Definitions
   •Model Checking
   The process of converting one form( e.g., C Program ) to a
   model( e.g, Boolean Equation ), and then checking if the model
   satisfies certain properties( e.g., Satisfiable? )
   •Model Checker
   
   Tool that perform model checking
   
   May even find the parameters such that model invalidate the
   specified properties
   
   Example, CBMC ( assert-based C Bounded Model Checker )
   

   View Slide

7. 4/14/11
   •AutoGen
   Current Implementation
   

   View Slide

8. 4/14/11
   Solution : Effectiveness
   •What we need ?
   
   Implementation of the standard C library functions, which
   exhibits the same behaviour (for model-checking).
   
   Implementation need to be efficient for model-checking.
   
   To model the functions that takes values from user or
   environment, generate values.
   ✔
   Provide the generated values to the end-user for testing.
   •What we don’t need ?
   
   No need for actual implementation of the function.
   ✔
   Except in few cases, exact implementation is required (e.g.,
   for ctype, Math library).
   

   View Slide

9. 4/14/11
   Solution : Effectiveness
   Strategy
   •Model Implementation
   
   Global Array for every function
   ✔
   Also considered as Input variable
   ✔
   Model Checker generates value for it
   
   which can be provided to End-User
   
   Access through incrementing index
   ✔
   Thus different values for each call
   •Exact implementation.
   
   Functions from stringh, ctype.h, math.h
   

   View Slide

10. 4/14/11
    Solution : Effectiveness
    • Supported Headers
    
    math.h
    
    ctype.h
    
    string.h
    
    pthread.h
    
    unistd.h
    
    fcntl.h
    
    stdio.h
    
    termios.h
    
    ...
    

    View Slide

11. 4/14/11
    Solution : Scalability
    •What we can do ?
    
    Reduce number of passes.
    
    Reduce the program size.
    •Improvements for better scalability
    
    Add IR Unparser and Annotator to AutoGen and combine
    passes -- Reduces Passes
    
    Add Program Slicer to slice the given C program in such way to
    produce the same behaviour as the original C program –
    Reduces Program size
    

    View Slide

12. 4/14/11
    Solution : Scalability
    

    View Slide

13. 4/14/11
    IRUnparser
    •Converts Intermediate Representation(IR) Model to C code
    
    Code should be Semantically Correct and Syntactically Valid
    •IR Model
    •ST, AST and Application Model
    •Access through PRISM APIs
    •Challenges
    
    Basically Tree Traversal
    
    Ordering of Datatype Definitions non-deterministic in IR
    ✔
    Solution : Topological Sort for Datatype dependencies
    
    Special Cases
    ✔
    Function Pointers
    ✔
    Typedefs, etc.
    

    View Slide

14. 4/14/11
    Solution : Scalability
    

    View Slide

15. 4/14/11
    Program Slicer
    •Slicer
    
    Removes Non-affecting statements w.r.t. slice points
    
    Smaller code – Easier to understand and model-check
    
    Slice points on Expressions on some line no.
    ✔
    Resulting program contains statements which affect execution till
    that line
    •Integration of IR Slicer with IRUnparser
    
    IR Slicer returns Set of Statements which should be present in
    Sliced Program
    
    Challenges
    
    Cases like removal of then statement from if..then construct
    
    All Labels should be preserved @ proper locations
    
    Line number information
    

    View Slide

16. 4/14/11
    Demo
    if(j==5) {
    j=i;
    j=1;
    }
    else
    i=2;
    for(cnt=0;cnt<5;cnt++) {
    j++;
    i=j;
    j+=5;
    }
    foo();
    i++;
    i++;
    return 1;
    }
    int i,j,cnt;
    int foo() {
    int local=5;
    local*=3;
    return local;
    }
    int bar(){
    i++; return 1;
    }
    int main()
    {
    foo();
    bar();
    j=++i;
    

    View Slide

17. 4/14/11
    Demo
    if(j==5)
    {
    j=1;
    }
    else
    i=2;
    for(cnt=0 ; cnt<5 ; cnt++)
    {
    j++;
    i=j;
    j+=5;
    }
    i++;
    return 1;
    }
    int main();
    int foo();
    int bar();
    int i;
    int j;
    int cnt;
    int bar()
    {
    i++;
    return 1;
    }
    int main()
    {
    bar();
    j=++i;
    

    View Slide

18. 4/14/11
    Conclusions and Future Extensions
    •This Project provides three enhancements to AutoGen, namely
    
    Implementation of common library functions for efficient Model-
    checking.
    
    Development of IR to C Unparser for code annotation, and
    
    Integration of Unparser with IR Slicer resulting in Program Slicer.
    •With these changes AutoGen is more scalable and effective test-data
    generation tool.
    •What Next ?
    
    Extension of unparser to support unparsing of C++.
    
    Slicing of Input C Source Code
    ✔
    Before any other processing
    

    View Slide

19. 4/14/11
    •“Automatic Test Data Generation for C Programs”, TRDDC, Pune.
    •C Bounded Model Checker Manual.
    •"The Standard C Library” by P. J. Plauger ( Prentice Hall Publication,
    1992 ).
    •C99 Standard
    •TCS PRISM API manuals
    References
    

    View Slide

20. 4/14/11
    Thank You
    

    View Slide