TMPA-2021: SPIDER: Specification-based Integration Defect Revealer

Vladislav Feofilaktov and Vladimir Itsykson, SPBSTU

SPIDER: Specification-based Integration Defect Revealer

TMPA is an annual International Conference on Software Testing, Machine Learning and Complex Process Analysis. The conference will focus on the application of modern methods of data science to the analysis of software quality.

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Transcript

1. 1 25-27 NOVEMBER SOFTWARE TESTING, MACHINE LEARNING AND COMPLEX PROCESS ANALYSIS SPIDER: Specification-based Integration Defect Revealer Vladislav Feofilaktov, Vladimir Itsykson, SPbPU, JetBrains Research
2. 2 Introduction Current practices of the software development: 1. Applications use external components and libraries 2. Mostly, external libraries are produced by third party developers Usually, these libraries are blackboxes that provide a public API
3. 3 Problems with external libraries 1. Libraries are provided without docs or with badly docs 2. Descriptions of API calls and types aren’t enough 3. Examples are uninformative 4. Library component semantics are unclear Integration errors
4. 4 // abstract file library FileReader file = new FileReader(path); // … file.close(); // … file.readLine(); // IOException: Stream closed Trivial example of integration error
5. 5 Aim of the research 1. Develop the approach: a. Allows to find integration errors b. Must be statically 2. Develop the tool: a. Implement the approach b. Use java bytecode as a target c. Evaluate the tool on real projects
6. 6 State of the art Existing approaches have some next restrictions: 1. Focuses only on the C language 2. Requires library source code to make analysis possible 3. Requires library code changes to write incode specification 4. Don’t allow to check a correctness of library usage
7. 7 Our approach 1. Formalize the library semantics 2. Formal definition of integration errors 3. Apply static analysis methods for automatic detection of integration errors
8. 8 Specification of external library LibSL (Library Specification Language) is a declarative language that allows writing components specification using extended finite state machines. LibSL allows to describe: 1. Structure of library 2. Public API functions 3. API functions behaviour 4. Library component’s behaviour Itsykson V. M. LibSL: Language for Specification of Software Libraries, Programmnaya Ingeneria, 2018, vol. 9, no. 5, pp. 209—220.
9. 9 Simple example of LibSL specification libsl "1.0.0"; library File version "1.0.0"; types { File (org.jetbrains.research.File); String (string); } automaton org.jetbrains.research.File : File { initstate Init; state Open; finishstate Close; shift Init -> Open(open); shift Open -> Close(close); fun open(name: String) requires isNameEmpty: name != “”; fun close(); } Set LibSL version Specification for library ‘File’ version 1.0.0 Set semantic types and their java qualified names Description of an automaton with 1. initstate 2. simple state 3. finishstate 4. shift from state “Init” to state “Open” on calling function “open” and another one 5. function “open” with one String argument 6. precondition with name “isNameEmpty” 7. just a function
10. 10 Definition of integration errors Types of integration errors: 1. API usage errors 2. Incorrect scenarios of library usage API usage errors: API function contracts violations Incorrect scenarios of library usage: 1. Incorrect order of API function calls 2. Incorrect finish state of automata
11. 11 High-level scheme of our approach Static Analyzer Library Approximation Generator Error List Library Specification Library Main Application Library Approximation
12. 12 Prototype implementation 1. Target platform JVM 2. LibSL parser 3. Static analyzer platform KEX We have developed prototype named SPIDER: Specification-based Integration Defect Revealer
13. 13 Kex • Java bytecode static analysis platform • Supports few modes and allows writing new ones • Uses special internal representation KFG • Transforms source program to logical predicates • Feed logical predicates to the SMT-solver A. Abdullin, M. Akhin and M. Belyaev, "Kex at the 2021 SBST Tool Competition," 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST), 2021, pp. 32-33, doi: 10.1109/SBST52555.2021.00014
14. 14 The way to check correctness 1. Use LibSL specification as a library oracle 2. Modify KFG to informate KEX about correct library behaviour 3. KEX analyzes the modified KFG to find errors
15. 15 Detailed scheme of SPIDER KEX Static Analyzer LibSL Parser KFG Builder Instrumentation Module Analysis Module Library Specification Library Compiled Binaries Main Application Compiled Binaries Error List ASG Library KFG Application KFG Instrumented Library KFG
16. 16 Summary of SPIDER • Implemented as KEX module • Written in Kotlin • Use LibSL parser* • Use Kfg library to build KFG • Z3 and boolector as SMT-solvers *https://github.com/vorpal-research/libsl
17. 17 Evaluation • Synthetic test projects ◦ To verify the correctness of built model ◦ To check KFG instrumentation ◦ To verify detection of all types of errors • Applicability approach to real-world library okHTTP* library with real curl implementation ◦ To check the project without changes ◦ To check the project with manually added errors *https://github.com/square/okhttp
18. 18 Conclusion Results: • An approach was developed to find integration errors • The approach was implemented as the tool named SPIDER • SPIDER was evaluated on synthetic and real projects Feature plans: • Add rest integration error types to SPIDER • Increase the performance • Evaluate SPIDER on many open source projects
19. 19 Thank You! Follow TMPA on Facebook TMPA-2021 Conference