Upgrade to Pro — share decks privately, control downloads, hide ads and more …

TMPA-2021: SPIDER: Specification-based Integration Defect Revealer

TMPA-2021: SPIDER: Specification-based Integration Defect Revealer

Vladislav Feofilaktov and Vladimir Itsykson, SPBSTU

SPIDER: Specification-based Integration Defect Revealer

TMPA is an annual International Conference on Software Testing, Machine Learning and Complex Process Analysis. The conference will focus on the application of modern methods of data science to the analysis of software quality.

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

5206c19df417b8876825b5561344c1a0?s=128

Exactpro
PRO

November 27, 2021
Tweet

Transcript

  1. 1 25-27 NOVEMBER SOFTWARE TESTING, MACHINE LEARNING AND COMPLEX PROCESS

    ANALYSIS SPIDER: Specification-based Integration Defect Revealer Vladislav Feofilaktov, Vladimir Itsykson, SPbPU, JetBrains Research
  2. 2 Introduction Current practices of the software development: 1. Applications

    use external components and libraries 2. Mostly, external libraries are produced by third party developers Usually, these libraries are blackboxes that provide a public API
  3. 3 Problems with external libraries 1. Libraries are provided without

    docs or with badly docs 2. Descriptions of API calls and types aren’t enough 3. Examples are uninformative 4. Library component semantics are unclear Integration errors
  4. 4 // abstract file library FileReader file = new FileReader(path);

    // … file.close(); // … file.readLine(); // IOException: Stream closed Trivial example of integration error
  5. 5 Aim of the research 1. Develop the approach: a.

    Allows to find integration errors b. Must be statically 2. Develop the tool: a. Implement the approach b. Use java bytecode as a target c. Evaluate the tool on real projects
  6. 6 State of the art Existing approaches have some next

    restrictions: 1. Focuses only on the C language 2. Requires library source code to make analysis possible 3. Requires library code changes to write incode specification 4. Don’t allow to check a correctness of library usage
  7. 7 Our approach 1. Formalize the library semantics 2. Formal

    definition of integration errors 3. Apply static analysis methods for automatic detection of integration errors
  8. 8 Specification of external library LibSL (Library Specification Language) is

    a declarative language that allows writing components specification using extended finite state machines. LibSL allows to describe: 1. Structure of library 2. Public API functions 3. API functions behaviour 4. Library component’s behaviour Itsykson V. M. LibSL: Language for Specification of Software Libraries, Programmnaya Ingeneria, 2018, vol. 9, no. 5, pp. 209—220.
  9. 9 Simple example of LibSL specification libsl "1.0.0"; library File

    version "1.0.0"; types { File (org.jetbrains.research.File); String (string); } automaton org.jetbrains.research.File : File { initstate Init; state Open; finishstate Close; shift Init -> Open(open); shift Open -> Close(close); fun open(name: String) requires isNameEmpty: name != “”; fun close(); } Set LibSL version Specification for library ‘File’ version 1.0.0 Set semantic types and their java qualified names Description of an automaton with 1. initstate 2. simple state 3. finishstate 4. shift from state “Init” to state “Open” on calling function “open” and another one 5. function “open” with one String argument 6. precondition with name “isNameEmpty” 7. just a function
  10. 10 Definition of integration errors Types of integration errors: 1.

    API usage errors 2. Incorrect scenarios of library usage API usage errors: API function contracts violations Incorrect scenarios of library usage: 1. Incorrect order of API function calls 2. Incorrect finish state of automata
  11. 11 High-level scheme of our approach Static Analyzer Library Approximation

    Generator Error List Library Specification Library Main Application Library Approximation
  12. 12 Prototype implementation 1. Target platform JVM 2. LibSL parser

    3. Static analyzer platform KEX We have developed prototype named SPIDER: Specification-based Integration Defect Revealer
  13. 13 Kex • Java bytecode static analysis platform • Supports

    few modes and allows writing new ones • Uses special internal representation KFG • Transforms source program to logical predicates • Feed logical predicates to the SMT-solver A. Abdullin, M. Akhin and M. Belyaev, "Kex at the 2021 SBST Tool Competition," 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST), 2021, pp. 32-33, doi: 10.1109/SBST52555.2021.00014
  14. 14 The way to check correctness 1. Use LibSL specification

    as a library oracle 2. Modify KFG to informate KEX about correct library behaviour 3. KEX analyzes the modified KFG to find errors
  15. 15 Detailed scheme of SPIDER KEX Static Analyzer LibSL Parser

    KFG Builder Instrumentation Module Analysis Module Library Specification Library Compiled Binaries Main Application Compiled Binaries Error List ASG Library KFG Application KFG Instrumented Library KFG
  16. 16 Summary of SPIDER • Implemented as KEX module •

    Written in Kotlin • Use LibSL parser* • Use Kfg library to build KFG • Z3 and boolector as SMT-solvers *https://github.com/vorpal-research/libsl
  17. 17 Evaluation • Synthetic test projects ◦ To verify the

    correctness of built model ◦ To check KFG instrumentation ◦ To verify detection of all types of errors • Applicability approach to real-world library okHTTP* library with real curl implementation ◦ To check the project without changes ◦ To check the project with manually added errors *https://github.com/square/okhttp
  18. 18 Conclusion Results: • An approach was developed to find

    integration errors • The approach was implemented as the tool named SPIDER • SPIDER was evaluated on synthetic and real projects Feature plans: • Add rest integration error types to SPIDER • Increase the performance • Evaluate SPIDER on many open source projects
  19. 19 Thank You! Follow TMPA on Facebook TMPA-2021 Conference