$30 off During Our Annual Pro Sale. View Details »

Context Is King: The Developer Perspective on the Usage of Static Analysis Tools

Context Is King: The Developer Perspective on the Usage of Static Analysis Tools

Our study about the usage of Automatic Static Analysis Tools in different software development contexts presented at the 25th edition of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER '18) in Campobasso, Italy. Preprint of the corresponding paper available at http://www.ifi.uzh.ch/seal/people/vassallo/VassalloSANER18.pdf

Carmine Vassallo

March 21, 2018
Tweet

More Decks by Carmine Vassallo

Other Decks in Technology

Transcript

  1. 1
    Context Is King:
    The Developer Perspective on the Usage of Static Analysis Tools.
    Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
    Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo

    View Slide

  2. 2
    Development Context Is King:
    The Developer Perspective on the Usage of Static Analysis Tools.
    Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
    Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo

    View Slide

  3. 3
    Development Context Is King:
    The Developer Perspective on the Usage of Static Analysis Tools.
    Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
    Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo

    View Slide

  4. 4
    Motivation

    View Slide

  5. 5
    ASAT
    (A) Automatic
    (S) Static
    (A) Analysis
    (T) Tool

    View Slide

  6. 6
    ASATs detect so+ware defects faster and cheaper than human
    inspec6on and tes6ng would do (Johnson et al., ICSE 2013).
    ASATs are common, but not ubiquitous (Beller et al., SANER 2016)

    View Slide

  7. 7
    Barriers when using ASATs
    Lack of effec6vely
    implemented quick fixes
    Johnson et al.,
    “Why don’t software developers use
    Static Analysis Tools to Find Bugs?”
    ICSE 2013
    High rate of false posi6ve
    warnings
    Low understandability of the
    warnings

    View Slide

  8. 8
    Usage of ASATs in one context
    Panichella et al.,
    “Would static analysis tools help
    developers with code reviews?”
    SANER 2015
    Zampetti et al.,
    “How open source projects use static
    code analysis tools in continuous
    integration”
    MSR 2017
    Build failures caused by ASATs are mainly
    due to coding standard viola.ons
    Developers use ASATs mainly for checking
    coding structure
    Code Review
    Continuous
    Integration

    View Slide

  9. ASATs configured differently in
    different contexts.
    9

    View Slide

  10. TO DO List
    10
    Development Contexts where ASATs are used
    Usage of ASATs in different contexts

    View Slide

  11. First Study:
    Development Contexts
    11

    View Slide

  12. First Study: Research Questions
    • RQ1: In which development contexts do
    developers use ASATs? 

    • RQ2: How do developers configure ASATs in
    different development contexts?
    12

    View Slide

  13. The Questionnaire
    13
    19 questions, 2 main topics:
    • Adoption of ASATs
    • Configuration of ASATs
    43 (69% industrial and 31%
    open-source) participants.

    View Slide

  14. Usage of ASATs
    14
    Frequency
    Multiple times per day
    Daily
    Weekly
    Monthly
    % Respondents
    0 10 20 30 40
    12
    19
    31
    38
    ASATs are integrated with the regular development

    View Slide

  15. 15
    Where ASATs are used

    View Slide

  16. Where ASATs are used
    16
    30% 33% 37%
    % Respondents
    Local Development Code Review Continuous Integration

    View Slide

  17. When ASATs are configured
    17
    Frequency
    Kick-off
    Monthly
    Never
    Weekly
    % Respondents
    0 15 30 45 60
    7
    20
    22
    51
    The majority of developers configure ASATs only once.

    View Slide

  18. 18
    How ASATs are configured
    of our respondents use the
    same configuration in
    different contexts.
    %
    75

    View Slide

  19. How ASATs are configured
    19
    Local Development Code Review Continuous Integration

    View Slide

  20. Second Study:
    ASATs usage in different contexts
    20

    View Slide

  21. Extended questionnaire & Interviews
    21
    Context-Based Usage
    25 participants
    11 professional
    developers
    • 6 companies
    Semi-structured
    interviews

    View Slide

  22. Second Study: Research Question
    • RQ3 Do developers pay attention to the same
    warnings in different development contexts?
    22

    View Slide

  23. Warnings in different contexts
    23
    Local Development Code Review Continuous Integration
    Developers pay attention to different warnings depending on the context.
    Code Structure
    Logic
    Error Handling
    Style Convention
    Redundancies
    Naming Conventions
    Error Handling
    Logic
    Style Convention
    1st
    2nd
    3rd
    1st
    2nd
    3rd
    1st
    2nd
    3rd

    View Slide

  24. Other factors while selecting warnings
    24
    Factors
    Severity of the Warnings
    Policies of the Development Team
    Application Type
    Team Composition
    None of the above
    Tool Reputation
    % Respondents
    0 15 30 45 60
    0
    6.1
    6.1
    12.1
    24.2
    51.5
    2.4
    2.4
    9.9
    19.5
    31.7
    34.1
    2.3
    7
    11.6
    18.6
    27.9
    32.6
    Continuous Integration Code Review Local Development
    Blocker, Cri>cal, Major, etc.
    “Team leader decides to adopt a strict
    policy regarding naming conven6ons.”
    “Short-term applica6ons don’t need to
    follow strict rules.”

    View Slide

  25. 25
    Findings

    View Slide

  26. How developers configure ASATs
    26
    Local Development Code Review Continuous Integration

    View Slide

  27. How developers perceive ASATs
    27
    Local Development Code Review Continuous Integration

    View Slide

  28. 28
    Biased Percep6on
    Towards Context-Awareness
    Holis6c Analysis of the
    Developers’ Behaviour

    View Slide

  29. 29
    Context Is King:
    The Developer Perspective on the Usage of Static Analysis
    Tools.
    Carmine Vassallo, Sebastiano Panichella, Fabio Palomba,
    Sebastian Proksch, Andy Zaidman, and Harald Gall.
    @ccvassallo
    [email protected]
    X
    Usage of ASATs in one context
    Panichella et al.,
    “Would static analysis tools help
    developers with code reviews?”
    SANER 2015
    Zampetti et al.,
    “How open source projects use static
    code analysis tools in continuous
    integration”
    MSR 2017
    Code Review
    Continuous
    Integration
    How ASATs are configured
    X
    Local Development Code Review Continuous Integration
    How developers perceive ASATs
    X
    Local Development Code Review Continuous Integration

    View Slide