Save 37% off PRO during our Black Friday Sale! »

Context Is King: The Developer Perspective on the Usage of Static Analysis Tools

Context Is King: The Developer Perspective on the Usage of Static Analysis Tools

Our study about the usage of Automatic Static Analysis Tools in different software development contexts presented at the 25th edition of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER '18) in Campobasso, Italy. Preprint of the corresponding paper available at http://www.ifi.uzh.ch/seal/people/vassallo/VassalloSANER18.pdf

84581630350da72785f7d2448549ca06?s=128

Carmine Vassallo

March 21, 2018
Tweet

Transcript

  1. 1 Context Is King: The Developer Perspective on the Usage

    of Static Analysis Tools. Carmine Vassallo, Sebastiano Panichella, Fabio Palomba, Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo
  2. 2 Development Context Is King: The Developer Perspective on the

    Usage of Static Analysis Tools. Carmine Vassallo, Sebastiano Panichella, Fabio Palomba, Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo
  3. 3 Development Context Is King: The Developer Perspective on the

    Usage of Static Analysis Tools. Carmine Vassallo, Sebastiano Panichella, Fabio Palomba, Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo
  4. 4 Motivation

  5. 5 ASAT (A) Automatic (S) Static (A) Analysis (T) Tool

  6. 6 ASATs detect so+ware defects faster and cheaper than human

    inspec6on and tes6ng would do (Johnson et al., ICSE 2013). ASATs are common, but not ubiquitous (Beller et al., SANER 2016)
  7. 7 Barriers when using ASATs Lack of effec6vely implemented quick

    fixes Johnson et al., “Why don’t software developers use Static Analysis Tools to Find Bugs?” ICSE 2013 High rate of false posi6ve warnings Low understandability of the warnings
  8. 8 Usage of ASATs in one context Panichella et al.,

    “Would static analysis tools help developers with code reviews?” SANER 2015 Zampetti et al., “How open source projects use static code analysis tools in continuous integration” MSR 2017 Build failures caused by ASATs are mainly due to coding standard viola.ons Developers use ASATs mainly for checking coding structure Code Review Continuous Integration
  9. ASATs configured differently in different contexts. 9

  10. TO DO List 10 Development Contexts where ASATs are used

    Usage of ASATs in different contexts
  11. First Study: Development Contexts 11

  12. First Study: Research Questions • RQ1: In which development contexts

    do developers use ASATs? 
 • RQ2: How do developers configure ASATs in different development contexts? 12
  13. The Questionnaire 13 19 questions, 2 main topics: • Adoption

    of ASATs • Configuration of ASATs 43 (69% industrial and 31% open-source) participants.
  14. Usage of ASATs 14 Frequency Multiple times per day Daily

    Weekly Monthly % Respondents 0 10 20 30 40 12 19 31 38 ASATs are integrated with the regular development
  15. 15 Where ASATs are used

  16. Where ASATs are used 16 30% 33% 37% % Respondents

    Local Development Code Review Continuous Integration
  17. When ASATs are configured 17 Frequency Kick-off Monthly Never Weekly

    % Respondents 0 15 30 45 60 7 20 22 51 The majority of developers configure ASATs only once.
  18. 18 How ASATs are configured of our respondents use the

    same configuration in different contexts. % 75
  19. How ASATs are configured 19 Local Development Code Review Continuous

    Integration
  20. Second Study: ASATs usage in different contexts 20

  21. Extended questionnaire & Interviews 21 Context-Based Usage 25 participants 11

    professional developers • 6 companies Semi-structured interviews
  22. Second Study: Research Question • RQ3 Do developers pay attention

    to the same warnings in different development contexts? 22
  23. Warnings in different contexts 23 Local Development Code Review Continuous

    Integration Developers pay attention to different warnings depending on the context. Code Structure Logic Error Handling Style Convention Redundancies Naming Conventions Error Handling Logic Style Convention 1st 2nd 3rd 1st 2nd 3rd 1st 2nd 3rd
  24. Other factors while selecting warnings 24 Factors Severity of the

    Warnings Policies of the Development Team Application Type Team Composition None of the above Tool Reputation % Respondents 0 15 30 45 60 0 6.1 6.1 12.1 24.2 51.5 2.4 2.4 9.9 19.5 31.7 34.1 2.3 7 11.6 18.6 27.9 32.6 Continuous Integration Code Review Local Development Blocker, Cri>cal, Major, etc. “Team leader decides to adopt a strict policy regarding naming conven6ons.” “Short-term applica6ons don’t need to follow strict rules.”
  25. 25 Findings

  26. How developers configure ASATs 26 Local Development Code Review Continuous

    Integration
  27. How developers perceive ASATs 27 Local Development Code Review Continuous

    Integration
  28. 28 Biased Percep6on Towards Context-Awareness Holis6c Analysis of the Developers’

    Behaviour
  29. 29 Context Is King: The Developer Perspective on the Usage

    of Static Analysis Tools. Carmine Vassallo, Sebastiano Panichella, Fabio Palomba, Sebastian Proksch, Andy Zaidman, and Harald Gall. @ccvassallo vassallo@ifi.uzh.ch X Usage of ASATs in one context Panichella et al., “Would static analysis tools help developers with code reviews?” SANER 2015 Zampetti et al., “How open source projects use static code analysis tools in continuous integration” MSR 2017 Code Review Continuous Integration How ASATs are configured X Local Development Code Review Continuous Integration How developers perceive ASATs X Local Development Code Review Continuous Integration