Continuous Code Quality: Are We (Really) Doing That?

Our study on Continuous Code Quality in Continuous Integration presented at the IEEE/ACM International Conference on Automated Software Engineering (ASE '18) in Montpellier, France. Preprint of the corresponding paper available at http://doi.org/10.5281/zenodo.1341036

Transcript

1. Continuous Code Quality:
   Are We (Really) Doing That?
   Carmine Vassallo, Fabio Palomba, Alberto Bacchelli, and Harald C. Gall.
   @ccvassallo
   IEEE/ACM International Conference on Automated Software Engineering, Montpellier (France), 2018
   

   View Slide

2. 2
   “Improving software quality and reducing risks”
   (Paul Duvall)
   

   View Slide

3. 3
   Commit
   Repository
   Build
   Check
   Build Server
   CCQ Service
   Continuous Code Quality (CCQ)
   Configure
   

   View Slide

4. How do developers apply CCQ?
   4
   

   View Slide

5. 5
   CCQ Data Collection
   

   View Slide

6. 6
   Commit
   Repository
   Build
   Check
   Build Server
   CCQ Service
   CCQ Data Collection
   

   View Slide

7. 7
   Commit
   Repository
   Build
   Check
   Build Server
   CCQ Service
   CCQ Data Collection
   

   View Slide

8. 8
   Commit
   Repository
   Build
   Check
   Build Server
   CCQ Service
   CCQ Data Collection
   

   View Slide

9. 9
   Commit
   Repository
   Build
   Check
   Build Server
   CCQ Service
   CCQ Data Collection
   

   View Slide

10. 10
    Commit
    Repository
    Build
    Check
    Build Server
    CCQ Service
    119 Java
    projects
    148,734 builds
    (5 years)
    CCQ Data Collection
    

    View Slide

11. 11
    CCQ Indicators
    

    View Slide

12. The quality of most builds is not inspected
    only 11% of the builds have been checked
    Projects with longer change history exhibit lower rate
    12
    Code Quality Checking Rate
    Fraction of builds that have been checked
    

    View Slide

13. New quality inspections are performed after several builds
    On average 1 check every 18 builds
    Developers tend to perform code quality check at the end of a sprint
    13
    Elapsed Frame/Time between Checks
    Frequency of the quality checks
    

    View Slide

14. A low percentage of branches are scheduled for quality check
    Overall, only 36% of branches are checked
    14
    Checked Branches
    Percentage of branches containing at least one build subject to a code
    quality check
    

    View Slide

15. 15
    Continuous Code Quality:
    Are We (Really) Doing That?
    Carmine Vassallo, Fabio Palomba, Alberto Bacchelli,
    and Harald C. Gall.
    @ccvassallo
    vassallo@ifi.uzh.ch
    CCQ Dataset
    Code quality is not continuously checked
    Code Quality Inspection in CI
    Future Work
    Quality outcome of the current CCQ practice
    Code quality recommendation in CI
    Tendency to avoid checking feature branches
    Key scenarios in Continuous Code Quality
    Mature projects are less likely to apply CCQ
    X
    Commit
    Repository
    Build
    Check
    Build Server
    CCQ Service
    CCQ Data Collection
    

    View Slide