Evaluating Software Quality Practices in Industry in Europe

We evaluated the state of the practice in companies residing in Spain, France, Belgium and Netherlands with respect to the use of processes, techniques and tools related to software quality. Through an online survey we aimed to obtain a better insight in the software-quality-related practices in companies involved in producing and maintaining software. We wanted to determine the most common industry practices with respect to software quality, how these practices vary across companies, and how this diverges from the state-of-the-art in software quality research. Presented at Benevol 2012 (http://swerl.tudelft.nl/bin/view/Main/BENEVOL2012).

Transcript

1. Evalua&ng)So,ware)Quality) Prac&ces)in)Industry) in)Europe) Preliminary*Results* Université*de*Mons* * Javier*Pérez,*Tom*Mens,*Jorge*Pinna*Puissant* Alexander*Serebrenik* Technische*Universiteit*Eindhoven* *
2. BENEVOL 2012 Portefeuille*TIC**  ERDF*project*lead*by*CETIC*(2007M2013)*  CEIQS:*Center*of*experSse*in*engineering*and*quality*of*systems**   aimed*at*developing*a*porUolio*of*innovaSve*techniques*allowing*local* companies*to*master*the*diversity,*complexity,*quality*and*rapid*evoluSon*of* informaSon*systems*  workpackage*QUALGEN*   collaboraSon*between*FUNDP*and*UMONS*since*2010* *  Supported*by*Wallonia* Context) 2* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant*
3. BENEVOL 2012 Explore*qualityMrelated*so\ware*development* pracSce*in*industry*  Target:*Companies*involved*in*so\ware*development*or** so\ware*maintenance*in*Europe*  Procedure:*OnMline*quesSonnaire* * Compare*this*across*different*countries*  Survey*in*4*languages*(English,*French,*Spanish,*Dutch)* * Objec&ves) 3* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant*
4. BENEVOL 2012 Online*survey*carried*out*in*the*Walloon*region*  Carried*out*from*29/5*to*30/6*2012*  44*full*useful*responses*from*188*parScipants* Obtained*iniSal*results*  Popularity*of*processes,*tools*and*techniques*  Some*pracSces*seem*to*vary*with*company*size* * Pilot)Study) 4* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant*
5. BENEVOL 2012 Pilot)Study:)Popularity)Results) quality)improvement)tools)and)techniques) 5* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* Most)popular)  Version*control*(97,6%)*and*bug*tracking*(92,7%)*plaUorms*  Wide*use*of*tesSng:*97,7%*  Design*pagerns:*72,7%*  Refactoring:*58,5%* Moderate)popularity)  Design*improvement*(e.g.*code*smell*reducSon):*42,9%*  Bad*quality*detecSon*tools:*36,8%*  Metrics*and*visualisaSon*tools:*35,7%*  Dynamic*analysis*tools*(profiling*etc.):*36,6%* Unpopular)  Use*of*quality*models:*19,4%*
6. BENEVOL 2012 Pilot)Study:)Popularity)Results) process)related) 6* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* Most)popular)  Agile*pracSces*(63,6%)*  Change*and*configuraSon*management*processes*(73,8%)* Moderate)popularity)  Development*processes*(45,5%)*  Test*processes*(46,5%)*  Quality*support/improvement*process*(32,4%)* All*respondents*believe*that*quality(assurance(and*tes.ng*are*very* important*for*project*success*
7. BENEVOL 2012 * * Pilot)Study:Effect)of)Company)Size) 7* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant*  Agile*pracSces*popular*regardless*of*company*size*  No*clear*difference*in*development*process*across*company*sizes*  Quality*process*mostly*used*by*big*and*medium*companies**  Micro*companies*behave*differently*(regarding*processes*and*agile)*
8. BENEVOL 2012 Topics*addressed*  Use*of*a*parScular*development*process*  Use*of*structural*so\ware*quality*measurement*and*improvement*  Use*of*tesSng*  Use*of*quality*models*and*quality*standards*  Development*tool*support*for*the*above** * Survey*procedure*  Online*quesSonnaire:*October*–*November*2012*  46*quesSons*  Convenience*sampling*–*invitaSons*sent*to*potenSal*respondents*  171*useful*responses**(unSl*15*November)*   155*from*NL*(51%),*BE*(18%),*ES*(16%),*FR*(15%)* Survey)Summary) 8* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant*
9. BENEVOL 2012 Company)Size)Distribu&on) 9* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* Micro − 33 (21%) Small − 42 (27%) Medium − 18 (12%) Big − 60 (39%) NA's − 2 (1%)
10. BENEVOL 2012 Usage*of*development*processes*  No*difference*between*countries*(more*imposed*in*France)*  Difference*by*company*size,*processes*increasedly*used*and* followed*by*company*size:* Processes) 10* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 10%* 20%* 30%* 40%* 50%* 60%* 70%* 80%* 90%* 100%* Micro* Small* Medium* Big* Used* Always*or*frequently* strictly*followed*
11. BENEVOL 2012 Usage*of*agile*methodologies*  Generalized*in*all*countries*  Balance*between*selfMchoice*and*imposed*for*small*and*big* companies* Agile)Methodologies) 11* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 10%* 20%* 30%* 40%* 50%* 60%* Micro* Small* Medium* Big* Imposed* Self*choice* Not*used*
12. BENEVOL 2012 Design*artefacts*and*documentaSon*  Documented*design?*No*difference*by*country*(48%*M*65%)**  Increased*use*and*imposiSon*by*size,*selfMchosen*in*small* companies* * * Design) 12* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 5%* 10%* 15%* 20%* 25%* 30%* 35%* 40%* 45%* 50%* Micro* Small* Medium* Big* Imposed* Self*choice* Not*used*
13. BENEVOL 2012 UpdaSng*design*artefacts*and*documentaSon*  Bigger*companies*update*more*frequently*  Small*companies*update*less*frequently*than*micro* * Design) 13* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 20%* 40%* 60%* 80%* 100%* Micro* Small* Medium* Big* Never/scarcely* Frequent/conSnuously*
14. BENEVOL 2012  No*significant*difference* between*countries* (popularity*between* 69%*and*77%)*  More*imposed*in*big* companies*  Less*used*in*micro* companies* * * Use)of)Design)PaKerns) 14* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 20%* 40%* 60%* 80%* 100%* Micro* Small* Medium* Big* Imposed* Self*choice* Not*used*
15. BENEVOL 2012  Increased*usage*by*company*size* * * Change)Management)Processes) 15* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 20%* 40%* 60%* 80%* 100%* Micro* Small* Medium* Big* Imposed* Self*choice* No*
16. BENEVOL 2012 By*company*size*  Increased*usage*and*imposiSon*by*company*size* * * Test)Processes) 16* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 20%* 40%* 60%* 80%* 100%* Micro* Small* Medium* Big* Imposed* Self*choice* Not*used*
17. BENEVOL 2012 By*company*size*  Similar*distribuSon*for*big*and*micro*  Medium*15%*M*25%**  Small*5%*M*15%* * * Tes&ng)&me) 17* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 50%* 100%* Micro* Small* Medium* Big* Less*than*5%* Between*5%*and* 15%* Between*15%*and* 25%* More*than*25%*
18. BENEVOL 2012 By*company*size*  Increasedly*more*used*and*imposed*by*company*size* * * * Quality)Processes) 18* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 20%* 40%* 60%* 80%*100%* Micro* Small* Medium* Big* Imposed* Self*choice* No* I*don't*know*
19. BENEVOL 2012 By*company*size*  Increased*usage*by*company*size*  More*“I*don’t*knows”*in*medium*companies* * * * Quality)Models) 19* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* 0%* 50%* 100%* Micro* Small* Medium* Big* Yes* No* I*don't*know*
20. BENEVOL 2012 Not*evenly*distributed*  Results*might*be*biased*  Get*more*responses*or*adjust*the*results* * Country)&)Size)Distribu&on) 20* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant* Micro Small Medium Big Belgium 7 8 1 12 France 8 5 3 6 Netherlands 16 17 8 37 Spain 2 12 6 5
21. BENEVOL 2012 Not*clear*differences*between*countries*  Small*differences*for*some*parScular*cases* ** Differences*company*size*  No*difference*for*agile*methodologies*  Usage*increases*by*size*for*the*rest:*processes,*usage*and*updates*of* design*documents,*tesSng*processes*and*dedicated*Sme,*quality* processes*and*models*  ImposiSon*of*processes*or*pracSces*increases*with*company*size* *  Help*for*gathering*more*contacts*and*improving*the*sample* distribuSon* * Conclusions) 21* So\ware*Quality*PracSces*in*Europe****************|****************Pérez,*Serebrenik,*Mens,*Pinna*Puissant*