Naming the Pain in Requirements Engineering

Talk at the conference Software Engineering 2016 in Vienna, Austria. It describes the current status of the NaPiRE project which has the aim to capture the state of the practice and problems in requirements engineering.
http://www.re-survey.org

Transcript

1. Wien, 2016-02-24 Stefan Wagner Naming the Pain in Requirements Engineering
2. You can copy, share and change, film and photograph, blog, live-blog and tweet this presentation given that you attribute it to its author and respect the rights and licences of its parts. based on slides by @SMEasterbrook und @ethanwhite
3. There are mostly isolated investigations of methods in requirements engineering research.
4. Research is not driven by problems from industry.
5. An empirical understanding of the state of the practice and problems in RE is necessary for relevant research.
6. 1 Ideas and Design
7. Research questions and methods 1. What is the status quo in RE? 2. What is the status quo in RE process standards? 3. What is the status quo in RE improvement? 4. What contemporary problems exist in RE and how do they manifest themselves? 5. Are there observable patterns of expectations, status quo, and problems in RE? Bi-yearly World-wide
8. International Replication Global Replication Validation Preparation Germany International Communities Conceptualisation of research questions Presentation & discussions at communities Initial creation of questionnaire Internal validation External validation Industrial pilot Initiation First interpretation Creation Distribution List Implementation / correction Invitation Synthesis & reporting 1. Replication (Netherlands) Dissemination Creation distribution list Invitation Data analysis & interpretation Final reporting Online Survey Spreadsheet Data Analysis & Interpretation Creation Distribution List Invitation Creation distribution list Baseline Report Yearly RE Community Report Replication Report Survey Results Creation distribution list Invitation Data analysis & interpretation Planning PROMISE Repository Data Analysis & Interpretation Invitation Data analysis & interpretation Current Stage
9. Validation Preparation Germany International Communities Conceptualisation of research questions Presentation & discussions at communities Initial creation of questionnaire Internal validation External validation Industrial pilot Initiation First interpretation Implementation / correction 1. Replication (Netherlands) Creation distribution list Invitation Data analysis & interpretation Final reporting Online Survey Spreadsheet Baseline Report Replication Report Survey Results Creation distribution list Invitation Data analysis & interpretation
10. International Replication Global Replication Initiation First interpretation Creation Distribution List Invitation Synthesis & reporting 1. Replication (Netherlands) Dissemination Creation distribution list Invitation Data analysis & interpretation Final reporting Data Analysis & Interpretation Creation Distribution List Invitation Creation distribution list Baseline Report Yearly RE Community Report Replication Report Survey Results Creation distribution list Invitation Data analysis & interpretation Planning PROMISE Repository Data Analysis & Interpretation Invitation Data analysis & interpretation Current Stage
11. Req Elicitation Technique Interview Scenario Prototyping Facilitated Meetings Observation Req Documentation Technique Structured req list Domain/business process model Use case model Goal model Data model Non-functional req Textual Semi-formal Formal Technology Req Test Alignment Approach Req review by tester Coverage by tests Acceptance criteria Test derivation from models Req Change Approach Product backlog update Change requests Trace management Impact analysis Activity Req Elicitation Req Documentation Req Change Management Req Test Alignment P 1-5 P 6-13 P 14-20 P 21-24 Actor Req Engineer Test Engineer Req Standard Application Practice Control Tailoring Req Eng Process Standard P 25-28 Req Standard Defintion Compliance Development Tool support Quality assurance Project management Knowledge transfer Process complexity Communication demand Willigness to change Possibility of standardisation P 26-45 Req Improvement Means Continuous improvement Strengths/weaknesses Own business unit/role Req Eng Improvement P 46--48 Theory
12. Req Elicitation Technique Interview Scenario Prototyping Facilitated Meetings Observation Req Documentation Technique Structured req list Domain/business process model Use case model Goal model Data model Non-functional req Textual Semi-formal Formal Technology Req Test Alignment Approach Req Change Approach Product backlog update Change requests Trace management Impact analysis Activity Req Elicitation Req Documentation Req Change Management P 1-5 P 6-13 P 14-20
13. 2 Results from Germany D. Méndez Fernández, S. Wagner. Naming the pain in requirements engineering: a design for a global family of surveys and first results from Germany. Information and Software Technology 57:616–643, 2015.
14. 58 completed questionnaires Custom Software Development 36 % IT Consulting 36 % Project Management Consulting 35 % Software Process Consulting 31 % Standard Software Development 28 % Embedded Software Development 7 %
15. Workshops with stakeholders Change requests Prototyping Other Agile approaches RE reference model with artefacts 48 % 80 % 58 % 48 % 44 % 7% How do you elicit requirements?
16. Common RE problems 0" 0" 0" 1" 1" 2" 0" 3" 4" 1" 5" 1" 3" 3" 5" 7" 7" 1" 9" 11" 16" 0" 1" 2" 2" 3" 3" 5" 8" 8" 9" 9" 11" 11" 12" 13" 13" 16" 20" 22" 22" 31" 0" 5" 10" 15" 20" 25" 30" Weak"domain"knowledge"" Weak"rela9onship"to"customer" High"degree"of"innova9on"vs."need"for"formal"acceptance" Technically"unfeasible"reqs." Insufficient"support"by"project"lead" Weak"access"to"customer"needs" Vola9le"domain"" Unclear"responsibili9es" Insufficient"support"by"customer" Terminological"problems" Unclear"nonMfunc9onal"reqs." Gold"pla9ng" Missing"traceability" Communica9on"flaws"in"team" Inconsistent"reqs." Communica9on"flaws"to"customer" Underspecified"reqs." Separa9on"reqs."from"known"solu9ons" Time"boxing" Moving"targets" Incomplete"/"hidden"reqs." Overall"frequency" Cause"for"project"fail" Overall'view'on'problems''in'rela0on'to'project'fail Problem'#'1'(n=50) Problem'#'2'(n=48) Problem'#'3'(n=45) Problem'#'4'(n=40) Problem'#'5'(n=38) 31 16 22 22 20 16 11 9 1 13 13 12 11 11 9 9 8 8 7 7 5 3 3 1 1 4 3 5 3 3 2 2 2 0 0 0 0 0 0 5 10 15 20 25 30
17. 1" 5" 1" 3" 3" 5" 1" Insufficient"support"by"customer" Terminological"problems" Unclear"nonMfunc9onal"reqs." Gold"pla9ng" Missing"traceability" Communica9on"flaws"in"team" Inconsistent"reqs." Communica9on"flaws"to"customer" Underspecified"reqs." Separa9on"reqs."from"known"solu9ons" Time"boxing" Moving"targets" Incomplete"/"hidden"reqs." 1 5 3 3 1 1 4 Common RE problems
18. 0" 0" 0" 1" 1" 2" 0" 3" 1" 1" 3" 3" 0" 1" 2" 2" 3" 3" Weak"domain"knowledge"" Weak"rela9onship"to"customer" High"degree"of"innova9on"vs."need"for"formal"acceptance" Technically"unfeasible"reqs." Insufficient"support"by"project"lead" Weak"access"to"customer"needs" Vola9le"domain"" Unclear"responsibili9es" Insufficient"support"by"customer" Terminological"problems" Unclear"nonMfunc9onal"reqs." Gold"pla9ng" Missing"traceability" Communica9on"flaws"in"team" Inconsistent"reqs." 3 3 1 1 3 3 3 2 2 2 0 0 0 0 0
19. 3 Early results from 2nd run D. Méndez Fernández, S. Wagner, M. Kalinowski, M. Felderer, P. Mafra, A. Vetrò et al. Naming the Pain in Requirements Engineering: Contemporary Problems, Causes, and Effects in Practice. Submited to Empirical Software Engineering.
20. Sample 10 Countries 228 Companies
21. Formal/semi-formal goal models are rarely used in practice. Central Europe North/Eastern Europe North America South America 5 % 18 % 1 % 2 % 5 % 4 % 7 % 7 % Semi-formal Formal
22. Causes for incomplete/hidden requirements
23. Causes for incomplete/hidden requirements
24. Effects of incomplete/hidden requirements
26. An empirical understanding of the state of the practice and problems in RE is necessary for relevant research.
27. Prof. Dr. Stefan Wagner e-mail stefan.wagner@informatik.uni-stuttgart.de phone +49 (0) 711 685-88455 WWW www.iste.uni-stuttgart.de/se Twitter prof_wagnerst ORCID 0000-0002-5256-8429 Institute of Software Technology Slides are available at www.stefan-wagner.biz.
28. Pictures Used in this Slide Deck Alex (https://flic.kr/p/84ThFe) Zürich Neumünster Basis by Ikiwaner (https://upload.wikimedia.org/wikipedia/commons/a/ a9/Zuerich_Neumuenster_Basis.jpg) Carlota and Isabel Islands by Storm Crypt (https://flic.kr/p/5DjfE2) The Bosch Multi-Storey Car Park by rykerstribe (https://flic.kr/p/53hosQ)