Naming the Pain in Requirements Engineering

A talk given at the 2015 European Open Symposium on Empirical Software Engineering on our ongoing initiative on establishing an empirical basis for requirements engineering.

Transcript

1. Pain Requirements Engineering www.uni-stuttgart.de Naming the Stefan Wagner @prof_wagnerst (joint work with Daniel Méndez Fernández, Michael Felderer, Marcos Kalinowski, Antonio Vetró et al.) EOSESE 2015, Lille, France 3. December 2015 in
2. You can copy, share and change, film and photograph, blog, live-blog and tweet this presentation given that you attribute it to its author and respect the rights and licences of its parts. Slide is based on suggestions by @SMEasterbrook und @ethanwhite
3. There are mostly isolated investigations of methods in requirements engineering research.
4. Research is not driven by problems from industry.
5. An empirical understanding of the state of the practice and problems in RE is necessary for relevant research.
6. Technische Universität München 1 Idea and Design
7. NaPiRE focusses on five main research questions Research questions and methods 1. What is the status quo in RE? 2. What is the status quo in RE process standards? 3. What is the status quo in RE improvement? 4. What contemporary problems exist in RE and how do they manifest themselves? 5. Are there observable patterns of expectations, status quo, and problems in RE? „Theory of expectation“ „Grounded Theory“ Bi-yearly World-wide
8. International Replication Global Replication Validation Preparation Germany International Communities Conceptualisation of research questions Presentation & discussions at communities Initial creation of questionnaire Internal validation External validation Industrial pilot Initiation First interpretation Creation Distribution List Implementation / correction Invitation Synthesis & reporting 1. Replication (Netherlands) Dissemination Creation distribution list Invitation Data analysis & interpretation Final reporting Online Survey Spreadsheet Data Analysis & Interpretation Creation Distribution List Invitation Creation distribution list Baseline Report Yearly RE Community Report Replication Report Survey Results Creation distribution list Invitation Data analysis & interpretation Planning PROMISE Repository Data Analysis & Interpretation Invitation Data analysis & interpretation Current Stage
9. Validation Preparation Germany International Communities Conceptualisation of research questions Presentation & discussions at communities Initial creation of questionnaire Internal validation External validation Industrial pilot Initiation First interpretation Implementation / correction 1. Replication (Netherlands) Creation distribution list Invitation Data analysis & interpretation Final reporting Online Survey Spreadsheet Baseline Report Replication Report Survey Results Creation distribution list Invitation Data analysis & interpretation
10. International Replication Global Replication Initiation First interpretation Creation Distribution List Invitation Synthesis & reporting 1. Replication (Netherlands) Dissemination Creation distribution list Invitation Data analysis & interpretation Final reporting Data Analysis & Interpretation Creation Distribution List Invitation Creation distribution list Baseline Report Yearly RE Community Report Replication Report Survey Results Creation distribution list Invitation Data analysis & interpretation Planning PROMISE Repository Data Analysis & Interpretation Invitation Data analysis & interpretation Current Stage
11. Req Elicitation Technique Interview Scenario Prototyping Facilitated Meetings Observation Req Documentation Technique Structured req list Domain/business process model Use case model Goal model Data model Non-functional req Textual Semi-formal Formal Technology Req Test Alignment Approach Req review by tester Coverage by tests Acceptance criteria Test derivation from models Req Change Approach Product backlog update Change requests Trace management Impact analysis Activity Req Elicitation Req Documentation Req Change Management Req Test Alignment P 1-5 P 6-13 P 14-20 P 21-24 Actor Req Engineer Test Engineer Req Standard Application Practice Control Tailoring Req Eng Process Standard P 25-28 Req Standard Defintion Compliance Development Tool support Quality assurance Project management Knowledge transfer Process complexity Communication demand Willigness to change Possibility of standardisation P 26-45 Req Improvement Means Continuous improvement Strengths/weaknesses Own business unit/role Req Eng Improvement P 46--48 Theory
12. Req Elicitation Technique Interview Scenario Prototyping Facilitated Meetings Observation Req Documentation Technique Structured req list Domain/business process model Use case model Goal model Data model Non-functional req Textual Semi-formal Formal chnology Req Test Alignment Approach Req Change Approach Product backlog update Change requests Trace management Impact analysis Activity Req Elicitation Req Documentation Req Change Management P 1-5 P 6-13 P 14-20
13. Technische Universität München 2 Results from Germany D. Méndez Fernández, S. Wagner. Naming the pain in requirements engineering: a design for a global family of surveys and first results from Germany. Information and Software Technology 57:616–643, 2015.
14. 58 completed questionnaires Custom Software Development 36 % IT Consulting 36 % Project Management Consulting 35 % Software Process Consulting 31 % Standard Software Development 28 % Embedded Software Development 7 %
15. Workshops with stakeholders Change requests Prototyping Other Agile approaches Elicitation Q 16 Current reference model Q 17 RE reference model with artefacts RE reference model with roles RE reference model predefined by process No reference model RE reference model 48 % 44 % 18 % 15 % 80 % 58 % 48 % 44 % 7%
16. Common RE problems 0" 0" 0" 1" 1" 2" 0" 3" 4" 1" 5" 1" 3" 3" 5" 7" 7" 1" 9" 11" 16" 0" 1" 2" 2" 3" 3" 5" 8" 8" 9" 9" 11" 11" 12" 13" 13" 16" 20" 22" 22" 31" 0" 5" 10" 15" 20" 25" 30" Weak"domain"knowledge"" Weak"rela9onship"to"customer" High"degree"of"innova9on"vs."need"for"formal"acceptance" Technically"unfeasible"reqs." Insufficient"support"by"project"lead" Weak"access"to"customer"needs" Vola9le"domain"" Unclear"responsibili9es" Insufficient"support"by"customer" Terminological"problems" Unclear"nonMfunc9onal"reqs." Gold"pla9ng" Missing"traceability" Communica9on"flaws"in"team" Inconsistent"reqs." Communica9on"flaws"to"customer" Underspecified"reqs." Separa9on"reqs."from"known"solu9ons" Time"boxing" Moving"targets" Incomplete"/"hidden"reqs." Overall"frequency" Cause"for"project"fail" Overall'view'on'problems''in'rela0on'to'project'fail Problem'#'1'(n=50) Problem'#'2'(n=48) Problem'#'3'(n=45) Problem'#'4'(n=40) Problem'#'5'(n=38) 31 16 22 22 20 16 11 9 1 13 13 12 11 11 9 9 8 8 7 7 5 3 3 1 1 4 3 5 3 3 2 2 2 0 0 0 0 0 0 5 10 15 20 25 30
17. 0" 3" 4" 1" 5" 1" 3" 3" 5" 1" 5" Vola9le"domain"" Unclear"responsibili9es" Insufficient"support"by"customer" Terminological"problems" Unclear"nonMfunc9onal"reqs." Gold"pla9ng" Missing"traceability" Communica9on"flaws"in"team" Inconsistent"reqs." Communica9on"flaws"to"customer" Underspecified"reqs." Separa9on"reqs."from"known"solu9ons" Time"boxing" Moving"targets" Incomplete"/"hidden"reqs." 1 5 3 3 1 1 4 3 5 0 Common RE problems
18. 0" 0" 0" 1" 1" 2" 0" 3" 1" 1" 3" 3" 0" 1" 2" 2" 3" 3" Weak"domain"knowledge"" Weak"rela9onship"to"customer" High"degree"of"innova9on"vs."need"for"formal"acceptance" Technically"unfeasible"reqs." Insufficient"support"by"project"lead" Weak"access"to"customer"needs" Vola9le"domain"" Unclear"responsibili9es" Insufficient"support"by"customer" Terminological"problems" Unclear"nonMfunc9onal"reqs." Gold"pla9ng" Missing"traceability" Communica9on"flaws"in"team" Inconsistent"reqs." Communica9on"flaws"to"customer" Underspecified"reqs." 3 3 1 1 3 3 3 2 2 2 0 0 0 0 0 Common RE problems
19. Technische Universität München 3 Early Results from 2nd Run D. Méndez Fernández, S. Wagner, M. Kalinowski, M. Felderer, P. Mafra, A. Vetrò et al. Naming the pain in requirements engineering: Contemporary Problems, Causes, and Effects in Practice. Submited to Empirical Software Engineering.
20. Countries that conducted surveys in 2014/15 Austria, Brazil, Canada, China, Estonia, Finland, Germany, Ireland, Luxembourg, Netherlands, Norway, Spain, Sweden, USA
21. Formal/semi-formal goal models are rarely used in practice. Central Europe North/Eastern Europe North America South America 5 % 18 % 1 % 2 % 5 % 4 % 7 % 7 % Semi-formal Formal
22. Causes for incomplete/hidden requirements
23. Effects of incomplete/hidden requirements
24. An empirical understanding of the state of the practice and problems in RE is necessary for relevant research. Slides are available at www.stefan-wagner.biz.
25. Pictures Used in this Slide Deck Zürich Neumünster Basis by Ikiwaner (https://upload.wikimedia.org/wikipedia/ commons/a/a9/Zuerich_Neumuenster_Basis.jpg) Carlota and Isabel Islands by Storm Crypt (https://flic.kr/p/5DjfE2) The Bosch Multi-Storey Car Park by rykerstribe (https://flic.kr/p/53hosQ)