Take Care of Your Modes! An Investigation of Defects in Automotive Requirements

I held this talk at the 2016 REFSQ conference in Githenburg, Sweden

Transcript

1. Take Care of Your Modes! An Investigation of Defects in Automotive Requirements REFSQ 2016, Gothenburg, Sweden March 15, 2016 Andreas Vogelsang, Henning Femmer Institut für Informatik Technische Universität München Christian Winkler MAN Truck & Bus AG Munich, Germany @andivogelsang
2. Modes in Natural Language Requirements 2 “The air conditioning must maintain the desired temperature if the engine is running.” Requirement quality defects Implicit usage of modes
3. What is (considered as) a mode? A mode is an information that • may change during the runtime of a system • is characterized by a discrete set of values • is relevant for more than one specific vehicle function Examples: • High temperature • Ignition • Engine operation • ACC active 3 Vogelsang, Femmer, Winkler: Systematic Elicitation of Mode Models for Multifunctional Systems, RE’15
4. Study Design 4 • RQ1: How many defects in NL requirements specifications mention modes? • RQ2: Which types of requirement defects are issued by mode-related findings? • RQ3: Which types of problems categorize the mode-related findings? Requirement quality defects Implicit usage of modes
5. Data Collection and Analysis 5 Findings classification System-level requirements specification Review protocol Findings extraction Review Findings preparation Defect Type=?, Mode-related: Yes/No 134 review findings 64 review findings Performed by MAN Performed by us Findings preparation: Removal of • Process improvements • Spelling mistakes • Rejected findings • Duplicate findings Study objects: 2 review protocols for 2 system-level requirements specs (41 requirements in total)
6. Findings Classification 6 Defect type: ISO/IEC/IEEE 29148: • Necessary • Implementation Free • Unambiguous • Consistent • Complete • Singular • Feasible • Traceable • Verifiable Mode-related: Criteria taken from previous study [1]: • Yes • No [1] Vogelsang, Femmer, Winkler: Systematic Elicitation of Mode Models for Multifunctional Systems, RE’15 Addition to the requirement: “must be shown to the driver in specific situations“ This requirement is not necessary anymore because, at MAN, the daytime light is a separate lamp. IRA (defect type): 0.68 (substantial agreement ) IRA (mode-related): 0.42 (moderate agreement )
7. Study Results – RQ1 and RQ2 7 30/64 (46%) review findings are mode-related > 60% of findings for complete and unambiguous are mode-related
8. Let’s look into the details… 8 Defect type: Unambiguous What does “active” in this case mean? That a warning is active or that the possibility to warn is active. Where is the definition of “active”, “activation”, etc.? Defect type: Complete Missing requirement: During initialization of the vehicle, the optical icons must be activated.
9. RQ3: Types of Problems Related to Modes 9 Category Problem description P1 (8) A specific mode/situation is not considered in the function. P2 (8) A precise mode definition is missing. P3 (5) The reaction of a function to a mode switch is unclear. P4 (4) It is unclear how modes are logically connected in a function. P5 (2) The mode definition must change according to the context. P6 (2) The validity of a mode is doubted.
10. Modes in Requirements Engineering “The air conditioning must maintain the desired temperature if the engine is running.” FKN1 FKN2 FKN3 FKN4 FZGFKN1 FZGFKN2 What does “engine running” in this case exactly mean? What happens in case of start/stop? We need a model of modes for a system! Precise definitions Source for requirements Tracing Thank you. @andivogelsang vogelsan@in.tum.de