Are Non-Functional Requirements Really non-functional?

I gave this talk at SE 2017 in Hannover.

Transcript

1. Are “Non-functional” Requirements really Non-functional? Jonas Eckhardt1, Andreas Vogelsang2, Daniel Méndez Fernández1 @andivogelsang andreas.vogelsang@tu-berlin.de Software Engineering, Hannover, 2017 originally presented at ICSE 2016 An investigation of Non-functional Requirements in Practice 1Technische Universität München 2Technische Universität Berlin February 23, 2017 1
2. Let‘s play a game… Functional, Non-functional, or What? “There must be sufficient development documentation. This concerns the code documentation (inline comments).” “The delay between passing a [message] and decoding of the first loop message shall be < [x] seconds.” “Critical software components shall be restored upon failure when feasible.” A classification always depends on • Perspective • Purpose 2
3. What NFRs are vs. What practitioners consider as NFRs vs. What practitioners document as NFRs 3
4. Study Design • 11 requirements specifications • 5 different companies • 1841 requirements NFR NFR NFR NFR NFR • 346 NFRs (19%) • Data Preparation 346  530 NFRs • Classification RQ1: What NFR classes are documented in practice? RQ3: How many NFRs describe system behavior? RQ4: What kind of behavior do NFRs describe? 4
5. Study Design: Classification Requirement Representational Black-Box (Interface) Glass-Box Behavioral Architecture State Quality View [1]: System View [2]: “There must be sufficient development documentation. This concerns the code documentation (inline comments).” “The delay between passing a [message] and decoding of the first loop message shall be < [x] seconds.” “Critical software components shall be restored upon failure when feasible.” Maintainability Representational Efficiency Black-Box Reliability Architecture [1] ISO/IEC 9126: Software engineering -- Product quality -- Part 1: Quality model [2] Broy: Rethinking Nonfunctional Software Requirements, IEEE Computer, 2015 5
6. Study Results: What NFR classes are documented in practice? Summary: 27% (144) of all NFRs are classified as functionality Interpretation: Requirements labeled as NFR often describe functionality 6
7. Study Results: How many NFRs describe system behavior? Summary: 75% of all NFRs describe behavior of the system, 25% describe representational aspects. Interpretation: Many requirements labeled as NFR actually describe behavior 7
8. Study Results: What kind of behavior do NFRs describe? Summary: 69% of all behavioral NFRs describe interface behavior, 21% describe architectural behavior, 10% describe behavior related to states of the system. Interpretation: NFRs are not non-functional 8
9. Conclusions from the Study • Requirements labeled as NFR often describe functionality • Many requirements labeled as NFR actually describe behavior • NFRs are not non-functional 9
10. Labeling requirements as NFRs does make a difference! Eckhardt, Vogelsang, Méndez: On the Distinction of Functional and Quality Requirements in Practice, PROFES 2016 • Do you document NFRs? • Do you distinguish between NFRs and FRs in the documentation?” 12 13 88 75 yes no Survey with 103 practitioners: 10 Reasoning not clear • Contrary arguments “Both are requirements” vs. “[they] are different” • Same line of reasoning “increase awareness” by distinguishing and by not distinguishing NFRs and FRs Unclear decision implications • Impact on testing Distinguishing leads to more specialized tests, but also to not testing them at all • Lack of guidance “There is no real guidelines how to do it”
11. @andivogelsang andreas.vogelsang@tu-berlin.de • What are the consequences in practice? • What is the purpose of classifying requirements? • How can different perspectives on classifying requirements be integrated? Summary Thank you. We classified 27% of the overall NFR population as Functionality. FRs are often labeled as NFRs Most (69%) of the behavioral NFRs describe the same type of behavior as FRs do (i.e. black-box interface behavior). NFRs are not non-functional Manual analysis of documented requirements labeled as NFR