Requirements: Just in Time for System Evolution

Talk on system change and software requirements at the University of Cincinnati, October 2014.

Transcript

1. Requirements: Just-In-Time for System Evolution Neil Ernst Researcher, Software Engineering Institute nernst@sei.cmu.edu • http://neilernst.net • @neilernst
2. Overview • The inevitability of change and why should we care • Understanding change • Dealing with changes • Case study: RE in open source 2
3. Just-In-Time Requirements practices are different yet effective
4. “Traditional” RE 4 Requirements team separate and siloed, “transom- style” handoffs
5. “Traditional” RE 5 Typically (if not ideally) done once, at inception
6. “Traditional” RE 6 Store artifacts in management tool
7. 7 Pejoratively called:
 Big Requirements Up Front
8. Understanding Change • Just-in-time is a response to this • Context: evolving and agile software development 8
9. (Source: Godfrey & Tu, 2000) E.G. Linux Kernel 9
11. Lehman’s ‘Laws’ 1. Continuing change—A software system will become progressively less satisfying to its users over time, unless it is continually adapted to meet new needs. 2. Increasing complexity—A software system will become progressively more complex over time, unless explicit work is done to reduce its complexity. 3. Self-regulation—The process of software evolution is self-regulating, with close to normal distribution of the product and process artifacts that are produced. 4. Conservation of organizational stability—The average effective global activity rate on an evolving software system does not change over time; that is, the amount of work that goes into each release is about the same. 5. Conservation of familiarity—The amount of new content in each successive release of a software system tends to stay constant or decrease over time. 6. Continuing growth—The amount of functionality in a software system will increase over time, in order to please its users. 7. Declining quality—A software system will be perceived as declining in quality over time, unless its design is carefully maintained and adapted to new operational constraints. 8. Feedback System—Successfully evolving a software system requires recognition that the development process is a multi-loop, multiagent, multilevel feedback system. 11 Godfrey, M. W. and German, D. M. (2013), On the evolution of Lehman's Laws. J. Softw. Evol. and Proc.. doi: 10.1002/smr.1636
12. Time Functionality User needs identify requirements first release enhancement phase freeze and replace replacement delivered enhancement phase conventional development Lateness Shortfall Inappropriateness Longevity Adaptability (shaded area) (slope of line) Source: Adapted from Davis 1988, pp1453-1455 User Requirements Always Grow 12
14. Scale And Complexity 14
16. 16 http://www.youtube.com/watch?v=GiPe1OiKQuk
17. “Normal” Accidents 17 Perrow, Charles. Normal Accidents: Living with High-Risk Technologies New York: Basic Books, 1984.
19. Cynefin (Kih-Neh-Vin) 19 More: http://cognitive-edge.com/library/more/video/introduction-to-the-cynefin-framework/ Cannot gather upfront Get these right! Creativity Don’t really need RE here
20. “Let’s Build A Filter” 20
21. 21 Plan Driven Requirements Value Driven Requirements Date Resources Date Resources Fixed Estimated source: DSDM via Leffingwell 2011
22. Just-In-Time RE e.g. specification by example, behavior- driven development (BDD), feature driven development, user stories, acceptance testing. lightweight and iterative (?) 22 developers talk to “Product Owner” assume change and react, rather than plan RE is ongoing and continuous
23. Research Questions 1. How does JIT RE manifest itself in practice? 2. What problems might be encountered? 23
24. Methodology Choose 3 software projects that are successful, relatively distinct and open. Study each project’s software process holistically, starting at the task level. Choose a representative requirement for detailed study. 24
25. Flexible Indexing 25 Inverted index: terms point to containing documents Flexible indexing: add frequency, weights, boosts directly to index
26. 2004: Idea proposed on wiki (Doug Cutting) 2006: Email discussion about implementation (Grant Ingersoll) 2008: First JIRA ticket created (Michael McCandless) 2010: Feature branch integrated into trunk (all) 2012: Feature ships as 4.0 Alpha 2009: Feature progress misses release 2.9; code later released for stress testing by others (McCandless) 2009: Unicode incompatibility detected (Robert Muir)
27. "Have you tried any actual tests swapping these approaches in as your terms index impl?” “No – changing something like this requires a lot of coding, so it's better to do thought experiments first to winnow down the options." “ Mike, this change to byte[ ] in TermRef will break backwards compatibility, without some special attention paid to the utf-16 to utf-8 conversion. “ In my opinion it would be better to think in the future how we can improve lucene in the following ways: The term dictionary should be more "DFA-friendly", [etc.] “
28. Observations (Lucene) • Requirements arise organically. Never nailed down. • Strategic vision emergent rather than directed. No hard deadlines. • JIRA is central to RE process. • Detailed knowledge lives inside the developer/requester. 28
29. Common Practices (Jit In Practice) • Just-in-Time Requirements • Feature-oriented • As-needed Traceability • Exploratory and iterative development • Community-mindedness 29
30. Departures • No big-picture thinking • No separate prioritization • Unclear feature provenance • No repeatable elicitation or reuse 30
31. Understanding R.E. Practices Ad-hoc List Links Models Req Spec Ad-hoc Simple Priority Multiple dimensions Reflective Requirements+Management Requirements+Analysis Personal)projects Firefox Lucene CONNECT 31
32. But ... “I chose RE as a research area [after seeing] that insufficient RE causes inconsistent, incomplete and incorrect requirements specifications and is responsible for a significant number of problems encountered in development projects.” – Klaus Pohl, preface to RE textbook 32
33. Ultimate Empirical Question Amount of RE Software Value Idealized RE Perception Reality? Diminishing RE return 33
34. Related Work • Plenty of industry focus: • Leffingwell, Agile Manifesto, Agile BABOK, BDD etc. • Social nature of RE in OSS explored by Walter Scacchi. • Emmanuel Letier exploring requirements prediction. • Marco Lormans and Arie van Deursen worked on requirements coverage. • Niu et al. “Traceability-enabled refactoring for managing just-in-time requirements” • Heck and Zaidman: “Horizontal traceability for just-in-time requirements: the case for open source feature requests” 34
35. 35 Neil Ernst • @neilernst • neilernst.net Just-In-Time Requirements practices are different yet effective when? how? why?