Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Requirements: Just in Time for System Evolution

141272f109fbf660ffa001647f17d368?s=47 Neil Ernst
October 21, 2014

Requirements: Just in Time for System Evolution

Talk on system change and software requirements at the University of Cincinnati, October 2014.


Neil Ernst

October 21, 2014


  1. Requirements: Just-In-Time for System Evolution Neil Ernst Researcher, Software Engineering

    Institute nernst@sei.cmu.edu • http://neilernst.net • @neilernst
  2. Overview • The inevitability of change and why should we

    care • Understanding change • Dealing with changes • Case study: RE in open source 2
  3. Just-In-Time Requirements practices are different yet effective

  4. “Traditional” RE 4 Requirements team separate and siloed, “transom- style”

  5. “Traditional” RE 5 Typically (if not ideally) done once, at

  6. “Traditional” RE 6 Store artifacts in management tool

  7. 7 Pejoratively called:
 Big Requirements Up Front

  8. Understanding Change • Just-in-time is a response to this •

    Context: evolving and agile software development 8
  9. (Source: Godfrey & Tu, 2000) E.G. Linux Kernel 9

  10. 10

  11. Lehman’s ‘Laws’ 1. Continuing change—A software system will become progressively

    less satisfying to its users over time, unless it is continually adapted to meet new needs. 2. Increasing complexity—A software system will become progressively more complex over time, unless explicit work is done to reduce its complexity. 3. Self-regulation—The process of software evolution is self-regulating, with close to normal distribution of the product and process artifacts that are produced. 4. Conservation of organizational stability—The average effective global activity rate on an evolving software system does not change over time; that is, the amount of work that goes into each release is about the same. 5. Conservation of familiarity—The amount of new content in each successive release of a software system tends to stay constant or decrease over time. 6. Continuing growth—The amount of functionality in a software system will increase over time, in order to please its users. 7. Declining quality—A software system will be perceived as declining in quality over time, unless its design is carefully maintained and adapted to new operational constraints. 8. Feedback System—Successfully evolving a software system requires recognition that the development process is a multi-loop, multiagent, multilevel feedback system. 11 Godfrey, M. W. and German, D. M. (2013), On the evolution of Lehman's Laws. J. Softw. Evol. and Proc.. doi: 10.1002/smr.1636
  12. Time Functionality User needs identify requirements first release enhancement phase

    freeze and replace replacement delivered enhancement phase conventional development Lateness Shortfall Inappropriateness Longevity Adaptability (shaded area) (slope of line) Source: Adapted from Davis 1988, pp1453-1455 User Requirements Always Grow 12
  13. 13

  14. Scale And Complexity 14

  15. 15

  16. 16 http://www.youtube.com/watch?v=GiPe1OiKQuk

  17. “Normal” Accidents 17 Perrow, Charles. Normal Accidents: Living with High-Risk

    Technologies New York: Basic Books, 1984.
  18. 18

  19. Cynefin (Kih-Neh-Vin) 19 More: http://cognitive-edge.com/library/more/video/introduction-to-the-cynefin-framework/ Cannot gather upfront Get these

    right! Creativity Don’t really need RE here
  20. “Let’s Build A Filter” 20

  21. 21 Plan Driven Requirements Value Driven Requirements Date Resources Date

    Resources Fixed Estimated source: DSDM via Leffingwell 2011
  22. Just-In-Time RE e.g. specification by example, behavior- driven development (BDD),

    feature driven development, user stories, acceptance testing. lightweight and iterative (?) 22 developers talk to “Product Owner” assume change and react, rather than plan RE is ongoing and continuous
  23. Research Questions 1. How does JIT RE manifest itself in

    practice? 2. What problems might be encountered? 23
  24. Methodology Choose 3 software projects that are successful, relatively distinct

    and open. Study each project’s software process holistically, starting at the task level. Choose a representative requirement for detailed study. 24
  25. Flexible Indexing 25 Inverted index: terms point to containing documents

    Flexible indexing: add frequency, weights, boosts directly to index
  26. 2004: Idea proposed on wiki (Doug Cutting) 2006: Email discussion

    about implementation (Grant Ingersoll) 2008: First JIRA ticket created (Michael McCandless) 2010: Feature branch integrated into trunk (all) 2012: Feature ships as 4.0 Alpha 2009: Feature progress misses release 2.9; code later released for stress testing by others (McCandless) 2009: Unicode incompatibility detected (Robert Muir)
  27. "Have you tried any actual tests swapping these approaches in

    as your terms index impl?” “No – changing something like this requires a lot of coding, so it's better to do thought experiments first to winnow down the options." “ Mike, this change to byte[ ] in TermRef will break backwards compatibility, without some special attention paid to the utf-16 to utf-8 conversion. “ In my opinion it would be better to think in the future how we can improve lucene in the following ways: The term dictionary should be more "DFA-friendly", [etc.] “
  28. Observations (Lucene) • Requirements arise organically. Never nailed down. •

    Strategic vision emergent rather than directed. No hard deadlines. • JIRA is central to RE process. • Detailed knowledge lives inside the developer/requester. 28
  29. Common Practices (Jit In Practice) • Just-in-Time Requirements • Feature-oriented

    • As-needed Traceability • Exploratory and iterative development • Community-mindedness 29
  30. Departures • No big-picture thinking • No separate prioritization •

    Unclear feature provenance • No repeatable elicitation or reuse 30
  31. Understanding R.E. Practices Ad-hoc List Links Models Req Spec Ad-hoc

    Simple Priority Multiple dimensions Reflective Requirements+Management Requirements+Analysis Personal)projects Firefox Lucene CONNECT 31
  32. But ... “I chose RE as a research area [after

    seeing] that insufficient RE causes inconsistent, incomplete and incorrect requirements specifications and is responsible for a significant number of problems encountered in development projects.” – Klaus Pohl, preface to RE textbook 32
  33. Ultimate Empirical Question Amount of RE Software Value Idealized RE

    Perception Reality? Diminishing RE return 33
  34. Related Work • Plenty of industry focus: • Leffingwell, Agile

    Manifesto, Agile BABOK, BDD etc. • Social nature of RE in OSS explored by Walter Scacchi. • Emmanuel Letier exploring requirements prediction. • Marco Lormans and Arie van Deursen worked on requirements coverage. • Niu et al. “Traceability-enabled refactoring for managing just-in-time requirements” • Heck and Zaidman: “Horizontal traceability for just-in-time requirements: the case for open source feature requests” 34
  35. 35 Neil Ernst • @neilernst • neilernst.net Just-In-Time Requirements practices

    are different yet effective when? how? why?