Future Software Requirements for S2I2 Proposal for High Energy Physics

Presented at a brainstorming workshop on creating a center for computing in high energy physics. I discussed some key criteria to consider in gathering requirements in this space. See https://indico.cern.ch/event/575443/

Transcript

1. © 2016 Carnegie Mellon University Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 [Distribution Statement A] This material has been approved for public release and unlimited distribution. Software Requirements for S2I2 - HEP Neil Ernst Senior Researcher
2. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 2 Copyright 2016 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. [Distribution Statement A] This material has been approved for public release and unlimited distribution. Please see Copyright notice for non-US Government use and distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu. Architecture Tradeoff Analysis Method® and ATAM® are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. TSPSM DM-0004278
3. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 3 • A US federally funded research and development centre (FFRDC) • (see also JPL, RAND, Los Alamos) • Sponsored by Department of Defense and hosted by Carnegie Mellon University in Pittsburgh • Created in 1985 to respond to DoD software crisis • Exemplar work: • Capability Maturity Model (CMM) (spun off) • Early work in s/w architecture (Bass, Kazman, Clements) • Ultra large scale systems report (Northrop 06) • Team Software Process (Watts Humphrey) • Software Product Lines • CERT (cybersecruity emergency response) first ‘CERT’ in US What is the Software Engineering Institute
4. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 4 Traditional requirements problem: find software components that together with domain constraints, satisfy the requirements. Now: plan to satisfy requirements we may not know we have. The Requirements Evolution Problem
5. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 5
6. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 6 Time Capabilities Tech Capabilities s LHC Capabilities Software System Capabilities
7. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. Time Functionality User needs identify requirements first release enhancement phase freeze and replace replacement delivered enhancement phase conventional development Lateness Shortfall Inappropriateness Longevity Adaptability (shaded area) (slope of line) Source: Adapted from Davis 1988, pp1453-1455 7 User requirements always grow
8. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 8 • The S2I2 site says big science of the future will be dealing with “petabytes of data every day”. Currently LHC stores ~100 PB per year of raw data (discarding much more). • “100 times more data” than today • Raise your hand if you are willing to bet that the actual amount processed is within an order of magnitude of that. • Like any forecast, we must base it on our priors (and some well- grounded reasoning, and budget) • Not that we can’t guess well, but that our guesses necessarily have wide uncertainty. Exercise
9. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 9 Architecture Practices Agile architecting, TSP + ACE, & Architecture Evolution Workshop Conformance Review Assurance and argumentation AADL & Views and Beyond Attribute-Driven Design & Architecture Improvement Workshop Quality Attribute Workshop &
 Mission Thread Workshop Architecture Tradeoff Analysis Method (ATAM), System ATAM, SoS Architecture Evaluation, Active Reviews for Intermediate Design, & Virtual Upgrade Validation BUSINESS 
 AND MISSION GOALS ARCHITECTURE SYSTEM
10. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 10 “Normal” accidents Perrow, Charles. Normal Accidents: Living with High-Risk Technologies New York: Basic Books, 1984.
11. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 11
12. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 12 http://www.youtube.com/watch?v=GiPe1OiKQuk
13. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 13 More: http://cognitive-edge.com/library/more/video/introduction-to-the-cynefin-framework/ Cynefin (Kih-Neh-Vin) Cannot gather upfront Get these right! Creativity Don’t really need RE here
14. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 14 Ultimate Empirical Question Amount of RE Software Value Idealized RE Perception Reality? Diminishing RE return
15. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 15 • When system go-live happens in 10 years, you WILL be running on outdated software and hardware • Tech cycle is always faster than system dev. cycle (exceptions: FB, Twitter, Amazon) • Many organizations refuse to believe this! (we are not all FB) Fundamental Constraint
16. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 16 • Modularize for evolution. • Tradeoff: integration risk. • Modularize for release. • Tradeoff: duplication • Abstract for architecture and for release. • Tradeoff: leaky abstractions • Defer decisions until Last Responsible Moment. Iterate and Prototype. • Tradeoff: schedule impact, duplicated work • Evaluate architecture approach regularly with business goal scenarios. • Tradeoff: cost, process buy-in. Future-proofing approaches
17. Software Requirements for S2I2 - HEP Dec 8 2016 © 2015 Carnegie Mellon University D i s t r i b u t i o n S t a t e m e n t A : A p p r o v e d f o r P u b l i c R e l e a s e ; D i s t r i b u t i o n i s U n l i m i t e d 17 Traditional Plan Driven Requirements Iterative & Value Driven Requirements Date Resources Date Resources Fixed Estimated source: DSDM via Leffingwell 2011 @neilernst – nernst@sei.cmu.edu
18. Software Requirements for S2I2 - HEP Dec 8 2016 © 2015 Carnegie Mellon University D i s t r i b u t i o n S t a t e m e n t A : A p p r o v e d f o r P u b l i c R e l e a s e ; D i s t r i b u t i o n i s U n l i m i t e d 18 @neilernst – nernst@sei.cmu.edu Thanks!
19. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 19 Example of performance scenario Users initiate 1,000 transactions per minute stochastically under normal operations and these transactions are processed with an average latency of two seconds. Gather Requirements: Source of stimulus • external event • event triggered by system under consideration Environment • normal mode • overload mode Response • processes stimuli • changes level of service Stimulus • periodic • stochastic • sporadic Artifact • system Response measures • latency • deadline • throughput • …
20. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 20 Scrutinize Requirements: Utility Tree - 2
21. Software Requirements for S2I2 - HEP Dec 8, 2016 © 2015 Carnegie Mellon University [Distribution Statement A] This material has been approved for public release and unlimited distribution. 21 Quality attribute utility trees provide a mechanism for translating the business drivers of a system into concrete quality attribute scenarios. A utility tree shows the prioritization of quality attribute requirements, realized as scenarios. The utility tree serves to make concrete the quality attribute requirements, forcing architect and customer representatives to define relevant quality attributes precisely. Those scenarios rated high in importance and high in difficulty provide the most critical context against which the architecture can be analyzed. These scenarios are candidates for the ASRs. Scrutinize Requirements: Utility Tree - 1