Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Open Design Proposals

abingham
December 17, 2014

Open Design Proposals

This presentation is about Open Design Proposals, pattern for developing designs in the open, harnessing the power of review, consensus, and archives.

This was presented at XP Days Ukraine 2014.

abingham

December 17, 2014
Tweet

More Decks by abingham

Other Decks in Programming

Transcript

  1. @sixty_north Open Decisions in Architectural Evolution A Simple Proposal for

    Improved Decision Making 1 Austin Bingham @austin_bingham Wednesday, December 17, 14
  2. Changes to architecturally relevant aspects of a system, and particularly

    to the form of that architecture. What is architectural evolution? 5 Wednesday, December 17, 14
  3. Architectural changes are far-reaching by definition What complicates architectural evolution?

    8 ‣ Deployment ‣ Licensing ‣ UI ‣ Legacy data compatibility ‣ Performance ‣ Developer education ‣ Third-party integration ‣ ...and much more! Wednesday, December 17, 14
  4. Common elements from successful experiences Key qualities of Open Design

    Proposals 11 ‣ Light enough / heavy enough ‣ Public and open for comment ‣ Standardized formats ‣ Archives ‣ Non-trivial review Wednesday, December 17, 14
  5. 14 Easy to grasp, easy to start, easy to modify,

    easy to move on Self-evidence is central Wednesday, December 17, 14
  6. 19 “We intend PEPs to be the primary mechanisms for

    proposing major new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions." PEP-0001 http://creativecommons.org/licenses/by/3.0/ Tendenci Open Source Developers at Schipul Wednesday, December 17, 14
  7. ‣ Simple text files • Standard format • Version controlled

    ‣ Pre-PEP discussion on mailing list ‣ Rejected PEPs are retained ‣ PEPs are publicly accessible and discussed ‣ Reference implementation often required Details of PEPs 20 Wednesday, December 17, 14
  8. Java Specification Request 22 The international Java community develops and

    evolves Java™ technology specifications using the Java Community Process (JCP.) The JCP produces high- quality specifications using an inclusive, consensus-based approach that produces a Specification, a Reference Implementation (to prove the Specification can be implemented,) and a Technology Compatibility Kit (a suite of tests, tools, and documentation that is used to test implementations for compliance with the Specification.) Experience has shown that the best way to produce a technology specification is to gather a group of industry experts who have a deep understanding of the technology in question and for a strong technical lead work with that group to create a first draft. Agreement on the form and content of the draft is then built using an iterative process that allows an ever-widening audience to review and comment on the document. Java Community Process 2.9 Wednesday, December 17, 14
  9. Details of JSRs 23 ‣ Initiated by community • Initial

    draft written by committee ‣ Much heavier than PEPs • But that's by necessity! ‣ Membership is required • But it's free for individuals Wednesday, December 17, 14
  10. How the biggest thing ever was built Internet Requests for

    Comment (RFCs) 24 Wednesday, December 17, 14
  11. 25 In outline, the process of creating an Internet Standard

    is straightforward: a specification undergoes a period of development and several iterations of review by the Internet community and revision based upon experience, is adopted as a Standard by the appropriate body... and is published. In practice, the process is more complicated, due to (1) the difficulty of creating specifications of high technical quality; (2) the need to consider the interests of all of the affected parties; (3) the importance of establishing widespread community consensus; and (4) the difficulty of evaluating the utility of a particular specification for the Internet community. RFC 2026 Wednesday, December 17, 14
  12. Details of RFCs 26 ‣ Standardized ASCII format ‣ Drafts

    published through well-known shared directory ‣ Discussions happen in various venues • Archives of discussions are required ‣ Established "maturity levels" through which RFCs pass ‣ Archives of all RFCs are maintained Wednesday, December 17, 14
  13. 28 ‣ Proposals developed by teams • Standard file format

    (Word) ‣ Archived on central file server ‣ Mail list discussions ‣ Discussion meetings • Representatives from other groups / areas • Everyone welcome Wednesday, December 17, 14
  14. 29 ‣ Process developed to support large architectural change ‣

    Designed to promote accessibility and adoption ‣ Simple text files / restructured text ‣ Discussions on mailing lists, in meetings, and in review tools Wednesday, December 17, 14
  15. Technologies: ‣ Restructured text / sphinx ‣ mailman list server

    and archive ‣ gerrit Results: ‣ Over 100 detailed, reviewed design documents ‣ Archive of discussions and decisions ‣ Broad understanding of why and how the system was being constructed 30 Wednesday, December 17, 14
  16. Formal reviews / inspections Michael Fagan, 1976, IBM 32 Meetings

    Roles Process Data collection Metrics Wednesday, December 17, 14
  17. Are meetings really necessary for design reviews? Lawrence Votta, 1993,

    Bell Labs 33 Synergy Education Deadline Teams find faults better than individuals Competition Meetings tend to find false-positives Process Less-experienced learn from more-experienced “Education by observation” is not very effective Meetings impose a schedule Deadlines can be imposed without meetings Egos give incentives to contribute/improve Competition can be achieved without meetings “Inspections are part of official process.” Facts, not tradition, should determine process Meetings No Meetings Wednesday, December 17, 14
  18. 34 4% of defects found in meetings Lawrence Votta, 1993,

    Bell Labs Are meetings really necessary for design reviews? Wednesday, December 17, 14
  19. 35 Diane Kelly & Terry Shepard, 2003, RMCC Compare groups

    vs. individuals for code reviews Largely confirmed Votta’s findings. Reading Meeting 1.7 defects/hr. 1.2 defects/hr. Reading is 50% more efficient Wednesday, December 17, 14
  20. 36 Reidar Conradi, 2003, Ericsson Norway/NTNU/Agder Univ. Measure impact of

    reading techniques on UML inspections 75 25 Reading Meeting 20 80 % Time spent % Defects found Wednesday, December 17, 14
  21. 37 Meeting are good for finding false-positives so keep them

    short and small Wednesday, December 17, 14
  22. 38 Frank Blakely et al., 1991, HP Cost-effectiveness of inspections

    vs. testing 21 defects found in inspection 4 would have been found in testing Wednesday, December 17, 14
  23. As reported in “Peer Reviews in Software”, Wiegers Cost savings

    from reviews 39 Hewlett-Packard AT&T Bell Labs Bell Northern Research IBM Error-detection cost reduced by a factor of 10. 10-fold quality improvement. 14% productivity increase. Prevented 33 hours of maintenance per defect discovered. 2-4x speed detection- time improvement versus testing. 1 hour of inspection saved 20 hours of testing and 82 hours of rework (if defect had made it to customers.) Maintenance cost for inspected programs was 1/10th of that for uninspected programs. Imperial Chemical 10:1 ROI, saving $21.4 million per year. 3% project effort in inspections reduced testing defects by 30%. Design and code inspections cut integration effort in half. Litton Data Systems Wednesday, December 17, 14
  24. Finding defects in early phases avoids miswork in later phases

    Upstream inspection is powerful 40 "Bellcore found that 44 percent of all bugs were due to defects in requirements and design reaching the programmers." Tom Gilb, Optimizing Software Inspections Programmers can do whatever you ask them to do! Wednesday, December 17, 14
  25. Feedback and response to reviews may their most important result

    Reviews complete a loop 41 Generate artifact Review artifact Learn Wednesday, December 17, 14
  26. "Research study after research study has shown that inspections can

    detect up to 90% of the errors in a software product before any test cases have been run. And that signifies an extremely effective process." Robert Glass 42 Wednesday, December 17, 14
  27. 43 ...the same studies show that the cost of inspections

    is less than the cost of the testing that would be necessary to find the same errors. What we have here is an effective process that is also cost-effective. And that’s a pretty nice combination. Robert Glass Wednesday, December 17, 14
  28. Get the right input at the right time - early!

    48 Harness the power of peer review Wednesday, December 17, 14
  29. Giving developers an explicit voice in design has big benefits

    Enfranchise and inform developers 52 "Tower of Babel" - M.C. Escher Wednesday, December 17, 14
  30. Accepted and rejected designs should be available for future reference

    Archive of designs 53 Wednesday, December 17, 14
  31. A cheap and easy way to start with Open Design

    Proposals Mail list and a web server 55 Wednesday, December 17, 14
  32. Different approaches work for different organizations Grow the system as

    needed 56 ‣ Review tools ‣ Revision control ‣ Meetings ‣ Document formats ‣ Processes Wednesday, December 17, 14