Shipping Software at Microsoft

Presentation I gave a few years ago at a local conference on some of the principles behind how we shipped WinXP SP2 and Win Server 2003 R2, back in the day.

Transcript

1. All sections to appear here Inside the Machine A peek into Microsoft’s development practices Mark Harris Enterprise QA Manager Grange Insurance
2. Microsoft A Software Product Company  Core Competency  Cultural Focus  “Glory Shelves”  “Ship it” awards  Insult: “not a shipper”  Invest in a lot more D than R -- $9.5B in FY2010
3. Myths vs Reality Myth Reality Program Managers make all the creative decisions. Program management, dev, and test work together on feature definition. “Product X version 2” is a complete rewrite of version 1 Code is written incrementally, version to version and day to day. Microsoft puts the bugs into the product intentionally. We suck. Formal testing gets put in as an afterthought at the end. Testing starts as soon as core functionality is there – if not sooner. Dev/test/pm don’t have to use the product internally, so it doesn’t see real use until beta 1 The product group “Dogfoods” VERY early. Most of development time is spent writing code. Most of development time is spent fine-tuning and fixing broken functionality.
4. The Daily Cycle Design Code Build Test Review
5. Core Feature Team DEV TEST Quality PM Coding, Architecture Debugging Customer Advocate Quality Measurement Feature Design Requirements Analysis Process
6. Decision making  There are 3 and only 3 variables  T - Time  F – Features F + Q = T  Q – Quality  Learn to focus the energy - not filter it  Change happens - focus on what matters  You will ship with bugs  The last bug is found when the last customer dies  … OR when the software dies (FoxPro RIP)  People will always screw up a good project - don’t add them unless it really hurts already
7. Transparency of Issues  Select key progress indicators (KPI) and owners for each  Owners responsible for driving KPI to acceptable levels and reporting KPI status  Everyone has a voice – all issues have merit  Standard, well-known venue for issues: Windows Shiproom
8. Shiproom: “Can we ship TODAY?”  DAILY with key project leadership  Follow-ups from prev meeting (5 min)  Reports on key project indicators (20 min)  Summarize decisions made & expecations for next meeting (5 min)  Expensive meeting: focus, focus, focus  Make decisions in the meeting: if you need someone else, call them in  Publicize decisions made as broadly as possible: regular status mail, public web site with KPI’s & follow-ups  Keep records of commitments and decisions made
9. Shiproom Agenda
10. How change happens in Release Phase  Trio of PM, Dev and Test must accept change and get War Team approval  Dev makes change and has 2 code code reviews  Change released to build via FIS  Build confirms FIS is correct and rolls it into next daily build  War Room  Core driving group during release phase  Meets at least once a day - 5 days a week - more towards end  Makes all the decisions for the release
11. The Quality Criteria  Bugs are always there – Perception is Reality  Always fix repro’d crashes, hangs, security breaches and data corruption  Fix all the bugs the users will hit  Defer some fixes  Insure the features work as advertised  Dev and Test MUST understand every bug, fix, and potential work around  PM MUST understand the long-term impact of every bug, fix, and work-around  Long stress runs are required  Push the system beyond its limits for an extended time
12. Release Criteria  Each milestone gated by measurable, actionable release criteria Functionality Legal Reliability Customer Feedback (OEM, TAP, MSIT) Performance Security Scalability Documentation Performance Compatibility Interoperability Localization
13. Criteria Metric Springboard Feature Complete Windows Firewall IE Software Update Services Active DCR's for product Documentation Complete Book of Xpsp2 complete and posted Release notes authored, edited, and ready to release Help files updated for all springboard features and xpsp2 changes RC deployable in production for all corporate/consumer customers Outstanding reported beta program blockers OTG deployment blockers # clients deployed in OTG TAP deployment blockers TAP customers have completed escrow testing and signed off on release total selfhost votes on release candidate selfhost "Yes" votes percentage No outstanding MSRC or critical/important security issues Known outstanding critical/important security vulnerabilities Outstanding MSRC bulletins against RC Security audit complete and signed off Known active common criteria-blocking issues Release Criteria – XPsp2 Sample
14. Early Exposure  Selfhost  use the product for your day-to-day operations while you develop it  Benefits  Exposes problems with the code early (integration issues and simple bugs)  Gives team personal incentive to fix problems  Incremental, regular beta releases  Measure and drive customer satisfaction during development. Prioritize customer bugs.  Benefits  Early exposure makes for easier fixes (usually)  Beta customer joy == paying customer joy
15. Daily Builds IDW/IDS Builds RC/Beta Builds RTM User Base Quality
16. Selfhost  Windows Division runs weekly builds on:  Development & Test Machines  Release Servers  Spec Servers  Bug Db  Source Code Control  MSIT  Select Workloads  Process for feedback
17. Understanding Realties  Listen to your developers & testers  Specifications are never complete  Treat them as “guides”  Feature complete is achievable  “Code complete” is not  No feature change requests == no one’s using it  Milestone lengths must stay small - 8-12 weeks  Minimize the code “churn”  Keep bug counts under control
18. Characterize the Process  Agile?  Scrum?  XP?  Waterfall?  Spiral?  TDD? Respect each other Build on success Expect Excellence
19. QUESTIONS?