Share
Embed
Shipping Software at Microsoft
Presentation I gave a few years ago at a local conference on some of the principles behind how we shipped WinXP SP2 and Win Server 2003 R2, back in the day.
Mark Harris
December 03, 2009
Transcript
19 slides
-
All sections to appear here Inside the Machine A peek
into Microsoft’s development practices Mark Harris Enterprise QA Manager Grange Insurance -
Microsoft A Software Product Company Core Competency Cultural
Focus “Glory Shelves” “Ship it” awards Insult: “not a shipper” Invest in a lot more D than R -- $9.5B in FY2010 -
Myths vs Reality Myth Reality Program Managers make all the
creative decisions. Program management, dev, and test work together on feature definition. “Product X version 2” is a complete rewrite of version 1 Code is written incrementally, version to version and day to day. Microsoft puts the bugs into the product intentionally. We suck. Formal testing gets put in as an afterthought at the end. Testing starts as soon as core functionality is there – if not sooner. Dev/test/pm don’t have to use the product internally, so it doesn’t see real use until beta 1 The product group “Dogfoods” VERY early. Most of development time is spent writing code. Most of development time is spent fine-tuning and fixing broken functionality. -
Core Feature Team DEV TEST Quality PM Coding, Architecture Debugging
Customer Advocate Quality Measurement Feature Design Requirements Analysis Process -
Decision making There are 3 and only 3 variables
T - Time F – Features F + Q = T Q – Quality Learn to focus the energy - not filter it Change happens - focus on what matters You will ship with bugs The last bug is found when the last customer dies … OR when the software dies (FoxPro RIP) People will always screw up a good project - don’t add them unless it really hurts already -
Transparency of Issues Select key progress indicators (KPI) and
owners for each Owners responsible for driving KPI to acceptable levels and reporting KPI status Everyone has a voice – all issues have merit Standard, well-known venue for issues: Windows Shiproom -
Shiproom: “Can we ship TODAY?” DAILY with key project
leadership Follow-ups from prev meeting (5 min) Reports on key project indicators (20 min) Summarize decisions made & expecations for next meeting (5 min) Expensive meeting: focus, focus, focus Make decisions in the meeting: if you need someone else, call them in Publicize decisions made as broadly as possible: regular status mail, public web site with KPI’s & follow-ups Keep records of commitments and decisions made -
How change happens in Release Phase Trio of PM,
Dev and Test must accept change and get War Team approval Dev makes change and has 2 code code reviews Change released to build via FIS Build confirms FIS is correct and rolls it into next daily build War Room Core driving group during release phase Meets at least once a day - 5 days a week - more towards end Makes all the decisions for the release -
The Quality Criteria Bugs are always there – Perception
is Reality Always fix repro’d crashes, hangs, security breaches and data corruption Fix all the bugs the users will hit Defer some fixes Insure the features work as advertised Dev and Test MUST understand every bug, fix, and potential work around PM MUST understand the long-term impact of every bug, fix, and work-around Long stress runs are required Push the system beyond its limits for an extended time -
Release Criteria Each milestone gated by measurable, actionable release
criteria Functionality Legal Reliability Customer Feedback (OEM, TAP, MSIT) Performance Security Scalability Documentation Performance Compatibility Interoperability Localization -
Criteria Metric Springboard Feature Complete Windows Firewall IE Software Update
Services Active DCR's for product Documentation Complete Book of Xpsp2 complete and posted Release notes authored, edited, and ready to release Help files updated for all springboard features and xpsp2 changes RC deployable in production for all corporate/consumer customers Outstanding reported beta program blockers OTG deployment blockers # clients deployed in OTG TAP deployment blockers TAP customers have completed escrow testing and signed off on release total selfhost votes on release candidate selfhost "Yes" votes percentage No outstanding MSRC or critical/important security issues Known outstanding critical/important security vulnerabilities Outstanding MSRC bulletins against RC Security audit complete and signed off Known active common criteria-blocking issues Release Criteria – XPsp2 Sample -
Early Exposure Selfhost use the product for your
day-to-day operations while you develop it Benefits Exposes problems with the code early (integration issues and simple bugs) Gives team personal incentive to fix problems Incremental, regular beta releases Measure and drive customer satisfaction during development. Prioritize customer bugs. Benefits Early exposure makes for easier fixes (usually) Beta customer joy == paying customer joy -
Selfhost Windows Division runs weekly builds on: Development
& Test Machines Release Servers Spec Servers Bug Db Source Code Control MSIT Select Workloads Process for feedback -
Understanding Realties Listen to your developers & testers
Specifications are never complete Treat them as “guides” Feature complete is achievable “Code complete” is not No feature change requests == no one’s using it Milestone lengths must stay small - 8-12 weeks Minimize the code “churn” Keep bug counts under control -
Characterize the Process Agile? Scrum? XP?
Waterfall? Spiral? TDD? Respect each other Build on success Expect Excellence