jugs, do you see them? A 5 gallon and a 3 gallon. Fill one of the jug with exactly 4 gallons of water and place it on the scale and the timer will stop. It must be precise, one ounce of more or less will result in detonation. If you're still alive in 5 minutes, we'll speak.”
world Engineers build software Tight deadlines Yes Yes Wrong information can be disastrous Exploding bombs World domination Cancelled/delayed projects Low quality software Lost data The ending Usually happy end. Sometimes happy end.
Review Test calls resolves opens resolves submits belongs to implements requests comments on submits as belongs to tests uses edits submitted into moves defines defines works with Process Information ships from ships created on built on Schedule Product Test Job Executable Integration Branch Change Source File Procedure / Method Class / Type Review Feature / Defect Person Jacek Czerwonka, Nachiappan Nagappan, Wolfram Schulte, Brendan Murphy: CODEMINE: Building a Software Development Data Analytics Platform at Microsoft. IEEE Software 30(4): 64-71 (2013)
Green dots are branches with high benefit and low cost Red dots are branches with high cost but low benefit Each dot is a branch Christian Bird, Thomas Zimmermann: Assessing the value of branches with what-if analysis. SIGSOFT FSE 2012
Green dots are branches with high benefit and low cost Red dots are branches with high cost but low benefit Each dot is a branch Christian Bird, Thomas Zimmermann: Assessing the value of branches with what-if analysis. SIGSOFT FSE 2012 If high-cost-low-benefit had been removed, changes would each have saved 8.9 days of delay and only introduced 0.04 additional conflicts.
How is video game development different from software development? Emerson R. Murphy-Hill, Thomas Zimmermann, Nachiappan Nagappan. ICSE 2014 Understanding and improving software build teams. Shaun Phillips, Thomas Zimmermann, Christian Bird. ICSE 2014 A field study of refactoring challenges and benefits. Miryung Kim, Thomas Zimmermann, Nachiappan Nagappan. SIGSOFT FSE 2012. Refactoring
Worthwhile How do users typically use my application? 80.0% 99.2% What parts of a software product are most used and/or loved by customers? 72.0% 98.5% How effective are the quality gates we run at checkin? 62.4% 96.6% How can we improve collaboration and sharing between teams? 54.5% 96.4% What are the best key performance indicators (KPIs) for monitoring services? 53.2% 93.6% What is the impact of a code change or requirements change to the project and its tests? 52.1% 94.0% What is the impact of tools on productivity? 50.5% 97.2% How do I avoid reinventing the wheel by sharing and/or searching for code? 50.0% 90.9% What are the common patterns of execution in my application? 48.7% 96.6% How well does test coverage correspond to actual code usage by our customers? 48.7% 92.0% More at http://aka.ms/145Questions
Games Studios, and Turn 10. Thanks to interns Ken Hullett, Sauvik Das, Jeff Huang, Gifford Cheung, Thomas Debeauvais, Erik Harpstead and visiting researchers Tim Menzies and Emerson Murphy-Hill. Xbox Live Influence of games and achievements on (paid) Xbox live memberships Influence of friends on titles played Characterizing players with Xbox Live data Gameplay Impact of social behavior on retention (Beta of a AAA title) Influence of gameplay on skill (Halo Reach) => CHI 2013 Assists in a car racing game (Forza 4) => FDG 2014 How to create a successful initial session in games => CHI Play 2014 Engineering Differences between game and traditional software development => ICSE 2014 Lessons learned from game development (ongoing) Mining software repositories from games (ongoing) Exploratory Personalization with Avatars in Xbox Geographic influence, temporal influence, and structural influence
of player base, sampled randomly 200k players who played 25M races Assist usage Assist transitions Thomas Debeauvais, Thomas Zimmermann, Nachiappan Nagappan, Kevin Carter, Ryan Cooper, Dan Greenawalt, Tyson Solberg: An Empirical Study of Driving Skill in Forza Motorsports 4. FDG 2014
Medium Hard Advanced Expert Stability prevents the car from spinning when cornering too fast ON OFF Traction prevents the car from spinning when accelerating ON OFF Braking supports the player when he/she brakes or should brake Assisted w/ ABS ABS OFF Shifting helps the player in passing gears Automatic w/o clutch Manual w/o clutch Manual w/ clutch Line overlays the optimal trajectory to follow on the track Full Brake OFF Damage determines how much the performance of the car can change during the race Cosmetic Limited Simulation
disabling an assist Factor More likely to keep the assist disabled … Significant for … Number of races Players who disable an assist early All assists Races per day Players who race fewer games a day All assists Rear-wheel drive (race before) Players who drove a car with rear-wheel drive All assists Car Performance Index (race before) Players who drove a car with lower PI All assists Position (race before) Players who finished first All assists but Traction and Clutch Career mode (race before) Players who did not play career mode Autobrake, ABS, Autoshift, Full line, Brake line
my team. Just this week, we’ve had 8 hours of meeting to design our core gameplay loops based directly on your data. Quite literally, we project your data and the player profiles on the wall while we design. Its awesome.” About our work on a different game title.
supports Questions user cares about Inductive Engineering Tim Menzies, Christian Bird, Thomas Zimmermann, Wolfram Schulte, Ekrem Kocaganeli. The Inductive Software Engineering Manifesto: Principles for Industrial Data Mining. MALETS 2011