Measuring Hard-to-Measure Things (Inactives, Pricing, Collaboration)
This talk covers some of the hard-to-measure things we're studying at GitHub. Stories include: cross-sectional survey project, pricing experiment with attitudinal data, and a Think Aloud with a research sneak attack.
encouraging them to share experiences that depict the why to the what of data. ✴ Qualitative insights often inform how we shape questions for our quantitative studies (surveys, large data set analysis). 23 GitHub
beginning (inception) – When newcomers sign up, poke around, & experiment. It’s harder to ﬁnd them after they leave (rely upon email outreach). 2. The messy-but-sticky middle – When newcomers are regularly active; in GitHub where the workﬂows & workarounds happen (they imprint onto & are imprinted by the product experience). 3. The end – Where newcomers have abandoned the product; GitHub “inactives,” of which a large number are “omg duplicates!” & project-sensitive dormant accounts. GitHub
Private repositories 663 36% GitHub T-shirt 324 17% Merged Pull Request 311 17% Git Training 265 14% GitHub Training 189 10% “Other” 103 6% 64% indicated they would get more value out of something else. 24% wanted practical training in Git or GitHub. 34% reported that publicly consumable goods (e.g. t-shirt, merged PR) would be more valuable. GitHub
needed a feature: branch permissions. ✴ More permissions = more complexity. ✴ Competitor products offered them. ✴ Pressure was on! We wanted to be thoughtful with how we solved the motivation & goals behind the request. GitHub