About CB Hi, I’m Chrissie Brodigan ✴ Live in Sausalito ✴ Trained as a historian ✴ Focus on gender & labor ✴ Competitive ﬁgure skater ✴ Built GitHub’s early UXR practice (2013 - 2016) Twitter: @tenaciouscb
more complete story ✴ Experienced both highly marginalizing & empowering work conditions. ✴ Skilled, professional, & organized workers in their own labor union. ✴ Were part of a process that changed constitutional law.
changed our approach Our team realized we were asking new users to tell us about skills they didn’t have. We changed strategy to ask about skills outside of git and GitHub that new users did have. (Kind of like a census.)
& analyze a single cohort’s data with two types of studies: ✴ Prospective – identify outcomes as they happen in real time. ✴ Retrospective – look back at variables over time and identify how they contributed to known outcomes. We “accidentally on-purpose” designed a longitudinal study
Q. Which VCS are you using? Strong pattern in the yellows & greens, which represent “Nothing” & “SVN.” As programming experience increases people are much more likely to be using another VCS vs. GitHub.
Golden Ticket ✴ Classic controlled experiment, but with a nice twist. ✴ 39,800 eligible candidates between the treatment & control. ✴ Coupons for free private repositories (FPR) to individuals with 1+ year of tenure.
Ticket email ✴ Sent a total of 39,800 emails ✴ “Free private repositories for @name” ✴ “Free for life” ✴ Misunderstandings about the offer ✴ Good email deliverability, but . . . ✴ Overall low redemption rate
exit survey provides us with insight into why people did or did NOT engage in one or both of the ﬁrst two activities (redemption & creation). Understanding attitudes helps inform what levers we can design and pull with experiences to effect change in behaviors. Attitudinal Data
Which would you value the most? Good # % Private repositories 663 36% GitHub T-shirt 324 17% Merged Pull Request 311 17% Git Training 265 14% GitHub Training 189 10% “Other” 103 6% 64% indicated they would get more value out of something else. 24% wanted practical training in Git or GitHub. 34% reported that publicly consumable goods ( t-shirt) would be more valuable.
The Collaboration Study ✴ Customers told us they needed a complex feature: branch permissions. ✴ Competitor products offered branch permissions. ✴ Designing for a large audience, means we need to be thoughtful and deliberate. Solve for human motivations & goals behind feature requests.
Finding the story ✴ Products have new users, tenured users, and inactive users, understanding each experience provides a more complete view. ✴ Researching hard-to-reach places– reading open text responses & listening to humans share their motivations and goals is how you ﬁnd the story. ✴ Research is your ﬂashlight.
Wrapping Up 1. What’s obvious vs. interesting in your data? 2. How can you gather and use attitudinal data to study perception of value? 3. Where does a sneak attack make sense? 4. How will you uncover a more complete story?