How to measure developer love and reach

Video: https://www.youtube.com/watch?v=HmB7gTaKYI8

Two things a developer relations program should understand are ‘How much do our developers like us?’ and ‘How many people are they telling about us?’ The answers shape the growth of our communities, but it’s not obvious how to put numbers to them. In this talk, I’ll share examples of how real companies use data to calculate the love & reach for their communities and use that data to prioritize DevRel efforts. Then I’ll describe how you can start doing the same today to win the hearts and minds of more developers! No expensive BI tools or data science required.

Transcript

1. @dzello · #DevRelCon
   How to measure developer
   Love and Reach with the
   Orbit Model
   @dzello · #DevRelCon
   Josh Dzielak · DevRelCon Earth · July 7, 2020
   

   View Slide

2. @dzello · #DevRelCon
   I’m Josh
   Developer advocate
   Co-founder & CTO at Orbit
   He/him
   Formerly
   

   View Slide

3. @dzello · #DevRelCon
   ● Product based on the model
   ● Community CRM for developers
   ● Early access: orbit.love
   

   View Slide

4. @dzello · #DevRelCon
   Orbit Model backstory
   November 2014
   Keen IO
   2016
   Algolia
   

   View Slide

5. @dzello · #DevRelCon
   Orbit Model backstory
   2018
   DeveloperMode
   2019 - Orbit
   

   View Slide

6. @dzello · #DevRelCon
   How do I measure my community's
   engagement and growth?
   
   What contribution should I ask each
   community member to make?
   
   Which members should I prioritize
   spending time with?
   
   How much value are we creating for the
   community?
   
   

   View Slide

7. @dzello · #DevRelCon
   Today’s agenda
   ● Orbit Model overview
   ● Love: Measuring engagement
   ● Reach: Measuring influence
   ● Discussion! (throughout)
   

   View Slide

8. @dzello · #DevRelCon
   Orbit Model overview
   

   View Slide

9. @dzello · #DevRelCon
   What is the Orbit Model?
   Space-themed mental model and visual canvas for
   building communities, born out of working with developers.
   A)
   A metaphor for community, in the way the funnel is a
   metaphor for marketing and sales.
   B)
   

   View Slide

10. @dzello · #DevRelCon
    What is the Orbit Model?
    An open source project that you can contribute ideas,
    documentation, and experiences to.
    D)
    Vocabulary and set of metrics for measuring the
    impact and ROI of community.
    C)
    An algorithm for doing a bitwise OR operation
    E)
    

    View Slide

11. @dzello · #DevRelCon
    What is the Orbit Model?
    D. An open source project
    C. Vocabulary and set of metrics
    ✅ A-D are correct!
    A. Mental model
    B. Metaphor for community
    

    View Slide

12. @dzello · #DevRelCon
    Model objectives
    Member experience
    ⚡ Faster community activation
    Smoother onboarding
    Higher retention
    Team impact
    Metrics and reports
    Better team organization
    ⚙ Scalable programs
    

    View Slide

13. @dzello · #DevRelCon
    

    View Slide

14. @dzello · #DevRelCon
    

    View Slide

15. @dzello · #DevRelCon
    

    View Slide

16. @dzello · #DevRelCon
    

    View Slide

17. @dzello · #DevRelCon
    

    View Slide

18. @dzello · #DevRelCon
    Design and offer mutually beneficial opportunities to members
    that increase their love and reach.
    Can we
    hire you?
    Can we include your blog
    post in our newsletter?
    Do you want mentorship
    or training?
    Would you like some swag?
    Want to give a talk at our
    next event?
    

    View Slide

19. @dzello · #DevRelCon
    Love: measuring engagement
    “Love is the currency that drives community.”
    —Erin Frey, Orbit community member
    

    View Slide

20. @dzello · #DevRelCon
    ”Love is a member's level of engagement and investment
    in the community. Someone with high love is highly active
    and plays key roles in the community like contributing,
    moderating, and organizing.”
    —orbit-love/orbit-model on GitHub
    

    View Slide

21. @dzello · #DevRelCon
    5 Steps to
    Measuring
    Love
    1. Define your activities
    2. Give each activity a score
    3. Record when activities happen
    4. Assign love + orbit level
    5. Generate reports and discuss
    

    View Slide

22. @dzello · #DevRelCon
    1. Define your activities
    What activities matter in your community?
    Activities are realized opportunities
    

    View Slide

23. @dzello · #DevRelCon
    Need
    activities?
    The Orbit Model Airtable
    Templates has you covered
    http://template.orbit.love
    

    View Slide

24. @dzello · #DevRelCon
    2. Assign scores
    Give each activity a score from 1-10 based on how much
    engagement & responsibility it indicates.
    +1 point:
    Joined mailing list
    +10 points:
    Gave user conference keynote
    +6 points:
    ‍♂ Successful support interaction
    

    View Slide

25. @dzello · #DevRelCon
    3. Record activities
    

    View Slide

26. @dzello · #DevRelCon
    4. Assign love
    Easy way to calculate
    love: take the highest
    activity score in the
    last 6 months.
    

    View Slide

27. @dzello · #DevRelCon
    4. Assign orbit level
    Choose orbit level
    base on love + your
    other knowledge of
    the member.
    

    View Slide

28. @dzello · #DevRelCon
    5. Generate reports
    

    View Slide

29. @dzello · #DevRelCon
    5. Discuss
    Member experience
    Who are our highest-love developers?
    Who is ramping up? Who’s gone dormant?
    Do we have activities in place for members
    with each degree of love?
    Team impact
    How is love changing over time?
    What changes can we attribute to our
    work as a DevRel team?
    How does love compare across locations
    or customer segments?
    

    View Slide

30. @dzello · #DevRelCon
    Reach: understanding influence
    ”Reach is a measure of a community member’s sphere of influence and
    takes into account their reputation, credibility, and degree of
    connectedness.”
    —orbit-love/orbit-model on GitHub
    

    View Slide

31. @dzello · #DevRelCon
    4 Steps to
    Measuring
    Reach
    1. Define your signals
    2. Give each signal a value
    3. Record value for each member
    4. Discuss
    

    View Slide

32. @dzello · #DevRelCon
    1. Define signals
    ● Social media followers and engagement
    ● Stars and followers on GitHub
    ● Reputation scores on communities like StackOverflow
    ● Title and position inside their company
    ● Pagerank or estimated traffic to their website or blog
    

    View Slide

33. @dzello · #DevRelCon
    2. Give signals a value
    

    View Slide

34. @dzello · #DevRelCon
    3. Record member values
    

    View Slide

35. @dzello · #DevRelCon
    4. Discuss
    Member experience
    Who are our highest-reach developers?
    Do we have programs in place for members
    with each degree of reach?
    What are our member’s goals when it comes
    to increasing their reach?
    Team impact
    How is reach changing over time?
    What changes can we attribute to our
    work as a DevRel team?
    How is reach helping drive awareness?
    Referrals? Brand building?
    

    View Slide

36. @dzello · #DevRelCon
    Thank you DevRelCon!
    Say hello: @dzello
    Learn more: https://orbit.love/
    

    View Slide