Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PYROS Build: Metrics

PYROS Build: Metrics

Regan Davis

July 14, 2022
Tweet

More Decks by Regan Davis

Other Decks in Technology

Transcript

  1. metrics
    regan davis
    BUILD | deep dive #3
    PYROS


    creating products that are
    made to measure


    View Slide

  2. regan davis
    product leader from chicago, il
    BACKGROUND
    EIR @ 1871


    Sr. Director of Product @ SpotHero


    Advisor, Coach


    regandavis.com
    VP of Product @ Jellyvision


    VP of Product @ info.com

    View Slide

  3. regandavis.com
    CORE METRICS
    TRACKING PLAN
    OPTIMIZATION
    “DATA DRIVEN”
    agenda

    View Slide

  4. regandavis.com
    what,
    not why
    analytics tell you

    View Slide

  5. regandavis.com
    compare &
    contrast
    data’s purpose:

    View Slide

  6. regandavis.com
    pirate metrics
    (Credit: Dave McClure, Melanie Blake via Medium)

    View Slide

  7. regandavis.com
    ACTIVATION
    ACQUISITION
    RETENTION
    REFERRAL
    REVENUE
    (Credit: Dave McClure, Melanie Blake via Medium)

    View Slide

  8. regandavis.com
    ACQUISITION how do people
    fi
    nd you and
    (eventually) turn into customers?
    how good is their
    fi
    rst customer
    experience?
    ACTIVATION
    (Credit:Melanie Blake via Medium)

    View Slide

  9. regandavis.com
    customer journey
    airport parking customer journey
    ACQUISITION ACTIVATION ACTIVATION
    AWARENESS KNOWLEDGE INTENT

    View Slide

  10. regandavis.com
    Let’s try it.
    Customer journey sheet

    View Slide

  11. regandavis.com
    User/Logo retention: % of new users
    who are still active 3-6 mo. later


    L7/L30 retention: Number of days
    that users are active per week/month
    RETENTION
    REFERRAL Invite rate: % of new users who sent at
    least one invite in the
    fi
    rst X days


    Invite conversion: % of received
    invites that sign-up in the next X days


    Virality factor: % of new users who
    have come from an invite
    (Credit:Melanie Blake via Medium)

    View Slide

  12. regandavis.com
    funnels
    ACQUISITION
    TRACKING PLAN

    View Slide

  13. regandavis.com
    conversion

    View Slide

  14. regandavis.com
    start with
    What’s the main goal
    of your user?
    Look


    Book


    Park
    compare parking options


    reserve and pre-pay for parking


    show up at garage to guaranteed spot

    View Slide

  15. regandavis.com
    next
    What are the paths they take
    to get there?
    • capture an event for each step in the path


    • create a funnel from the steps that are common

    View Slide

  16. regandavis.com
    track
    Events and Properties
    • Screen


    • Trigger


    • Action (Event)


    • Extra info (Properties)


    • Description

    View Slide

  17. regandavis.com

    View Slide

  18. regandavis.com
    Let’s try it.

    View Slide

  19. regandavis.com
    where to start

    View Slide

  20. regandavis.com
    Craigslist seller flow OPTIMIZATION
    Conversion Rate: 0.03%

    View Slide

  21. regandavis.com
    Craigslist seller flow

    View Slide

  22. regandavis.com
    Craigslist seller flow
    20%
    50%
    80%
    95%
    50%
    80%

    View Slide

  23. regandavis.com
    hypothesis driven
    development
    “DATA DRIVEN”

    View Slide

  24. measure your bets.
    regandavis.com

    View Slide

  25. regandavis.com
    “The only difference between
    screwing around and science is
    writing down your results.“
    — Adam Savage, Mythbusters

    View Slide

  26. meet Shannon.
    regandavis.com
    Shannon Rashedi


    Sr. Product Manager, Mobile

    View Slide

  27. View Slide

  28. account holders
    1. higher conversion rates


    2. more frequent customers


    3. better customer service

    View Slide

  29. regandavis.com
    what,
    not why
    analytics tell you

    View Slide

  30. talk to people


    View Slide

  31. View Slide

  32. regandavis.com
    What do we think will happen?
    OPPORTUNITY
    METRIC
    hypothesis
    PROBLEM
    HYPOTHESIS

    View Slide

  33. regandavis.com
    ten
    ideas.
    2 experiments.

    View Slide

  34. regandavis.com
    quick
    rebook

    View Slide

  35. regandavis.com
    We hypothesize that showing Quick
    Rebook to guest users will help
    customers [see the value of an
    account] and increase [account signup
    X%*]
    • Quick Rebook Click-through Rate
    (from home screen)


    • Account sign-up rate
    METRIC
    BET (HYPOTHESIS)
    (*Note: numbers are obfuscated.)
    Quick Rebook

    View Slide

  36. regandavis.com
    cancel
    button

    View Slide

  37. regandavis.com
    We hypothesize that asking for an
    account when users try to cancel will
    … increase [account signup Y%*]
    • Account sign-up rate


    • Customer service calls


    • Cancellation rate
    METRIC
    BET (HYPOTHESIS)
    (*Note: numbers are obfuscated.)
    Quick Rebook

    View Slide

  38. so did it work?

    View Slide

  39. regandavis.com
    quick rebook
    • major boost in signup


    • more use of the app overall

    cancel button
    • it did increase signups…


    • but likely caused more issues
    😱

    View Slide

  40. “Experiment with multiple
    solutions. When one of those
    experiments is a failure, you're
    not starting from scratch again.”
    —Shannon Rashedi, Sr. PM, Mobile
    regandavis.com

    View Slide

  41. put the user at the heart of your ideas


    use analytics to measure if your guess was right


    keep learning — talk to your users often
    focus on who


    measure what


    ask why
    regandavis.com

    View Slide

  42. FAILURE SUCCESS
    regandavis.com

    View Slide

  43. regandavis.com
    “genius” actual genius

    View Slide

  44. regandavis.com

    View Slide

  45. regandavis.com

    View Slide

  46. questions?
    NEXT STEPS
    regandavis.com/feedback


    [email protected]
    regandavis.com
    feedback


    contact

    View Slide