$30 off During Our Annual Pro Sale. View Details »

Ship It: Staying Lean at LivingSocial

Ship It: Staying Lean at LivingSocial

Andrew Atkinson
PRO

April 13, 2013
Tweet

More Decks by Andrew Atkinson

Other Decks in Programming

Transcript

  1. Ship it: Staying
    Lean at
    LivingSocial
    Andy Atkinson

    View Slide

  2. Your projects?
    Actual
    Expected

    View Slide

  3. ...even worse?
    Actual
    Expected

    View Slide

  4. Move quickly
    Focus
    Ruby
    +

    View Slide

  5. Andy Atkinson
    SOFTWARE ENGINEER

    View Slide

  6. LivingSocial is the local marketplace to buy and share
    the best things to do in your city

    View Slide

  7. CONSUMER WEB

    View Slide

  8. View Slide

  9. SHOW PRICES?

    View Slide

  10. Counter-intuitive! Showing
    prices converted worse.

    View Slide

  11. View Slide

  12. BUILD, MEASURE, LEARN

    View Slide

  13. Lean startup
    Try concepts
    Preserve focus
    despite growth

    View Slide

  14. TEAM GROWTH
    2012
    2010
    a/b testing, iterations, start small.
    still works!

    View Slide

  15. SLOW ROLLOUT
    Release process
    Release features to segments of users first.

    View Slide

  16. MOBILE AND NATIVE
    Release process
    Add mobile and native device support eventually.

    View Slide

  17. Engineering process
    Build it Make it pretty Ship it Make it fast
    Make  it  work
    Add  tests
    Finalize  design  and  
    user  experience
    Verify  and  measure  
    usage
    Scaling

    View Slide

  18. Engineering culture
    Optimize for quick recovery from mistakes.
    Learn from mistakes. Share remediations with team.

    View Slide

  19. Test ideas cheaply

    View Slide

  20. PURCHASE FEEDBACK

    View Slide

  21. TWO ASSUMPTIONS
    Consumers would give feedback
    1 2
    1
    2 Merchants would find the feedback valuable
    PURCHASE FEEDBACK

    View Slide

  22. OVERALL: SUCCESS
    Surveyed millions of people. Collected millions of
    records.
    Could have validated product interest with merchant
    using sample and survey data
    PURCHASE FEEDBACK
    Could have validated with consumers before building

    View Slide

  23. Test small changes that could
    have a big impact.

    View Slide

  24. QUICK CHECKOUT

    View Slide

  25. HYPOTHESIS
    Increase purchase conversion
    1
    QUICK CHECKOUT
    $

    View Slide

  26. MISSED TARGET
    Tested on a small set of deals (good)
    Failed to achieve a statistically significant increase in
    purchase conversion.
    QUICK CHECKOUT
    Removed code

    View Slide

  27. FACEPILE

    View Slide

  28. FACEPILE
    WINNER!
    Better subscription conversion from paid clicks
    originating from Facebook ads.
    5%
    Social proof

    View Slide

  29. Automate something after doing
    it manually becomes painful, but
    not before.

    View Slide

  30. WAIT LIST FEATURE

    View Slide

  31. HYPOTHESES
    Additional purchases on deals from additional
    inventory
    1
    WAIT LIST FEATURE
    $
    More inventory allocation from merchant for re-runs
    2

    View Slide

  32. MISSED TARGET
    Learned we typically had the maximum inventory from
    hotels
    Purchase conversion from wait list subscribers was <
    10%.
    WAIT LIST FEATURE
    Learned the majority of deals do not sell out
    Small opportunity,
    removed code.

    View Slide

  33. Automated reporting
    merchant notification
    Did not build:

    View Slide

  34. Test riskiest assumptions first

    View Slide

  35. LOYALTY PRODUCT

    View Slide

  36. Would consumers sign up?
    1 2 Would merchants sign up?
    LOYALTY PRODUCT
    Would integration
    with third-party work?
    Quality of the data?
    3

    View Slide

  37. DID NOT LAUNCH
    Working with third-party added extra time to our
    development speed
    LOYALTY PRODUCT
    Could have validated riskiness assumptions first
    (merchant enrollment)

    View Slide

  38. ESCAPES NEW DEVELOPMENT
    Booking
    Air
    Tours
    Payment
    Mobile

    View Slide

  39. ESCAPES NEW DEVELOPMENT
    Manually loading inventory
    Building piece by piece (air, packaged tours, mobile)
    Deferring merchant payment project while working
    with hotels that don’t require it
    Starting small

    View Slide

  40. CONCLUSION
    Validate riskiest assumptions first
    Use a/b testing for optimization and new product
    development
    Optimize for iteration speed and quick recovery
    Test small ideas with big potential
    Make customer validation your primary goal

    View Slide

  41. Hiring
    Ruby, iOS, Java/Scala, remote is ok

    View Slide

  42. View Slide