Ship It: Staying Lean at LivingSocial

Transcript

1. Ship it: Staying
   Lean at
   LivingSocial
   Andy Atkinson
   

   View Slide

2. Your projects?
   Actual
   Expected
   

   View Slide

3. ...even worse?
   Actual
   Expected
   

   View Slide

4. Move quickly
   Focus
   Ruby
   +
   

   View Slide

5. Andy Atkinson
   SOFTWARE ENGINEER
   

   View Slide

6. LivingSocial is the local marketplace to buy and share
   the best things to do in your city
   

   View Slide

7. CONSUMER WEB
   

   View Slide

8. View Slide

9. SHOW PRICES?
   

   View Slide

10. Counter-intuitive! Showing
    prices converted worse.
    

    View Slide

11. View Slide

12. BUILD, MEASURE, LEARN
    

    View Slide

13. Lean startup
    Try concepts
    Preserve focus
    despite growth
    

    View Slide

14. TEAM GROWTH
    2012
    2010
    a/b testing, iterations, start small.
    still works!
    

    View Slide

15. SLOW ROLLOUT
    Release process
    Release features to segments of users first.
    

    View Slide

16. MOBILE AND NATIVE
    Release process
    Add mobile and native device support eventually.
    

    View Slide

17. Engineering process
    Build it Make it pretty Ship it Make it fast
    Make  it  work
    Add  tests
    Finalize  design  and  
    user  experience
    Verify  and  measure  
    usage
    Scaling
    

    View Slide

18. Engineering culture
    Optimize for quick recovery from mistakes.
    Learn from mistakes. Share remediations with team.
    

    View Slide

19. Test ideas cheaply
    

    View Slide

20. PURCHASE FEEDBACK
    

    View Slide

21. TWO ASSUMPTIONS
    Consumers would give feedback
    1 2
    1
    2 Merchants would find the feedback valuable
    PURCHASE FEEDBACK
    

    View Slide

22. OVERALL: SUCCESS
    Surveyed millions of people. Collected millions of
    records.
    Could have validated product interest with merchant
    using sample and survey data
    PURCHASE FEEDBACK
    Could have validated with consumers before building
    

    View Slide

23. Test small changes that could
    have a big impact.
    

    View Slide

24. QUICK CHECKOUT
    

    View Slide

25. HYPOTHESIS
    Increase purchase conversion
    1
    QUICK CHECKOUT
    $
    

    View Slide

26. MISSED TARGET
    Tested on a small set of deals (good)
    Failed to achieve a statistically significant increase in
    purchase conversion.
    QUICK CHECKOUT
    Removed code
    

    View Slide

27. FACEPILE
    

    View Slide

28. FACEPILE
    WINNER!
    Better subscription conversion from paid clicks
    originating from Facebook ads.
    5%
    Social proof
    

    View Slide

29. Automate something after doing
    it manually becomes painful, but
    not before.
    

    View Slide

30. WAIT LIST FEATURE
    

    View Slide

31. HYPOTHESES
    Additional purchases on deals from additional
    inventory
    1
    WAIT LIST FEATURE
    $
    More inventory allocation from merchant for re-runs
    2
    

    View Slide

32. MISSED TARGET
    Learned we typically had the maximum inventory from
    hotels
    Purchase conversion from wait list subscribers was <
    10%.
    WAIT LIST FEATURE
    Learned the majority of deals do not sell out
    Small opportunity,
    removed code.
    

    View Slide

33. Automated reporting
    merchant notification
    Did not build:
    

    View Slide

34. Test riskiest assumptions first
    

    View Slide

35. LOYALTY PRODUCT
    

    View Slide

36. Would consumers sign up?
    1 2 Would merchants sign up?
    LOYALTY PRODUCT
    Would integration
    with third-party work?
    Quality of the data?
    3
    

    View Slide

37. DID NOT LAUNCH
    Working with third-party added extra time to our
    development speed
    LOYALTY PRODUCT
    Could have validated riskiness assumptions first
    (merchant enrollment)
    

    View Slide

38. ESCAPES NEW DEVELOPMENT
    Booking
    Air
    Tours
    Payment
    Mobile
    

    View Slide

39. ESCAPES NEW DEVELOPMENT
    Manually loading inventory
    Building piece by piece (air, packaged tours, mobile)
    Deferring merchant payment project while working
    with hotels that don’t require it
    Starting small
    

    View Slide

40. CONCLUSION
    Validate riskiest assumptions first
    Use a/b testing for optimization and new product
    development
    Optimize for iteration speed and quick recovery
    Test small ideas with big potential
    Make customer validation your primary goal
    

    View Slide

41. Hiring
    Ruby, iOS, Java/Scala, remote is ok
    

    View Slide

42. Thanks!
    andy@webandy.com
    webandy.com
    andy.atkinson@livingsocial.com
    

    View Slide