Slide 1

Slide 1 text

Ship it: Staying Lean at LivingSocial Andy Atkinson

Slide 2

Slide 2 text

Your projects? Actual Expected

Slide 3

Slide 3 text

...even worse? Actual Expected

Slide 4

Slide 4 text

Move quickly Focus Ruby +

Slide 5

Slide 5 text

Andy Atkinson SOFTWARE ENGINEER

Slide 6

Slide 6 text

LivingSocial is the local marketplace to buy and share the best things to do in your city

Slide 7

Slide 7 text

CONSUMER WEB

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

SHOW PRICES?

Slide 10

Slide 10 text

Counter-intuitive! Showing prices converted worse.

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

BUILD, MEASURE, LEARN

Slide 13

Slide 13 text

Lean startup Try concepts Preserve focus despite growth

Slide 14

Slide 14 text

TEAM GROWTH 2012 2010 a/b testing, iterations, start small. still works!

Slide 15

Slide 15 text

SLOW ROLLOUT Release process Release features to segments of users first.

Slide 16

Slide 16 text

MOBILE AND NATIVE Release process Add mobile and native device support eventually.

Slide 17

Slide 17 text

Engineering process Build it Make it pretty Ship it Make it fast Make  it  work Add  tests Finalize  design  and   user  experience Verify  and  measure   usage Scaling

Slide 18

Slide 18 text

Engineering culture Optimize for quick recovery from mistakes. Learn from mistakes. Share remediations with team.

Slide 19

Slide 19 text

Test ideas cheaply

Slide 20

Slide 20 text

PURCHASE FEEDBACK

Slide 21

Slide 21 text

TWO ASSUMPTIONS Consumers would give feedback 1 2 1 2 Merchants would find the feedback valuable PURCHASE FEEDBACK

Slide 22

Slide 22 text

OVERALL: SUCCESS Surveyed millions of people. Collected millions of records. Could have validated product interest with merchant using sample and survey data PURCHASE FEEDBACK Could have validated with consumers before building

Slide 23

Slide 23 text

Test small changes that could have a big impact.

Slide 24

Slide 24 text

QUICK CHECKOUT

Slide 25

Slide 25 text

HYPOTHESIS Increase purchase conversion 1 QUICK CHECKOUT $

Slide 26

Slide 26 text

MISSED TARGET Tested on a small set of deals (good) Failed to achieve a statistically significant increase in purchase conversion. QUICK CHECKOUT Removed code

Slide 27

Slide 27 text

FACEPILE

Slide 28

Slide 28 text

FACEPILE WINNER! Better subscription conversion from paid clicks originating from Facebook ads. 5% Social proof

Slide 29

Slide 29 text

Automate something after doing it manually becomes painful, but not before.

Slide 30

Slide 30 text

WAIT LIST FEATURE

Slide 31

Slide 31 text

HYPOTHESES Additional purchases on deals from additional inventory 1 WAIT LIST FEATURE $ More inventory allocation from merchant for re-runs 2

Slide 32

Slide 32 text

MISSED TARGET Learned we typically had the maximum inventory from hotels Purchase conversion from wait list subscribers was < 10%. WAIT LIST FEATURE Learned the majority of deals do not sell out Small opportunity, removed code.

Slide 33

Slide 33 text

Automated reporting merchant notification Did not build:

Slide 34

Slide 34 text

Test riskiest assumptions first

Slide 35

Slide 35 text

LOYALTY PRODUCT

Slide 36

Slide 36 text

Would consumers sign up? 1 2 Would merchants sign up? LOYALTY PRODUCT Would integration with third-party work? Quality of the data? 3

Slide 37

Slide 37 text

DID NOT LAUNCH Working with third-party added extra time to our development speed LOYALTY PRODUCT Could have validated riskiness assumptions first (merchant enrollment)

Slide 38

Slide 38 text

ESCAPES NEW DEVELOPMENT Booking Air Tours Payment Mobile

Slide 39

Slide 39 text

ESCAPES NEW DEVELOPMENT Manually loading inventory Building piece by piece (air, packaged tours, mobile) Deferring merchant payment project while working with hotels that don’t require it Starting small

Slide 40

Slide 40 text

CONCLUSION Validate riskiest assumptions first Use a/b testing for optimization and new product development Optimize for iteration speed and quick recovery Test small ideas with big potential Make customer validation your primary goal

Slide 41

Slide 41 text

Hiring Ruby, iOS, Java/Scala, remote is ok

Slide 42

Slide 42 text