Booking.com - Speaker Deck

Slide 1

Slide 1 text

App experimentation as a strategy at Booking.com 3 reasons for running experiments on apps Jorden Lentze, September 2022 MOBILE APPS

Slide 2

Slide 2 text

Would it surprise you that 37.6% of all experiments at Booking.com are running on apps?

Slide 3

Slide 3 text

Who has ever run experiments on apps?

Slide 4

Slide 4 text

When I asked on Linkedin 62% Source: survey on Linkedin

Slide 5

Slide 5 text

01 Key takeaways Follow your customers 02 User experience is key for apps 03 Helps execute on your strategy

Slide 6

Slide 6 text

Experimentation at Booking.com 500+ concurrent experiments

Slide 7

Slide 7 text

90% 10% 9/10 experiments fail for users. Source: VWO 2016

Slide 8

Slide 8 text

01 Validate business decisions 02 Learn faster 03 Reduce deployment risk 04 Do not hurt our customers 05 Improve experience Beneﬁts of experimentation

Slide 9

Slide 9 text

Run experiments to add value and not features

Slide 10

Slide 10 text

01 Experimentation is part of our strategy Informs strategy 02 Validates strategic choices 03 Increases experiment quality

Slide 11

Slide 11 text

Source

Slide 12

Slide 12 text

● Test everything ● Anybody can run experiments ● No hippos ● Teams made for experimentation ● 100% access to data What does this mean?

Slide 13

Slide 13 text

And this results in………

Slide 14

Slide 14 text

2011 Start with the development of our ﬁrst app 1 team 2018 Increased focus 25 teams 2022 Key part of our strategy 60+ teams 2005 First experiment at Booking.com

Slide 15

Slide 15 text

1. Follow your customers Run experiments on the most important channels for your customers

Slide 16

Slide 16 text

01 Follow our customers Our customers are on our apps 02 Customers come straight to us 03 They come back to Booking more often using apps Guests love the ease of use of our apps Source: Booking earnings call

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Experimentation measurably helps improve the customer experience 1. Follow your customers 2. User experience is key for apps

Slide 19

Slide 19 text

Challenges for apps ● Smaller screens ● High expectations ● Speed matters

Slide 20

Slide 20 text

Conﬁdential + Proprietary A better experience trumps everything - Jared Spool

Slide 21

Slide 21 text

Small changes can inﬂuence the experience of our customers

Slide 22

Slide 22 text

Small changes can inﬂuence the experience of our customers This icon might be confusing guests?

Slide 23

Slide 23 text

100% bug ﬁx policy

Slide 24

Slide 24 text

Speed matters. We care about app performance

Slide 25

Slide 25 text

The higher up in the funnel, the bigger the impact

Slide 26

Slide 26 text

User experience is key for apps because: - small screens - high expectations - speed matters

Slide 27

Slide 27 text

3. Helps execute on your strategy Informs and validates your decisions

Slide 28

Slide 28 text

Informs strategy

Slide 29

Slide 29 text

Some thoughts to get you started

Slide 30

Slide 30 text