Product Insights for Airbnb

063b6d76e00dc665cb737cdac278c192?s=47 Edwin Chen
February 09, 2015

Product Insights for Airbnb

Evaluating the Airbnb booking experience.

063b6d76e00dc665cb737cdac278c192?s=128

Edwin Chen

February 09, 2015
Tweet

Transcript

  1. A Human Touch Evaluation Insights for Airbnb Edwin Chen @echen

    / echen.me
  2. Overview

  3. Building a Search and Discovery platform is hard. Building it

    for an expensive travel experience, when most people only make a few trips each year, is even harder.
  4. How well does Airbnb do?

  5. Is it easy to find a good listing? How about

    if I want to stay near the Eiffel Tower? What are problems with its “Similar Listings” algorithm? Where else could the experience be improved?
  6. None
  7. Where is Disneyland? Why am I being shown locations so

    far away?
  8. Horrible result. The room is pretty and quaint and the

    price is right. But again, it's just not what I'm looking for. This room is near downtown Madison, it's not a secluded place and there's no skiing. (from a search keyworded on “skiing”)
  9. Click logs in a dynamic marketplace can be sparse and

    difficult to interpret. Did users skip this listing because it was too expensive, or was it just not relevant? Was this search result page bad because of poor ranking or because all the good listings were already taken?
  10. Let’s grab additional data.

  11. Human Evaluation

  12. Tell me about an upcoming trip you’d like to make.

    “We love going to Disneyland and would like to go back sometime this year around the beginning of the summer. Ideally we would go for a week but it depends on price. I would want to go with my husband and kids because it's a really fun and special experience for all of us.”
  13. Now try to find a place to stay on Airbnb.

  14. Rate the first listing Fairly bad result. It looks like

    a nice play to stay but I don't like that it's actually part of another house. I would want to feel like we have total privacy instead of sharing certain areas. There are also an awful lot of rules about when and what you can do. I understand to an extent but seeing it all written out comes across as kind of negative. It's also not really that close to Disneyland.
  15. “It’s also not really that close to Disneyland.” Can we

    show how far listings are from nearby tourist attractions and landmarks? Why are locations that are so far away from Anaheim being shown?
  16. Many travelers are tourists unfamiliar with where they’re traveling. Why

    not make it easier for them?
  17. “There are also an awful lot of rules… I understand

    to an extent but seeing it all written out comes across as kind of negative.”
  18. Other users mention disliking rules as well “I try to

    stay away from hotels. I find that they are overpriced and dirty. I would rather rent a room in a house, if I couldn't find an entire house, than stay in a hotel. You would think that renting a room or house from someone there would be more rules than a hotel but it is not been my experience. Hotels impose silly rules, and I don't mean standard things like not destroying the room and noise, I mean like not allowing you to open windows or keep a door open to the outside. No thanks! I prefer Airbnb by far.” “It looks like a decent place and how when you first step outside you can see the water and it looks like you can go straight down to the ocean. The price is great for the amount of days I am staying as well and it looks a little homey… The seller also does not seem to have any rules that may complicate me in any way.” “From the pictures, I don't think it would be the kind of place that I would necessarily feel at home in… I don't like the fact that shoes have to be removed before entering the house, since I don't like walking around barefoot. Overall, I just don't think that this room would be the right fit for me.”
  19. Users turn to Airbnb for its friendliness and home-like feel.

    Would it help to boost listings with a friendly tone, and penalize listings with long lists of rules?
  20. Rate the second listing Horrible result. The 500 dollar security

    deposit puts me off immediately. We are looking for something affordable and while the nightly rate is okay the security deposit would just put it over the top. I would be worried we wouldn't get it back no matter how good of a shape we left the house. Also they seem particular about how everything is treated and we have two kids so it just wouldn't work well.
  21. Are security deposits a feature in the ranking models? Security

    deposits are a common gripe. New Airbnb users may be especially worried that an unscrupulous host will keep a security deposit without good reason.
  22. Rate the third listing Fairly bad result. I love that

    the price is so low. Plus there is room for my husband and two kids to sleep. However I don't like that you would be staying in an RV in someone's driveway. There isn't much privacy that way. Plus the distance is reasonable to San Diego and LA which means it's not really close enough to either one.
  23. Rate the fourth listing Fantastic result. If I'm understanding correctly,

    you get the entire house which is great. It looks beautiful on the outside and looks like there is a garage where we could hopefully park our van. The security deposit is not too high. The location isn't perfect but it's pretty good because the drive to Disneyland wouldn't be too long.
  24. Rate the fifth listing Pretty good result. The house looks

    nice outside and in. I like that you get the entire house to yourself. The security deposit is high so that makes me hesitant to want to book it. There is plenty of room for my husband and two kids. I like that we would be able to park in the driveway as we will have our own vehicle.
  25. Takeaways • How can we make it easier to gauge

    the distance to nearby attractions? • Should we penalize strict-looking listings with many rules? • Can we add security deposits as a feature in the ranking models?
  26. Search

  27. Note: may be a biased sample, from a set of

    100 raters.
  28. None
  29. How can we make search results better?

  30. None
  31. This listing is a little too cutesy for me. I

    am not a fan of the room in terms of decoration. It looks very clean though, and I wouldn't be too upset if I was "stuck" there. It looks like a really nice space. I would definitely look around a little more to find something that might be more comfortable in terms of style. (from a male user)
  32. This place is more in line with what i am

    looking for. Good decor and furnishings. The place looks modern and very HIP. Has a nice amount of space and nice windows and glass doors. It is almost ideal to what i pictured.
  33. Style matters Male users may not want pink, fluffy rooms.

    Younger users may want hipper, more modern listings. Some places just feel old and dingy. Can we learn the style of a listing, and match it against user demographics?
  34. Horrible result. I'm not entirely impressed with the host's response

    rate/time. It makes me think they would be hard to get in touch with if something were to occur or we needed something. The house itself is nice, but it doesn't look quite as good as other ones that are cheaper. I would pass on this one.
  35. Fairly bad result. This place looks a little bit bland.

    I think it's a little too expensive for what you get as well. I don't really think I would choose this place. It doesn't stand out to me.
  36. Cost is relative Some listings are on the higher end

    of the price range, but others may simply be overpriced. Price is probably already a ranking feature, but should overpricedness be as well? Airbnb already has a price prediction algorithm; can we use the residual as a measure of whether a listing is priced too high or too low?
  37. From the pictures, I don't think it would be the

    kind of place that I would necessarily feel at home in. The late check in also worries me, since I like to get on the road early. I don't like the fact that shoes have to be removed before entering the house, since I don't like walking around barefoot. Overall, I just don't think that this room would be the right fit for me.
  38. The pictures look nice, and it's appealing that it is

    near to the beach. However, this is a room, and I actually wanted a whole space to myself. Also, there are restrictive hours on when to arrive, and that's difficult for me. Plus the price is a bit more than I was hoping to pay for. I don't need bed and breakfast, just a bed.
  39. Check-in and check-out times can be problematic Are restrictive arrival

    and departure times used as a feature?
  40. This is the best one yet. The view from the

    patio is amazing. The price is the cheapest. The amenities are spot on. No need for a television when you have a place like this. We talked about going to Sardinia, and this would be a great reason why, just to spend a night at this place. This place is gorgeous.
  41. It looked acceptable. The location just wasn't as good and

    there weren't spectacular views. If I was going to settle for something that wasn't pretty then it would need to be very convenient, which this isn’t. (from a search whose other results had good views)
  42. Can we figure out from a listing’s description, reviews, and

    pictures whether it’s in a location with great views?
  43. This place looks beautiful! It is a little big, however,

    the price range is not too bad. It has romantic in the title and that is what caught my eye first. I am looking for a romantic vacation with my SO.
  44. The room is not ideal, it's not romantic at all.

    The location is unappealing, it is near an urban area and there is no mention of skiing. I'm looking for a place near a skiing area, I don't have a specific location, because I don't know where they are. The price is okay, it's just not what I want at all.
  45. Many vacations revolve around romance: honeymoons, weddings, anniversaries, and special

    dates. Can we detect which listings and trips are more romantic in mood? What are the other canonical reasons for travel?
  46. Based on all 21 pictures shown, it is an extremely

    attractive place and looks very clean. My girlfriend is particularly picky about living in hotels in regards to hygiene and only wants places that are squeaky clean and have proper mosquito nets. In addition, she wants a bed with enough space for two people and has an air conditioner so that we don't sweat during the nighttime. This location fits the bill for all the conditions, as well as my preferences and looks rather pleasing. (for a trip to Penang)
  47. I really would want something different with a different style,

    as well as a pool, we are really looking for a place with a pool because of the hot sun, in Vegas. It also seems a bit for small we like to have more room that this listing has in it.
  48. Certain amenities are more important, depending on the location and

    time Mosquito nets are important in Malaysia. Listings with air conditioning units are often a necessity in the summer. Is it worth learning these context-dependent preferences?
  49. Similar Listings

  50. “I would like to go to Las Vegas, Nevada. I

    would like to go with my girlfriend to see all the attractions of the city. I would also like to go hiking in the canyons nearby and visit Lake Mead.” Tell me about an upcoming trip you’d like to make.
  51. Fantastic result. This listing appeals to me because of it's

    location close to the action of the city. The price is very appealing because it is cheaper than the nearby hotels. The main reason I want to stay here instead of a hotel is is because this has cooking facilities- because of my diet I have to prepare my own special meals. I also like that the place is not fancy or pretentious, it is very basic and fits my needs well. Find a place you’d love to stay at.
  52. Bad result. The original search was for a whole home,

    so I don't like that I would be sharing with other people. I want my own private bathroom and space. I would be worried about making too much noise and disturbing roommates. I do like the location, though, and the room is quite nice. I would stay here as a last resort option. If this listing were unavailable, would you want to stay at Similar Listing #1 instead?
  53. The original listing was for the Entire Home. Why is

    a Private Room being shown as a similar listing? Adding a filter for the same listing type should be pretty simple.
  54. Pretty good result. I think the place looks nice and

    it is in a nice location. I do not like that it looks like someone is currently living there, with family photos on the wall and things like that. It would make me feel like i'm in someone elses space. The reviews are not very good, there are lots of complaints about smoke and noise and problems with the gate. If this listing were unavailable, would you want to stay at Similar Listing #2 instead?
  55. None
  56. Horrible result. Again, I do not want to stay in

    a place with others, I want my own space. I also liked the original listing because it was close to everything, but this location is too far from the strip and downtown, and I would need to rent a car to get around. I don't like that the home is in a golf community, because I don't like being in areas with stuck up people. I need to be closer to the city and all the fun attractions of Vegas, not in a residential area. If this listing were unavailable, would you want to stay at Similar Listing #3 instead?
  57. Why is a location so far away being suggested? Is

    distance from the original listing a feature?
  58. from a different set of 100 raters.

  59. None
  60. A great deal of Amazon purchases and YouTube watches come

    from Related books and videos. Would bookings similarly increase if Airbnb’s Similar Listings were boosted from their backwater status at the bottom of the page?
  61. New Experiences

  62. Neighborhood Discovery

  63. Travelers are often unfamiliar with the neighborhoods of a city.

    Can Airbnb design an experience that helps them pick which to stay in?
  64. While there's nothing at all wrong with this listing (and

    the price is fantastic), I'd just want to check that there's nothing better. Also, I'd want to research whether or not it's a safe and nice area. I might also decide to expand my search to include more expensive options. This way, I can weigh up whether the benefits of a more expensive option warrant the cost compared to a place like this.
  65. It looks great, and I'd definitely want to contact the

    host so long as I do some research and remain as enthusiastic. The price is reasonable and the apartment looks great. The main thing I would want to check is the area it's in, Lisburn. I don't know how convenient that is compared to the rest of Belfast and the Titanic quarter. If it was convenient, I would probably go for this.
  66. Helpful Reviews

  67. Would showcasing helpful reviews increase bookability?

  68. “This property is stunning! The pictures are beautiful and I

    like the proximity to Dublin. Reading the user reviews, it seems the hosts are gracious and warm, but don't get in your way unless you want them to. I like that. I could just sit in a lawn chair outside this building for the entire vacation and be content.” “It's a really well-decorated and lovely space. I also like that it's in Soho. The price is a little more than we'd like to spend ideally, but for the right place we'd make the sacrifice. Biggest worry I'd have is that this property only has two reviews. I'd be a little scared about committing more money than I would normally on a property with 2 reviews.” “Everything I said about the first rental applies to this one as well. The apartment is very modern and looks very clean (almost clinical). Though I always prefer design that is antique and warm in style, it is almost impossible to find and a clean, modern design is always nice as well. I can't think of much more to say about this apartment except that I notice it is missing half a star in the location category, so I would definitely have to read some of the reviews before I would consider booking it.”
  69. Useless reviews

  70. Helpful reviews

  71. None
  72. None
  73. None
  74. Can we crowdsource training data for building a “helpful reviews”

    classifier? Review-level features (host friendliness, cleanliness, location, etc.) would also be useful in ranking.
  75. Activity-Based Search

  76. Let’s try to book a skiing trip on Airbnb.

  77. “Within the next 2 months, I'd like to go to

    the northern part of my state (WI) to a skiing resort. I want to go for the weekend with my husband. I want to do this because we need a getaway and it would be romantic and we would still have plenty to do.”
  78. Horrible result. The room is not ideal, it’s not romantic

    at all. The location is unappealing, it is near an urban area and there is no mention of skiing. I'm looking for a place near a skiing area, I don't have a specific location, because I don't know where they are, just that we have them. The price is okay, it's just not what I want at all.
  79. Bad keyword matching Can we learn whether a keyword is

    about the host or the listing, and limit matches to the appropriate sections?
  80. Horrible result. The room is pretty and quaint and the

    price is right. But again, it's just not what I'm looking for. This room is near downtown Madison, it's not a secluded place and there's no skiing.
  81. Pretty good result. The house is beautiful and Lake Geneva

    is one of the nicest places in the state. I was looking to go up North is the only reason this isn't a good fit, this is in the southern part of the state. Otherwise it is beautiful and the price is great. It also lists skiing and other winter activities nearby. I would consider it if I couldn't find anything near where I want to go.
  82. “I didn't find anything that really fit what I was

    looking for. I think that had more to do with the search than what is available. If I don't have a specific location in mind, like I wanted a Northern retreat near skiing or winter activities, it's hard to find places. I think being able to pick a general geographical area could be a good thing to add. I think having tags of things to do (skiing didn't really work well as a tag for me) or activities near the places might make it easier as well.”
  83. It’s difficult to book a place when you know what

    you want to be doing, but not where you should be staying. How can Airbnb make this experience easier?
  84. Traveler Types

  85. Why do users travel, and with who? Twitter has news

    junkies and Beliebers. Dropbox has photo takers and collaborators. What are Airbnb’s clusters of travel types, and how can it design a better experience for them?
  86. from a sample of 100 respondents on Google Consumer Surveys

  87. None
  88. Hotels

  89. Many users turn to Airbnb as a replacement for hotels.

    So when do users still prefer to book a hotel and why?
  90. None
  91. 65% of users mentioned a quality control problem with Airbnb

  92. Users have had bad experiences with their hosts and listings

    “We've only ever used Airbnb once, and we had mixed feelings. The pricing was quite reasonable compared to a hotel (it was in Manhattan). On the other hand, the host was not very good. We walked into a very messy place with no clean sheets or towels, dirty dishes, and old food in the fridge.” “It can also be the case that the person's house is not very nice or they are unfriendly. This could be a detriment to the vacation experience. Indeed airbnb's fluctuating quality can be bad.”
  93. They often don’t trust hosts and descriptions as a result

    “Well, it's not so much that it's bad about Airbnb, in and of itself, but there are a few unscrupulous types who misrepresent their properties, and you don't really have any easy time getting reparations, from what I've heard.” “Also, it can sometimes be difficult to trust some of the people listing their places to stay. Its not an insured hotel. Is the unit safe to stay in? Did they wash the sheets? is the room clean? Are they really the owners? They could be scamming you or try to rob you while you are there.”
  94. Reviews and pictures can help, but they’re often not detailed

    enough (and pictures can be misleading too) “Airbnb also can be a crap shoot in terms of quality. Sometimes the physical place is not as good as it was advertised. Although this is usually reflected in the reviews of the place, sometimes they are not accurate or comprehensive enough.” “I wish it had more picture requirements. That way I could see what I was going to see what I got there more clearly. A clearer view of what I am going to be experiencing is it. Someway to make me feel more secure in a way.”
  95. In contrast, hotels are a more trusted brand that provide

    a standard level of quality “Sometimes it's hard to know what to expect with Airbnb. You never really know for certain that it'll be a good experience. The people renting out might be rude or overly anal. It's something that you can't really be 100% sure about. Hotels are almost always guaranteed to have good service. If not you can complain (and it's not like they'll kick you out). With Airbnb there's a greater chance for awkward interactions.” “You're also relying on reviews of others to make a decision to book a stay or not. You can do the same with hotels as well, rely on reviews, but with hotels, there also typically comes a standard of service that you come to expect from certain established hotel names. With renters you can hope to expect what a renter lists, sometimes expectations may not be met and that's a risk you take. Booking a hotel is usually a guarantee.”
  96. How could this improve? Amazon and Yelp allow user-submitted pictures.

    Should Airbnb? Can Airbnb highlight useful reviews? The personal, human interaction makes users more reluctant to leave negative feedback. Would anonymous reviews help?
  97. 45% of users found online booking on Airbnb a hassle

  98. Hotels, without the overhead of dealing with a host, are

    just simpler and faster “Also a hotel can be better in the fact that it can be booked on a moments notice without prior contact. I sometimes just simply want to deal with the simplicity of using a hotel.” “While you can instantly make a booking on airbnb you have to contact the host beforehand, and make arrangements whereas hotels nowadays you're practically guaranteed/hassle free booking from the convenience of an app without much discussion needed between you and the hotel.” “It can be hit or miss, sometimes you really don't want to have to deal with the host. In a hotel you can just check in very fast, and not have to deal with the social aspect of it. I like to have a consistent experience as well, I just want to have a no fuss experience. The booking process is easy on the site, but you still have to discuss things with the host whereas with a hotel you can just show up and check in.”
  99. It’s also difficult to book Airbnb at the last minute

    “I find a hotel more convenient. If you're traveling and don't book ahead you can probably find a hotel and book it easier than trying to find an airbnb listing near where you're traveling last minute.” “Airbnb does not offer a lot of options if you are looking for a short stay. It also doesn’t seem to be as useful if you’re looking for something last minute… I think there should be a section for last minute reservations. Also there should be an alternative available like an insurance just in case things are not as they were advertised. “
  100. It’s especially annoying when you have to contact multiple hosts

    “I found it difficult to book a room on the specific date that I wanted. I was not able to book one place and had to look for another place that was available.” “My host canceled on me and I had to find another place to stay.”
  101. Airbnb has an Instant Booking feature already, but many users

    didn’t even realize it (Some users thought the lightning bolt indicated a hot listing about to be booked, or a flash sale.) Can Airbnb market the feature better to users, and increase adoption among hosts?
  102. 30% of users found the forced human interaction a turnoff

  103. Part of this stems from a worry of disturbing the

    owner “Also, in a hotel there is a sense you can get a little wild. You can order pizza and leave it out on a table, for example, without worrying I might bring bugs into this nice couple's house or make their guest room smell funny.” “I felt weird coming home late from a wedding and fearing that I was disturbing the owner. “
  104. Meeting with the host can be difficult, and especially annoying

    after a long travel “I had to wait a few minutes to meet the landlord. It was cold out and didn't appreciate that. We couldn't reach the landlord, heard different stories, then we were told the key was under the doormat. So checkin was strange. ” “My flight arrived early and the host was at a party, so I felt awkward asking him to leave to meet me and give me the keys. I ended up just going to a movie theatre while I waited for him to come back, even though I wanted to sleep.”
  105. Some people also want to be left alone, without awkward

    interaction “There was some lack of privacy. In a hotel, I feel more anonymous. I felt obligated to talk to the apartment owner a lot.” “I think possible intrusion on the host part as they do have access to the room, apartment, house. It can be messy, and not cleaning properly. Not having standards and someone to complain about the place. Perhaps communication problems with the owner. It frustrates me when I have to call, email, and spend time arranging a place to stay.” “Hotels are almost always guaranteed to have good service. If not you can complain (and it's not like they'll kick you out). With Airbnb there's a greater chance for awkward interactions.”
  106. Companies like Uber grew in large part by removing the

    bothersome part of human interaction (calling a taxi), and making the rest (chatting with your driver) optional. Could Airbnb do the same?
  107. Summary

  108. Growth on Airbnb comes from two ways: 1. Making it

    easy for users to travel more. 2. Getting users to replace hotels or the accommodations they’d use otherwise.
  109. To encourage users to travel more, Airbnb could make the

    entire travel process easier. Where should I vacation if I want to go skiing? What neighborhood should I stay in? What should I do once I’ve arrived at my listing?
  110. I want to go skiing in Wisconsin. Where should I

    stay? This is impossible to figure out right now, and deters me from booking.
  111. I’m going to Paris for the first time. What neighborhood

    should I look under if I want to stay somewhere artsy with great food? This may be a great listing, but is it close to the Eiffel Tower and great restaurants? Is the neighborhood even safe?
  112. To replace existing accomodations, Airbnb needs to make a better

    online and offline booking experience.
  113. I want to spend my trip having fun, not navigating

    directions. Can Airbnb make it easier to find listings close to the attractions I plan to visit?
  114. Part of Airbnb’s appeal is its lower price. So why

    am I worrying about expensive security deposits that may cost more than my stay? Can features like these be incorporated into Airbnb’s discovery models?
  115. I want a host and listing I can trust. Can

    Airbnb surface more helpful reviews?
  116. What other problems does Airbnb need to fix?

  117. What types of users does it need to fix them

    for?
  118. What are the advantages of its competition?

  119. Airbnb knows more about me than a hotel. It knows

    where I’ve searched. I’ve left reviews of what I liked. I’ve booked multiple listings on the platform. I’ve even connected my Facebook account Taking a trip shouldn’t be a stressful chore. Can Airbnb use all this information to give me the ultimate travel experience?
  120. Homes are sweet. Let’s make travel even sweeter.

  121. hello@echen.me