Barcelona Adventurer Usability Testing Results

Barcelona Adventurer is a mobile exploration concept and user-tested prototype that encourages the user to explore Barcelona using gamification. The basic premise of the concept is that the user would wander around Barcelona with a gamified map looking for QR codes, which would give them more specific information about the area. Once the user unlocked certain areas of Barcelona, they would win achievements, which could be shared over social media sites. Our team led users to test Barcelona Adventurer by using timed performance testing on a number of tasks, such as setting up an account and "unlocking" a location using QR code scanning.

Transcript

1. Part 4: Barcelona Adventurer Team Torro: Jamie Crabb, Jessica Crabb, Tim Swihart II, Kaitlin Powell Evaluation Techniques Techniques We used thinking out loud and interview questions along with time calculations and observation to evaluate our interface. Our process had three steps: 1) Recording how long it takes for the user to complete a task for the first time, 2) Having the user repeat the task with time for them to think aloud and 3) Asking efficient interview questions that informed our design. We recorded a Skype conversation than ran while they tested our prototype. We collected their audio reactions to our interface and their answers to our interview questions. While one team member ran the study and managed the recording, another observed, took notes and recorded times. Why? We chose these techniques so that we could easily record first reactions to our interface and refer back to direct quotes and times when considering change. By regulating the dialogue, we can turn quotes into almost qualitative data, which is easier to organize and quantify. Recording users’ voices creates a slightly more professional atmosphere in which testers feel their opinions are important. Though timing might create a rushed feeling in the beginning, it is important to test how quickly users can complete tasks based on our original goal times. Users Our users were volunteer Georgia Tech students. Georgia Tech students living in Barcelona fill in one of our personas: Jamie the student. “Jamie is a 19 year old girl travelling in Barcelona on a college study abroad. She wants to have fun with the other students she met on the plane from the United States, while getting to know the city she’ll be living in. She wants to know more information about specific areas of the city, and wouldn’t mind exploring the city without paying for a guided tour. She goes online multiple times a day for school assignments, to check her email and online calendar, to watch television or movies, and to update her Facebook with details of her trip. She owns an iPhone and considers herself an expert user. She’s seen QR Codes all around her college campus on flyers, t-shirts, and ads, and she has an app that reads them downloaded on her phone.” Though they reflected a small demographic, because they were our first test subjects, they were able to quickly discern where our website succeeded and stumbled.
2. Tasks Create an account To interact with our website, users must create an account. The process could be expedited by using currently logged-into accounts such as Facebook, Twitter or their Google account. We also can provide them with a secure personal account on our server that takes in basic username, password and optional picture. Users can complete this process by making use of a gian “Sign Up Here” button or smaller “Join” link, then fill out a standard form and click a submit button. Log-in and view a locked location on a map The “Sign In” link is small and located in the standard top-right location. They progress through a standard form again. The map should make it easy for the user to understand where in Barcelona they have unlocked and where they have not. They should be able to easily find out basic information and view a picture of that location from the map’s interface by clicking on a lock. Unlock a location The user travels to a location on our map. When they have found a poster outfitted with a QR Code, they will be able to take scan that picture and learn more information or save the QR Code for later to learn more and Tweet about it after they have wandered around on their own. Users must scan a code with a smart phone and then view the unlock screen. View an unlocked location’s information page Once a QR Code is scanned and location will “unlock” many new features including any applicable achievements, a link to Google maps, comprehensive information about that location, travel tips and nearby places of interest. Access to these features can be accessed by a new icon replacing the lock on the map. Share achievement with social media Once a user unlocks their first location or a more advanced series of locations, they will unlock achievements. If the user desires, these achievements can be published to Twitter through our application. To Tweet about their badge, users must navigate to the footer bar on each page and click the Tweet button below the most recently unlocked badge.
3. Evaluation Results Timing Results Task 1: Create an account It took users more time than we originally thought to create an account. Generally, users took between 1 and 20 seconds longer than the novice goal. Task 2: Log-in and view locked location on a map Most users finished right at the expert goal for this task. One user was directly in between the novice and expert goals and one user took around 20 seconds above the average. Task 3: Unlock a location 83% of users unlocked the location before the expert goal time. Task 4: View an unlocked location’s information page Every user finished before the expert goal time. Task 5: Share achievement with social media This task yielded the most varied results. When users had previously seen the Tweet button on the footer, they finished well below the expert line. Two users searched through the website looking for the Tweet function. 0 20 40 60 80 100 120 140 Task 1 Task 2 Task 3 Task 4 Task 5 Expert Goal Novice Goal User 1 User 2 User 3 User 4 User 5 User 6 Time (Seconds)
4. Selected Think Aloud Results Task 1: Create an account “I look at the top-right when signing into accounts.” “Those buttons aren’t button-y.” Task 2: Log-in and view locked location on a map “Fairly standard, once again.” “An existing user has to go to this tiny little link.” “It says to click on the map, but obviously I can’t click on the map.” “It provides good information.” Task 3: Unlock a location “I am biased against QR Codes.” “It’s clean and easy to understand.” Task 4: View an unlocked location’s information page “This is nice that it gives you more data when you unlock it, but I recommend that when you unlock it, it gives you a bit of info, click on the badge to see more.” Task 5: Share achievement with social media “I expected to the Facebook, Twitter and Google Plus.” “The info bar down at the bottom is nice and it has what you need.”
5. Selected Interview Results We asked our user testers three questions: What was the easiest task? The hardest? Why? “Hitting the Tweet button took me one click.” “Finding the achievements was the hardest task. I am still not sure where they are,” “It’s really easy to see the Tweet button. It’s on every page.” “I think that signing was harder than it should have been.” “Signing in. It’s almost like muscle memory for registering for something.” “Scanning QR Codes was the hardest. I am not a fan of the scan.” What is your opinion of the graphic design elements of our website? “Pretty.” “Very, pretty. Fabulous.” “They look really good.” “It’s neat, very clean.” “It’s so colorful.” “I like the interface in general. I have a problem with the font on the navigation bar. It’s upsetting.” Is there anything you would like to see changed? “If you are going to have a requirement for something, you need to put it on the website.” “Make the signing in stuff a little bit bigger.” “I don’t like that sign up button here when I am already signed in.” “I didn’t bother reading that. I thought it was a part of the logo.” “I think over all its fine. Those tabs really bother me.” Easiest Task Task 1 Task 2 Task 3 Task 5 Hardest Task Task 1 Task 2 Task 5
6. Results Interpreted Implications After users created accounts, their default log-in location was the home screen. The large informational slides on the home screen became stumbling blocks when users tried to find ways to execute tasks. Users searched over the page when looking for a link to the map and sometimes when they tried to tweet. The footer was easy to navigate to for many users. The visual dialogue of the map –locks and icons- was understood by all, though some requested that the map be more interactive with roll-over image changes. The changes in status on the footer bar were one of the most dynamic parts of our design, though the suggestion for the user’s next adventure did not change. The layout of our website is standard and generally easy for users to navigate around, especially on form pages and the map page. One main critique of the usefulness of our website was the lack of visual relationship between achievements and locations. Our graphic design elements (color scheme, fonts and logo) were reacted to positively by most users, with some reservations on the navigation bar. There were some minor design inconsistencies commented upon by users. Changes Once logged in, we redirected to the map page rather than the home page. This way, users of our website do not have to stumble over information they no longer need when trying to use our website. Though the footer was one of the well-received aspects of our interface, we improved it by adding the user’s most recent unlocked location rather than a suggestion of where the user should go next. Users can also Tweet about that location, consistent with the most recently unlocked badge. To create a better relationship between the unlocked location and the unlocked badge, the most recent badge area on the footer is denoted by a specific badge icon rather than a generic one. Also, users can now view which locations will unlock certain achievement badges before they travel there on the locked location pop-up. We implemented some minor text and consistency changes to our interface. For example, view profile became view progress. This reflects the type of information that is shown on the user’s profile page and speaks to our future goal of interactive progress tracking of unlocking achievements.
7. Future Changes Many users expressed that we should edit our navigation bar to make it more useful. Some ideas we have for this include making the “Sign In” and “Join” links larger, implementing graphical links to the map and profile pages, choosing a more streamlined font for our links and affording clicking for every clickable link on our navigation bar. One of the most vital changes that we need to make is progress tracking and planning. Users can currently see their progress on the map page in text format. We would like to implement a graphical tracking interface that gives users details on their current progress and on how to unlock future achievements, affording planning of future excursions. QR code posters are how users will be actually unlocking locations. The scavenger hunt aspect of our game is one we have not been able to explore in testing. A fully designed poster would give information about our website, how to get started and information about the site they are currently at. These posters would advertise our website while being informative. Barcelona has many tourist attractions. By adding more locations and badges, are website can become robust and usable. More locations means more badges to unlock, more visibility in Barcelona and more users. Interactivity is something that our website needs to prioritize. The biggest space for user interaction is the map. Roll-over changes, affordances for clicking and details on demand could create a more confident user experience. Lessons Learned We learned the developing good user interfaces is a time-consuming process. If we could have done it differently, we would have wanted more time to spend on each section, so we could reflect on how each step could inform the next, rather than thinking two steps ahead. We learned about structuring teams to facilitate teamwork. Rather than specifying roles early on, we learned that cross-functionality would have made each person responsible for the whole, not their specific area of expertise. We learned that we are not our users. The gap between users and designers can only be lessened with planning and extensive user testing. Even small details of interfaces are taken for granted by designers. We learned that Barcelona is an amazing city and worth exploring!