International Business User Research: Methods and Tools
Qualitative Research on International Business Users
How to interview (business) users
Cross-cultural user research
Online (remote) User Research
Mouse tracking heatmap
Remote user testing
User feedback system
Alibaba.com User Research / Data Analysis 5 yrs of business user research covering both International Buyers and China Suppliers Explored and implemented various user research methods and tools: Online Survey, Remote User Testing, Focus Group, Interview, AB-Testing, Persona and other Data mining projects
interview (business) users • Cross-cultural user research Online (remote) User Research • Mouse tracking heatmap • Remote user testing • User feedback system
user study User surveys Design guidelines Style guides Performance testing Data analysis Stakeholder meeting (Brainstorming) Interviews Paper prototyping Rapid prototyping Diagnostic evaluation Subjective assessment Competitor analysis Contextual inquiry Heuristic evaluation Cognitive walkthrough User surveys Data analysis Field study User observation Parallel design A/B testing Remote evaluation Secondary research Focus group Storyboarding Persona Wizard of Oz Evaluating existing system Interface design patterns Card sorting Affinity diagramming Scenarios of use (User diary) Task analysis User research methods in product development lifecycle: Quantitative – Qualitative – Quantitative Concept FBRD Specs Checklist Review （ from The Persona Lifecycle: Keeping People in Mind Throughout Product Design ）
only have 2 weeks, at most cannot find users to come to your research …… As of December 31, 2012, the platform had around 36.7 million registered users from more than 240 countries and regions and showcased more than 2.8 million supplier storefronts. http://news.alibaba.com/specials/aboutalibaba/aligroup/index.html
Simple Tasks Automated Simple Tasks A/B Tests A/B Tests Various Forms Of Online Surveys Various Forms Of Online Surveys Mouse Tracking Heatmap Mouse Tracking Heatmap Online Focus Group or Phone/IM interviews Online Focus Group or Phone/IM interviews Remote User Testing Remote User Testing Project Progress
Pros Cons Online Interview / Focus Group Explore concept & solutions 6-12 6 ~10 days Rich insight, in depth Appointment for Int’l Focus Group Remote user testing Specific interaction solutions 6-12 6 ~10 days Effective in finding usability problems Appointment for Int’l Focus Group Online tests: (card sorting, simple tasks) Solution prototypes 50+ 10 days Quick concept proof, Effective in IA Limited testable prototype Online survey Specific questions 100+ 10 days Statistical Significant Questionnaire design Mouse Tracking HeatMap Launched product 200+ 3 days Visual, intuitive Single page analysis A/B test Launched product 200++ 15 days Accurate, Statistical Significant May need more time for results
managers) go to interview users together, to enrich shared understanding of our users. Interviewees: China Suppliers International Buyers Methods: Factory visits, tradeshows Phone interview, conference calls Online focus group Member: Mr. Zhou Company: Zhejiang Dale Garments Co., Ltd. Jane F. Ivanov President, Eve Alexander Maternity/Nursing Lingerie Location: USA Industry: Apparel Member since 2005
sync) moments in conversation Let user start telling stories, (where the richest insights lie) Active listening and humility is the most effective way to build trust and rapport Make user the expert and you are the novice, (vice-versa) Ask the participants to play the teacher role help to articulate the details Sometimes you need to show some areas of expertise to make them want to talk to you Near the end of the interview is a great opportunity to ask more audacious questions (about their business) Check your own worldview at the door, ready to embrace user’s mental model （ Empathy ） Don’t fixate on what you expect to learn, cultivate your own general curiosity (about roles in business decisions) *Interviewing Users: How to Uncover Compelling Insights (Portigal, Steve)
elicit participants to talk is an art in interview Three kinds of silence in Japanese conversational silence* Setting-the-stage silence: Indicate readiness for a shared experience Show mutual respect and humility Interviewer need to match with natural responses and thought pace Effort silence: Show efforts in thinking and a desire to contribute Interviewer should allow room to think, use silence to encourage the efforts Failure silence: Indicate confusion or resistance Interviewer should be attentive to this silence and step in to help *Interviewing Users: How to Uncover Compelling Insights (Portigal, Steve)
In a thinking aloud test, you ask test participants to use the system while continuously thinking out loud — that is, simply verbalizing their thoughts as they move through the user interface. May consider “Retrospective Think-aloud protocol”** after task completion • In Asia, it is impolite to tell someone they have a bad design. • It is embarrassing to admit that you cannot find something. East Asians need more encouragement, be attentive to silence Sometimes, young/inexperienced facilitator may be able to collect more negative feedbacks “Teach me how to do it”, asking user to play the teacher’s role * Thinking Aloud: The #1 Usability Tool http://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/ ** Kuusela, H., & Paul, P. (2000). "A comparison of concurrent and retrospective verbal protocol analysis". American Journal of Psychology (University of Illinois Press) 113 (3): 387–404.
zoom in “Westerners focus on a main or dominant foreground object, while people from East Asia pay more attention to context and background. (East Asian people who grow up in the West show the Western pattern.)” “When shown complex, busy scenes, Asian- Americans and non- Asian-Americans recruited different brain regions. The Asians showed more activity in areas that process figure-ground relations— holistic context—while the Americans showed more activity in regions that recognize objects.” How we see it: Culturally different eye movement patterns over visual scenes, Julie E. Boland, Hannah Faye Chua, & Richard E. Nisbett Differences Between East and West Discovered in People’s Brain Activity Carey Goldberg ， http://tech.mit.edu/V128/N9/culture.html Sharon Begley: West Brain, East Brain May explain this!
Problem: Designers think best in pictures, they need better visual way to see users’ interaction on our page elements Eye tracking is too expensive in $ and time Background ： ： ： ： Given a high intent/opportunity to click, the mouse move and click are much more likely to be closely related to the eye movement* * Summary of research on eye - mouse syn available from: Edmonds, A., White, R., Morris, D., Drucker, S. Instrumenting the Dynamic Web. Journal of Web Engineering (JWE), Vol. 6, No. 3 (2007), 243-260. Solution: •Embed click tracking JavaScript to collect user interactions on page. •Real-time adjustment of sampling ratio and/or targeting conditions, i.e. certain types of users, etc.
flow） Recruiting participants Participants registration and screening Inviting participant to testing Participant accept testing Record and review testing sessions Live Recruiting Ads* Online Registration Forms Phone call or IM TechSmith UserVUE UserVUE Morae Manager
： ： ： Used Ethnio Recruiting tool first, developed our own tool later. Basic Function ： ： ： ： Pop-in Live Recruiting Ads. Real time control On/Off, Pop-in ratio, etc. Advanced Function ： Conditioning on member types, reg. date, last visit time, etc. Real time control On/Off, Ratio, Conditions
and screening Live Recruiting support Basic Function ： ： ： ： Used Open Source LimeSurvey system for Registration form and survey Advanced ： ： ： ： Embed chat for answering questions and live recruiting support
(incl. 1st time visitors) Testing on users’ own computer Interactive feedback and communications on major process User Screen Recording and live broadcasting to Observers Can do additional user background interview, possible to do longitude follow-up studies No travel, low cost and low equipment requirements. Minimum requirement for users. Cons: No facial or body movement video if no webcam on user side. Possible interruption on user side, may be limited by Internet speed. Downloading and running application on user’s computer can be a problem. http://www.boltpeters.com/articles/versus.html http://www.techsmith.com/uservue/whyremotetest.asp
PM to confirm user testing tasks. Recruiting participants need to be representative to target user group. From testing plan to pilot testing, close involvement of designer and PM are essential for successful tests.
go to sign up for email notifications of the latest products? Task 3 of 4: How to find out the supplier's contact information? Task 4 of 4: How to check out more information about a product you are interested in? Make interaction prototype testable online: •Used OptimalWorkshop Chalkmark for simple IA tests; •Developed single page interaction prototype tests based on limesurvey and Clickheat mouse tracking
research requested Solution: Use panel, survey ads, email survey invitations to quickly collect large amount of target users feedbacks. Many classical marketing research model can provide deep insight for user research, U&A, perception model, etc. Questionnaire design with analysis in mind is the key to successful online survey. Be sure, not sorry
surveys: limited coverage and long time lag Online customer service and feedback form: manual processing low efficiency, no priority No transparency in user feedback handling, low motivation in participation Many user suggestions are not specific, not actionable Operation and management need more valuable voice from customers …… Solution: User feedback systems with product feature/function setting
post suggestions 2. User interacts on priorities and solutions 3. Company participate in feedback community, provide feedback on feedback Advantages Facilitate multiple product lines, configurable feedback types Integrated user feedback channel Unified feedback interface and community Provide long term customer satisfaction metrics Interaction among users ： Tag and vote on feedbacks Share and discuss solutions Interaction with company ： Transparent processing of user feedback Tracking feedback status on user feedbacks, increased user satisfaction and motivation
closed feedback Spam control, User admin, Voting limits, Scores, etc. Search and suggestion to consolidate duplicates Pre-defined feedback types and product lines Customer service team handle questions and complains Support Ticket tracking, integrate with existing BOPS 、 CS support email system Satisfaction rating on CS provider Suggestions are handled by designated PM PM handle TOP suggestions of related product lines, tracking, organizing, replying Top feedbacks User research provide integrated summary reports Provide user satisfaction metrics by product lines Follow-up studies on Top user feedbacks User feedback Customer service: Reply (if needed) Question & Complains User interaction and CS reply CS handling of question & complains Issue support ticket tracking Suggestions Users vote on priorities PM handle TOP suggestions User track feedback status
•Feedback on feedback Understand real users need •User research •Validate design ASAP •Avoid HiPPO’s design •User-Centered Interface/interaction Design effective solutions •Agile Development •Cross-functional small teams •Bottom-up communication •Fast iterations Fast iterative product development •Evaluating performance •Fast, actionable testing results •Multivariate tests •Clear and concise metrics Metric-based validation Collaborative teams and integrated process User Research Customer service User Research Product Management User Research Design and Development
for the better") refers to a philosophy or practices that focus upon continuous improvement of processes in manufacturing, engineering, supporting business processes, and management. It has been applied in healthcare, government, banking, and many other industries. When used in the business sense and applied to the workplace, kaizen refers to activities that continually improve all functions, and involves all employees from the CEO to the assembly line workers.
jun9
toremolo72
mkimura
gpeyre
javiergs
tsantalis
kentaroy47
naok615
okoso
usaito
yuyay
rudorudo11
moda0
hatefulcrawdad
geoffreycrofte
dougneiner
zakiyama
trallard
lara
carmenintech
bermonpainter
jponch
palkan
gr2m
akmur
