ADDC 2019 - Matthew Eng - Everyone Does Research: A Case Study on Inclusive Qualitative Research Methods for Product Teams at IBM.

IBM Design’s mission was to shift how it approached product strategy, but it led to friction between multidisciplinary teams grasping for a unified vision. Learn lessons from assembling a research team that broke bad data analysis habits and started inclusive generative and evaluative techniques. When I started doing UX research as a designer, we often felt our work and results were ignored by developers, architects, and product managers. After a series of stressful releases full of heated arguments between our multidisciplinary teams, I invested the post-release downtime to better understand the questions sought by the whole team. This lead to reorganizing the research, so that we conducted to include design, product management, and development. The research team gathered and surfaced data that each of the disciplines where focusing on separately to help construct a clearer direction for the product.

Recordings & more: https://addconf.com/

Transcript

1. Research is the Glue
   Inclusive techniques to share data
   @engmatthew
   

   View Slide

2. IBM Design
   @engmatthew
   2
   @engmatthew
   

   View Slide

3. !3
   @engmatthew
   

   View Slide

4. I do UX design and research
   @engmatthew
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. VS
    

    View Slide

11. VS
    

    View Slide

12. @engmatthew
    

    View Slide

13. @engmatthew
    

    View Slide

14. @engmatthew
    

    View Slide

15. @engmatthew
    

    View Slide

16. We like to think
    that invention
    comes as a
    flash of insight
    -Vaclav Smil
    “
    @engmatthew
    

    View Slide

17. Allowing one person to
    believe that he or she is a
    vessel for creativity and
    (innovation) is too much
    responsibility.
    “
    @engmatthew
    -Elizabeth Gilbert
    

    View Slide

18. It can also be
    beneficial to
    create a mashup
    of skills and
    duties
    -Lauren Marchese
    “
    @engmatthew
    

    View Slide

19. Everyone can contribute to research
    Research can work in Agile environments
    Research must facilitate data inclusion
    @engmatthew
    

    View Slide

20. @engmatthew
    Not every can lead a research project
    Not all research can fit into agile sprints
    

    View Slide

21. How we work
    @engmatthew
    

    View Slide

22. Squads
    Embedded Research
    !22
    Research
    PM
    Dev
    @engmatthew
    Design
    

    View Slide

23. Accounts
    Embedded Research
    !23
    Research
    PM
    Dev
    @engmatthew
    Design
    Access
    

    View Slide

24. Qualitative
    !24
    @engmatthew
    

    View Slide

25. !25
    @engmatthew
    

    View Slide

26. !26
    @engmatthew
    

    View Slide

27. Navigation Case Study
    @engmatthew
    

    View Slide

28. Tokyo Station Yoyogi Park
    @engmatthew
    

    View Slide

29. @engmatthew
    

    View Slide

30. @engmatthew
    

    View Slide

31. @engmatthew
    

    View Slide

32. @engmatthew
    

    View Slide

33. !33
    @engmatthew
    

    View Slide

34. Accounts
    Help
    Docs
    Network
    Containers
    DevOps
    Security
    Compute
    AI
    @engmatthew
    

    View Slide

35. Accounts
    Help
    Docs
    Network
    Containers
    DevOps
    Security
    AI
    Help and Assistance
    Network
    Compute
    VM
    Bare Metal
    Storage
    Compute
    Security
    Learn
    @engmatthew
    

    View Slide

36. Customer 1 Customer 2 Customer 3 Customer 4
    !36
    @engmatthew
    

    View Slide

37. Green Olive Tree Avnet
    Indeed Ogilvy
    Arrow
    Kollacode
    Customer 1 Customer 2 Customer 3
    Customer 4 Customer 5 Customer 6
    37
    

    View Slide

38. Seems like it is laid out by lines of business
    in IBM. Not necessarily the way I would
    architect an app
    “
    !38
    -Customer
    @engmatthew
    

    View Slide

39. IBM is currently dealing with a major rebranding exercise which is
    hurting its positioning. It is in the process of phasing out the SoftLayer
    and Bluemix brands
    !39
    “
    -Forbes
    @engmatthew
    

    View Slide

40. Reflections
    @engmatthew
    

    View Slide

41. 41
    Design
    PM
    Dev
    Discovery Validation Implementation
    Process
    How we worked together
    @engmatthew
    

    View Slide

42. 42
    Design
    PM
    Dev
    Discovery Validation Implementation
    Process
    How we worked together
    Research
    @engmatthew
    Customer and
    user interviews
    

    View Slide

43. 43
    Design
    PM
    Dev
    Discovery Validation Implementation
    Process
    How we worked together
    Research
    Analytic Trends
    Competition Adoption
    and Tickets
    @engmatthew
    

    View Slide

44. 44
    Design
    PM
    Dev
    Discovery Validation Implementation
    Process
    How we worked together
    Research
    Analyst
    NPS Feedback
    New Shining Thing
    @engmatthew
    Leadership
    

    View Slide

45. Changes
    @engmatthew
    

    View Slide

46. 46
    Design
    PM
    Dev
    Discovery Validation Implementation
    Process
    How we worked together
    @engmatthew
    Research Customer and
    user interviews
    

    View Slide

47. 47
    Design
    PM
    Dev
    Discovery Validation Implementation
    Process
    How we want to work together
    Research
    @engmatthew
    Who are our
    users?
    Will this make
    sense for them?
    How is this
    performing in
    the wild?
    

    View Slide

48. 50%
    marketing content
    (benefits + overview)
    50%
    technical content
    (features + use cases)
    marketing content
    (benefits with
    marketing tone)
    100%
    Technical content
    (benefits with technical
    tone + features + use
    cases)
    60%
    Pricing Info
    30%
    Marketing content
    5%
    Documentation
    5%
    Marketing
    Technical
    Pricing
    Docs
    Competitive Analysis
    How is your competition
    addressing the problem space?
    

    View Slide

49. Heuristic Evals
    Guide your teams through end-
    to-end heuristic evals to get
    away from individual screens
    and more focused on goals,
    tasks, and subtasks.
    
    User one language for heuristic
    principles.
    

    View Slide

50. Test Prep
    Write the steps and tasks
    together. Show where design,
    PM, and dev have questions,
    and possible variations.
    
    Align on what we are asking
    users to do and what metrics
    we are going to capture.
    

    View Slide

51. @engmatthew
    

    View Slide

52. Open up the testing sessions
    @engmatthew
    

    View Slide

53. Open up the synthesis
    @engmatthew
    

    View Slide

54. @engmatthew
    

    View Slide

55. View Slide

56. 56
    @engmatthew
    

    View Slide

57. Takeaways
    @engmatthew
    

    View Slide

58. Find what your team needs
    

    View Slide

59. Invest time in an open process
    

    View Slide

60. Be a guide
    

    View Slide

61. Q/A
    !61
    @engmatthew
    

    View Slide

62. Thank you
    !62
    @engmatthew
    

    View Slide