User Voices: Building a Chorus from Cacophony

#apereo13
Users do not speak with a single voice, nor are the loudest and most insistent necessarily the most representative. How does an institution effectively harness the myriad responses users have to a complex system like NYU Classes (the name for our local instance of Sakai 2.8), especially when it is being rolled out across multiple divisions in a very compressed timeframe? We will describe the ways NYU is addressing the challenge of integrating meaningful user input into the LMS decision-making processes of a large, multi-faceted university with a great variety of needs and teaching practices. It has developed a tiered feedback system through which a large User Advisory Group (UAG) is guided by a smaller UAG Executive Committee on which IT professionals and active educators and scholars collaborate to identify issues and focus feedback before passing it on to the implementation team and the University?s higher-level educational technology decision makers. Attendees will take away ideas both for structuring the collection of user input and for finding the most appropriate tools to facilitate the conversation.

Transcript

1. User Voices: Building a Chorus from Cacophony 04-Jun-2013 from 3:00 PM - 3:45 PM Room Diamond I
2. About us Robert Squillace, Associate Dean for Academic Affairs, New York University Jeff Pasch, NYU Classes Project Manager, New York University Francesca Socolick, NYU Classes Training and Support Lead, New York University
3. User Needs Challenges: The Signal and the Noise •  User needs differ, even contradict each other: Loudest Users ≄ Most Representative Users
4. User Needs Challenges: The Signal and the Noise •  User needs differ, even contradict each other: Loudest Users ≄ Most Representative Users •  User needs differ, even contradict each other: Ordinary Users ≄ Creative Users
5. User Needs Challenges: The Signal and the Noise •  User needs differ, even contradict each other: Loudest Users ≄ Most Representative Users •  User needs differ, even contradict each other: Ordinary Users ≄ Creative Users •  The level of faculty need ≄ the level of student need
6. User Needs Challenges: The Signal and the Noise •  User needs differ, even contradict each other: Loudest Users ≄ Most Representative Users •  User needs differ, even contradict each other: Ordinary Users ≄ Creative Users •  The level of faculty need ≄ the level of student need •  Users often articulate their needs in terms of known structures for meeting those needs, rather than identifying the actual need
7. User Needs Challenges: The User Needs Paradox Users will never be satisfied. We will always have jobs. User perceives unmet need Need is met What’s present implies what’s absent
8. Addressing User Needs Process for addressing user needs INFORMS Structure of the group(s) and the tools for addressing user needs
9. The Process 1.  Identify users needs (down to a granular level) 2.  Provide targeted feedback on those needs 3.  Review feedback to establish priorities 4.  Make actionable recommendations
10. How do we hear about user needs? Ticketing system Surveys Representative Group of Users Directly to Service Team Reported User Needs Fan/Hate mail Dr John Doe 3145 Broadway New York, NY (UAG)
11. Governance Structures Teaching Technology Committee LMS Subcommittee NYU Classes Project Team Technical Advisory Group User Advisory Group Provost
12. User Advisory Group – three parts 1. UAG Executive Committee Composition: ~20 faculty members, administrators, support providers Frequency: Bi-weekly, in-person Purpose: Identification of needs, review feedback, make recommendations Approach: In person discussion, online feedback tool - UserEcho
13. Identifying Needs - UserEcho
14. User Advisory Group – three parts 2. User Advisory Group (Full group) Composition: ~20 mostly faculty + ~20 UAG EC members Frequency: asynchronously, online Purpose: Provide targeted feedback and voting on topics identified by the UAG Executive Committee Method: UserEcho
15. User Advisory Group – three parts 3. User Advisory Planning Committee Composition: 2 faculty co-chairs, 2 members of NYU Classes Functional Team Meets: bi-weekly or as needed, in-person Purpose: Administer the process and tools
16. Process Flow LMS Subcommittee UAG Executive Committee UAG (full group) Identify Needs Identify areas for feedback Provide feedback and vote on ideas Review feedback & establish priorities Identify areas of interest Review recommendations and authorize work Recommend Changes
17. Example: UAG in action! Let’s look at an example of how the UAG applied this process. source: http://www.flickr.com/photos/tabor-roeder/4922831655
18. Example: UAG in action Jane’s Idea User Echo
19. Example: UAG in action
20. And then some others in the group saw Jane's idea, and began to add similar ideas around this theme... and these ideas began to get some votes. Example: UAG in action Jane’s Idea + others User Echo
21. Example: UAG in action
22. source: http:// www.flickr.com/ photos/kismihok/ 3797812551 "my course has multiple sections, and each exam needs to behave differently" "my exams need release dates, and I can't seem to find it!! I’m not using it now.” "What does 'high security' mean'? What's my IP address?" Example: UAG in action Based on UserEcho… How are assessment settings?
23. Collective ideas of assessments What are essential settings? What are additional settings? Example: UAG in action
24. Example: UAG in action What is essential? What is additional? Organization, information, visual cues
25. Example: UAG in action
26. Example: UAG in action Teaching Technology Committee LMS Subcommittee NYU Classes Project Team Technical Advisory Group User Advisory Group Provost
27. Example: UAG in action! “Users often articulate their needs in terms of known structures for meeting those needs, rather than identifying the actual need” source: http://www.flickr.com/photos/tabor-roeder/4922831655
28. Questions?