Renae Youngs of Minnesota State Arts Board_Workshop I Presentation

Renae Youngs is Director of Research and Evaluation at the Minnesota State Arts Board. In that role, she is tasked with documenting the impact of the arts on the people of Minnesota. Renae has also worked as a research and evaluation consultant in informal learning environments ranging from art museums to zoos helping clients to clarify their goals, design effective programs and exhibitions, learn about their visitors or audiences, and document the processes and outcomes of their work. She is an evaluation generalist with experience in both qualitative and quantitative methods, research design, and program evaluation. Her professional interests include evaluation capacity building and the use and usability of evaluation findings; she serves on the Board of Directors of the Visitor Studies Association. Renae is also active in professional service promoting institutions’ work with diverse audiences, including serving on the steering committee of the American Alliance of Museums’ LGBTQ Alliance Professional Network.

Transcript

3. Data for Evaluation A very short introduction Renae Youngs | Minnesota State Arts Board | March 25, 2017
4. Today’s goal  Equip you to make the most of evaluation in your work  Not learning how to do evaluation in 30 minutes What will we accomplish together?
5. Why evaluation? Evaluation is just one way to use data. But it’s not just about data. It’s a process and a habit, and it helps us to:  Improve our work  Increase our impact / the service we provide  Understand our audiences’ needs  Show evidence of our successes Evaluation is a very important way to use data.
6. Defining evaluation “The systematic collection of information about the activities, characteristics, and results of programs to make judgments about the program, improve or further develop program effectiveness, inform decisions about future programming, and/or increase understanding.” – [Patton (2008), p. 39]  NOT the following: a test, a verdict, an aesthetic assessment, a satisfaction survey, a hoop to jump  “Measurement” is a scary evaluation word. But it’s just systematic observation. What do artists do? Observe, reflect, create. Iterate. Improve. Evaluation is a very important way to use data.
7. Some other definitions  Data vs. information vs. knowledge  Evaluation vs. research  Quantitative vs. qualitative (data or methods)  Subjective vs. objective (data or information) Pull these out to sound cool at nerd parties.
8. When do we evaluate? Planning and evaluation are closely intertwined  Evaluation up front: seeking out and digesting relevant, useful information to shape a plan  Evaluation mid-course: testing, observing, iterating, and improving on an experience  Evaluation of finished products: determining the success of an experience Even when there’s no “data,” planning and reflection are evaluative: we articulate and examine the relationship between our actions and the world. Evaluation practices help us plan, learn, and reflect throughout a project.
9. What do we evaluate?  Front end  Formative  Summative  Remedial  Process  And more Different phases of work have different evaluation needs. Here are some types of evaluation.
10. Evaluation questions Evaluation serves many different users & purposes. Whose needs matter to you?  People creating the experience  Key stakeholders (audience, community, funder)  People who may expand or replicate the experience  People tasked with making decisions related to the experience (you boss, your gallery or venue) Some sample evaluation questions: a brave volunteer? Evaluation questions are driven by what you need to show, know, or improve related to your work.
11. Useful evaluation You should know how to make your work more successful at whatever it’s supposed to do.  Plan, goals, questions  Design and data  Results and sense-making  Informed course of action What should you end up with at the end* of the evaluation process? * It shouldn’t end! It’s a beautiful cycle!
12. Getting expert help  Knowledge of the field / relevant literature  Help you focus your goals and needs  Technical expertise in evaluation design  Coaching for capacity  Technical expertise & focus in analysis & reporting  Critical friend for decision-making What help should you expect from an evaluator? You probably don’t need a professional! But a good evaluation expert should be able to offer you at least most of these things.
13. Resources (1) Evaluation Handbook. W.K. Kellogg Foundation. https://www.wkkf.org/resource-directory/resource/2010/w-k- kellogg-foundation-evaluation-handbook. Getting Started with Program Evaluation: A Guide for Arts Organizations. Georgia Council for the Arts and National Assembly of State Arts Agencies. http://www.nasaa- arts.org/Member-Files/Evaluation_Guide.pdf. Principal Investigator’s Guide: Managing Evaluation in Informal STEM Education Projects. Center for the Advancement of Informal Science Education. http://www.informalscience.org/evaluation/pi-guide. Team-Based Inquiry Guide. National Informal STEM Education Network. http://nisenet.org/catalog/team-based-inquiry-guide. YouthARTS: Evaluation. Americans for the Arts. http://youtharts.artsusa.org/evaluation/approach.html. For your own adventures – evaluation learning resources and handbooks
14. Resources (2) ArtsEdSearch. Arts Education Partnership. http://www.artsedsearch.org. Arts Policy Library. Createquity. http://createquity.com/arts-policy- library/ Creative Placemaking Library. ArtPlace America. http://www.artplaceamerica.org/library Grantmakers in the Arts. Grantmakers in the Arts. http://www.giarts.org/ Informal Science. Center for the Advancement of Informal Science Education. http://www.informalscience.org. Knowledge Center. The Wallace Foundation. http://www.wallacefoundation.org/knowledge- center/Pages/default.aspx For your own adventures – sources of inspiration and information about arts- related outcomes and evaluation reports
15. Questions & discussion