How to Produce Course Evaluations that Matter

Course evaluations can be an effective means of collecting feedback from students on courses and instructors if you learn to avoid a few simple mistakes.

TRACS 2012

Transcript

1. HOW TO PRODUCE Course Evaluations THAT MATTER Wednesday, November 7, 12
2. THE ALTERNATIVE... “This class is worthwhile because I need it for the degree.” “In class I learn I can fudge answers and get away with it.” “I would sit in class and stare out the window at the squirrels. They’ve got a cool nest in the tree.” “TA steadily improved throughout the course... I think he started drinking and it really loosened him up.” Wednesday, November 7, 12
3. ...NOT THAT YOUR STUDENTS WOULD RESPOND THAT WAY. \ Wednesday, November 7, 12
4. LET’S INTRODUCE OURSELVES Wednesday, November 7, 12
5. OUR SOFTWARE HELPS PEOPLE MANAGE COLLEGES Me Mark Joe Brendan Pat Adam Nick Toby James Christian Isaac Grauke, CEO @isaacgrauke Wednesday, November 7, 12
6. SEARCH THE WEB FOR EFFECTIVE COURSE EVALUATIONS AND YOU CAN READ FOR DAYS Wednesday, November 7, 12
7. “Are course evaluations a reliable and valid assessment tool?” NOT RELIABLE, NOT VALID RELIABLE, NOT VALID RELIABLE & VALID Wednesday, November 7, 12
8. MOST RESEARCHERS AGREE... Highly Reliable Moderately Valid Wednesday, November 7, 12
9. TOP FACTORS INFLUENCING VALIDITY 1. Student’s prior interest in subject matter 2. Expected grades 3. The perceived workload 4. Reason for taking course Wednesday, November 7, 12
10. DO YOU AGREE? Wednesday, November 7, 12
11. EVERYTHING HINGES ON THE WAY EVALUATIONS ARE WRITTEN Wednesday, November 7, 12
12. “The shorter and the plainer the better.” Beatrix Potter ”Men of few words are the best men.” William Shakespeare “The trouble with so many of us is that we underestimate the power of simplicity.” Robert Stuberg Wednesday, November 7, 12
13. During the past year, the instructor’s presentation skills have: EXAMPLE 1 1. Increased a lot 2. Increased somewhat 3. Increased a little 4. Decreased. Wednesday, November 7, 12
14. During the past year, the instructor’s presentation skills have: EXAMPLE 1 1. Increased a lot 2. Increased somewhat 3. Increased a little 4. Decreased. Huh? What? Wednesday, November 7, 12
15. Did the papers serve to increase your knowledge and broaden your abilities? Were the instructor’s comments timely and helpful? EXAMPLE 2 Wednesday, November 7, 12
16. Did the papers serve to increase your knowledge and broaden your abilities? Were the instructor’s comments timely and helpful? EXAMPLE 2 1 2 3 4 Wednesday, November 7, 12
17. RULES OF THUMB WHEN WRITING QUESTIONS 1. Establish objectives 2. Be brief 3. Be clear/precise 4. Don’t combine questions Wednesday, November 7, 12
18. GUIDE TO ANALYZING EVALUATION RESULTS 1. In what ways, if any, do the data seem surprising? 2. Are there any patterns in the data? 3. What are you doing in the classroom that is resulting in these responses? 4. How will you respond to the feedback? 5. What additional feedback, if any, do you need? Wednesday, November 7, 12
19. WHAT HAVE YOU LEARNED FROM COURSE EVALUATIONS? Wednesday, November 7, 12
20. PAPER v. ONLINE Wednesday, November 7, 12
21. PAPER ONLINE High response rate Low response rate Easy to do in class Tough to do in class Expensive Cheap* Hard to analyze Easy to analyze Difficult to ensure anonymity Easy to ensure anonymity Wednesday, November 7, 12
22. INTRODUCING COURSE EVALUATIONS THROUGH POPULI Wednesday, November 7, 12
23. EVALUATIONS ATTACHED TO EACH COURSE Wednesday, November 7, 12
24. SEVEN QUESTION TYPES Wednesday, November 7, 12
25. INSTANT REPORTING Wednesday, November 7, 12
26. EXTRA GOODIES 1. Ensure absolute anonymity 2. Increase response rates by withholding grades 3. Advanced reporting Wednesday, November 7, 12
27. Questions? Wednesday, November 7, 12