If your college is like mine, student teaching evaluations are taken very seriously for both pay raises and promotion and tenure decisions. But yet another study is challenging the idea that student evaluations of teaching reliably measure what they’re intended to measure: instructional quality.
The new study, reported by Inside Higher Education (Jan, 19, 2022), builds on the well-documented correlation between students’ grades and how they rate teachers (i.e., students who earned better grades in a course tend to rate those instructors more highly than peers who got lower grades). After testing and eliminating other possible drivers of this correlation, the study, conducted at Ohio State U., asserts that it’s not about instructional quality, workload or grading stringency or leniency–instead, it is student grade satisfaction that drives this correlation.
Like other studies before it, the research (based on 19,158 evaluations in 2021) poses urgent questions about how institutions should use student evaluations of teaching, if at all. Many colleges and universities still widely use them for high stakes personnel decisions. One co-author said, “The burden of proof is really on the wrong side right now. Proponents of student evaluations of teaching should show that they actually improve student outcomes—achievement, persistence, completion. The best prior evidence that has been presented—that evaluations are positively correlated with student grades—clearly do not establish that, as our study shows once again.”
Educators have over the years toyed with the idea of somehow adjusting student ratings for grades given, to mitigate the known grade-rating effect. Some proponents of student evaluations have mused that better grades simply reflect better teaching. But several studies have discredited that idea. Crucially, the OSU study also rules out the notion that students simply reward easier classes with better ratings for the instructor, as the analysis takes students’ class sections, and what it calls “class fixed effects,” into account.
Compared to students who got an A, students who got an A-minus rated the class 0.22 points lower over all, on average, on a 5-point scale used at OSU. Students who got a B rated the class 0.34 points lower. Scores fell further for lower passing grades, to about 0.5 points lower than reported by A students. Students who failed rated their course more than 0.7 points lower than A students.