Tour of research on student evaluations of teaching [19-21]: Miller and Chamberlin 2000, Chamberlin and Hickey 2001, and Sprague and Massoni 2005

Let's continue our discussion of studies in Holman et al. 2019 "Evidence of Bias in Standard Evaluations of Teaching" listed as "finding bias". See here for the first entry in the series and here for other entries.

---

19.

Miller and Chamberlin 2000 "Women Are Teachers, Men Are Professors: A Study of Student Perceptions" reported on a study in which students in sociology courses were asked to indicate their familiarity with faculty members on a list, and, for faculty members that the student was familiar with, to indicate the highest education degree that the student thinks the faculty member has attained; listed faculty members were the faculty members in the sociology department, plus a fictitious man and a fictitious woman that footnote 6 indicates no student indicated a familiarity with. Results indicated that "controlling for faculty salary, seniority, rank, and award nomination rate, the level of educational attainment attributed to male classroom instructors is substantially and significantly higher than it is for women" (p. 294).

This study isn't about student evaluations of teaching and, from what I can tell, any implications of the study for student evaluations of teaching should be detectable in student evaluations of teaching.

---

20.

From what I can tell, the key finding mentioned above from Miller and Chamberlin 2000 did not replicate in Chamberlin and Hickey 2001 "Student Evaluations of Faculty Performance: The Role of Gender Expectations in Differential Evaluations", which indicated that: "Male versus female faculty credentials and expertise were also nonsignificant on items assessing student perceptions of the highest degree received by the faculty member, the rank of the faculty member, and whether the faculty member was tenured" (p. 10). Chamberlin and Hickey 2001 reported evidence of male faculty being rated differently than female faculty on certain items, but no analysis was reported that assessed whether these differences in ratings could be accounted for by plausible alternate explanations such as faculty performance.

---

21.

Sprague and Massoni 2005 "Student Evaluations and Gendered Expectations: What We Can't Count Can Hurt Us" analyzed data from 66 students at a public university on the East Coast and 223 students at a public university in the Midwest in 1999. Key data were student responses to a prompt to print up to four adjectives to describe the worst teacher that the student ever had and then to print up to four adjectives to describe the best teacher that the student ever had. Results were interpreted to indicate that "Men teachers are more likely to be held to an entertainer standard...[and]...Women teachers are held to a nurturer standard" (p. 791). Table V indicates that Caring is the most common factor for the best male teachers and that Uncaring is the second most common factor for the worst male teachers, so it's not obvious to me that the data permit a strong inference that men aren't also held to a nurturer standard.

---

Comments are open if you disagree, but I don't think that studies 19 and 20 report data indicating for unfair sex or race bias in student evaluations of teaching using a research design with internal validity, with internal validity referring to an analysis that adequately addresses plausible alternate explanations. Study 21 (Sprague and Massoni 2005) reported results suggesting a difference in student expectations for male faculty and female faculty, but I don't know that there's enough in that study to undercut the use of student evaluations in employment decisions.