Tour of research on student evaluations of teaching [34-36]: Martin 1984, Schuster and Van Dyne 1985, and Basow and Silberg 1987

Let's continue our discussion of studies in Holman et al. 2019 "Evidence of Bias in Standard Evaluations of Teaching" listed as "finding bias". See here for the first entry in the series and here for other entries.

---

34.

Martin 1984 "Power and Authority in the Classroom: Sexist Stereotypes in Teaching Evaluations" reported on data from 240 female students and 154 male students across nine course at a large Midwestern university. The instructors were three male social scientists, three female social scientists, and three female women's studies faculty. From the article (p. 488):

I made no attempt to match up male and female faculty on the basis of equal teaching skills (an impossible task in any case). I assumed that, if there were statistically significant differences between the ratings of an instructor by male and female students, then I could attribute these differences to sex bias.

Here is a sample from the results section (p. 490):

Results showed that, when male students were evaluating female social science instructors, high ratings on teaching effectiveness were strongly associated with high ratings on friendliness (r2 = 51 percent), smiles (r2 = 61 percent), eye contact (r2 = 61.6 percent), confidence (r2 = 62.9 percent), and decisiveness (r2 = 72.6 percent). I found no similarly strong associations in male students' ratings of male instructors or of women's studies instructors. No strong associations between teaching effectiveness and personal traits appeared when I examined female students' ratings.

This is interpreted as a bias (p. 492):

...more than their male colleagues, female instructors are likely to have their competence judged by male students on the basis of personal characteristics associated with feminine behavior, such as friendliness, frequent eye contact, and regular smiles. The message to women seems clear: if your institution bases personnel decisions on student evaluations, make sure your colleagues are aware of the possibility of sex bias.

I'm not sure that this sort of bias favors men over women, if the results mean that women social science instructors can improve their student evaluations by smiling and being friendlier but men social science instructors can't improve their student evaluations by smiling and being friendlier.

---

35.

Schuster and Van Dyne 1985 "The Changing Classroom" is an essay that reports no novel data. I'm not sure that it is properly placed in a list of studies "finding bias".

---

36.

Basow and Silberg 1987 "Student Evaluations of College Professors: Are Female and Male Professors Rated Differently?" reported responses from 553 male students and 527 female students at a small private college. Of the 22 female professors at the college who had taught at the college full-time at least one year, 16 were matched to a male professor by rank, division, and years of experience at the college.

Results indicated, among other things, that "...male students gave female professors significantly (p < .05) less positive ratings than they gave male professors on all dependent measures" and that "female students rated female professors significantly more negatively than they rated male professors on Instructor-Individual Student Interaction (p < .01), Dynamism/Enthusiasm (p < .05), and overall teaching ability (p < .01)" (p. 310).

However, Basow and Silberg 1987 notes that "...male professors may in fact be better overall teachers than female professors" (p. 313). Moreover (p. 313):

...the magnitude of the effect sizes is quite small, indicating that sex of instructor and sex of student account for only a small percentage of the variance in student ratings. For example, the combination of knowledge of teacher sex and student sex can predict only about 4% of the variance in scores on overall teaching ability.

---

Comments are open if you disagree, but I don't think that data from the 1980s or earlier are relevant for discussions of whether student evaluations of teaching should be used in employment decisions made in 2019 or beyond.