Tour of research on student evaluations of teaching [40-42]: Sidanious and Crane 1989, Feldman 1992, and Feldman 1993

Let's continue our discussion of studies in Holman et al. 2019 "Evidence of Bias in Standard Evaluations of Teaching" listed as "finding bias". See here for the first entry in the series and here for other entries.

---

40.

Sidanious and Crane 1989 "Job Evaluation and Gender: The Case of University Faculty" reported on data for 9,005 undergraduates evaluating 254 male instructors and 147 female instructors. Controlling for factors such as the student's sex and GPA and the instructor's rank and broad field, male instructors received higher evaluations than did female instructors. However, the discussion provides several caveats (pp. 192-193):

...the fact that, in general, men were perceived as being more competent than women need not be a function of gender stereotyping or bias; it is quite possible that men are, in fact more competent in their teaching roles...[and]...Even if these differences [in evaluations] are a function of gender bias rather than perceptual accuracy, the differences are too small to play any major role in how men and women are evaluated.

Holman et al. 2019 lists this study as "finding bias".

---

41.

Feldman 1992 "College Students' Views of Male and Female College Teachers: Part I: Evidence from the Social Laboratory and Experiments" is a review of experimental studies of student evaluations of teaching. I'm not sure that Feldman 1992 should count as an independent publication finding bias. Moreover, the sense of the literature provided in Feldman 1992 is a bit in tension with that provided in the Holman et al. 2019 "Evidence of Bias in Standard Evaluations of Teaching" list. Feldman 1992 includes studies up through 1990 and indicates the following (p. 342, emphasis in the original):

Most of the laboratory studies reviewed here (and summarized in the Appendix) found that male and female teachers did not differ in college student's overall evaluation of them as professionals (as indicated by students' perceptions of their overall teaching performance, their instructional ability, their effectiveness, and their competence and by whether or not students would take a course with them).

However, for studies from 1990 and earlier, the Holman et al. 2019 list includes zero "No Gender or Race Bias" entries and only one "Bias Favoring Women" entry, and the "Bias Favoring Women" entry isn't an experiment. Part of this tension might be due to the Holman et al. 2019 list being incomplete. Multiple studies listed in the Feldman 1992 references are not included in the Holman et al. 2019 list. For example, here is a passage from Mackie 1976 "Students' Perceptions of Female Professors" (p. 346, emphasis in the original):

Contrary to expectations, women teachers were perceived as more competent than male teachers in both task and socio-emotional spheres. Further, the males were not assigned a significantly higher prestige score.

And here is a passage from Basow and Distenfeld 1985 "Teacher Expressiveness: More Important for Males than Females?" (p. 51):

As other research has found (Elmore & LaPointe, 1974, 1975; Lombardo & Tocci, 1979), teacher sex did not have a main effect on student evaluations of teachers.

I have already discussed three relevant publications omitted from the Holman et al. 2019 list. The Feldman 1992 and 1993 reviews indicate that there are additional relevant studies omitted from the Holman et al. 2019 list.

---

42.

Feldman 1993 "College Students' Views of Male and Female College Teachers: Part II: Evidence from Students' Evaluations of Their Classroom Teachers" is a review of non-experimental studies of student evaluations of teaching. The abstract indicates (emphasis added):

Although a majority of studies have found that male and female college teachers do not differ in the global ratings they receive from their students, when statistically significant differences are found, more of them favor women than men. Across studies, the average association between gender and overall evaluation, while favoring women (average r = + .02), is so small as to be insignificant in practical terms.

This again raises the question of the representativeness of the Holman et al. 2019 list, at least for early studies. The Holman et al. 2019 list does not include Bausell and Magoon 1972 "Expected Grade in a Course, Grade Point Average, and Student Ratings of the Course and the Instructor". Here is the Feldman 1993 summary for the association between instructor sex and students' overall evaluation of the instructor in that study:

Bausell and Magoon (1972): 23 courses taught by women and 23 by men at the University of Delaware (excluding courses in the College of Economics, College of Nursing, and the Department of Secretarial Studies), academic year 1969-1970, matched on the semester course was taught, level of the course, and academic department within which the course was taught; single overall rating item ("Overall, how do you evaluate the instructor?").

The direction of the r of .03 (as derived from data on p. 171) and Z of 0.203 cannot be determined from information given.

The Holman et al. 2019 list does not include Brown 1976 "Faculty Ratings and Student Grades: A University-wide Multiple Regression Analysis". Here is the Feldman 1993 summary for the association between instructor sex and students' overall evaluation of the instructor:

Brown (1976): 2,360 course sections at the University of Connecticut, Spring semester of 1973; average score on the 8-item University of Connecticut Rating Scale for Instruction. r = + .04* (as given in Tables 2 and 3); Z = +1 .943*; N = 2,360 section ratings.

For what it's worth, I think that it is acceptable and preferable to not include studies from the 20th century in a review of research on bias in student evaluations of teaching, if the purpose of the list is to be informative for assessing the handling of student evaluations of teaching in 2019 and beyond. But if Holman et al. 2019 includes 20th century studies, it would be nice to have some indication about whether the inclusion of studies is representative.

---

Comments are open if you disagree, but I don't think that there are novel data in these publications that indicate an unfair bias. Even if novel data in these publications did indicate an unfair bias, I think that the data would be too old to be relevant for discussions of whether student evaluations of teaching should be used in employment decisions made in 2019 or beyond. (I'm using "novel data" to refer to data that is initially reported and to not refer to data presented again in reviews.)