Tour of research on student evaluations of teaching [37-39]: Kierstead et al. 1988, Buck and Tiene 1989, and Dukes and Victoria 1989
Let's continue our discussion of studies in Holman et al. 2019 "Evidence of Bias in Standard Evaluations of Teaching" listed as "finding bias". See here for the first entry in the series and here for other entries.
---
37.
Kierstead et al. 1988 "Sex Role Stereotyping of College Professors: Bias in Students' Ratings of Instructors" reported results for two experiments. The first experiment had 20 female college students and 20 male college students assigned to one of four conditions for a text description of a teaching situation: male/female professor and a professor who was/was not described a frequently spending time with students outside of class; the male professor received more favorable ratings than did the female professor (p<0.02).
The second experiment had 20 female college students and 20 male college students assigned to one of four conditions for a slide tape presentation of a lecture: male/female teacher and a teacher who was/was not smiling; the male teacher received more favorable ratings than did the female teacher (p<0.06).
Results in these two experiments were largely driven by the female target who was not sociable or who did not smile. Here are the means, on a scale from 1 for poor to 6 for outstanding:
5.3 for the sociable male professor
5.3 for the sociable female professor
5.4 for the unsociable male professor
4.4 for the unsociable female professor
---
4.1 for the smiling male teacher
4.2 for the smiling female teacher
4.5 for the unsmiling male teacher
3.3 for the unsmiling female teacher
Pooled standard deviations were about 0.65 for the "sociable" experiment conditions and about 0.9 for the "smiling" experiment conditions, so these are large differences between the sociable female and the unsociable female (about 1.4 standard deviations) and between the smiling female and the unsmiling female (about 1 standard deviation).
---
38.
Buck and Tiene 1989 "The Impact of Physical Attractiveness, Gender, and Teaching Philosophy on Teacher Evaluations" reported results for 42 undergraduate education majors at a state university in the Midwest across eight conditions, with experimental manipulations for instructor sex, instructor attractiveness, and whether an authoritarian or humanistic teaching perspective was attributed to the instructor. I did not see an indication of whether the instructors were described as college instructors or K12 instructors, but the photographs were described as being for persons about 21 years old.
Students rated the instructors on eight items. Many of the interactions had a p-value under 0.05 for a given evaluation item, but there was no main effect for instructor attractiveness and only one main effect for instructor sex (female instructors rated higher than male instructors on overall effectiveness). There was a statistically significant difference for seven of the eight items for teaching perspective (see Tiene and Buck 1987 for a discussion of the teaching perspective results).
I'll quote in full the Holman et al. 2019 summary of Buck and Tiene 1989:
An experiment was conducted at a Midwestern state university. The sample size was composed of 42 undergraduate seniors, mostly Caucasian females; 10 of the 42 students were male and 3 were black (all females). The students were given 1 of 4 different photographs- attractive or unattractive teachers who were either male or female. Each photograph also included a description of the teachers' teaching style/philosophy, divided by either an authoritarian or humanistic style. The results of the study showed that attractiveness did not have an effect on the ratings of instructor effectiveness. The study also found that authoritarianism was strongly associated with negative evaluations. However, contradictingly attractive authoritarian females were rated significantly more positively than the other 3 possibilities of authoritarian instructor characteristics.
It's not clear to me why Buck and Tiene 1989 is in the Holman et al. 2019 list for "Finding Bias" when Holman et al. 2019 has separate lists for "Bias Favoring Women" and "No Gender or Race Bias". The bias in favor of attractive authoritarian females would, if anything, suggest filing under "Bias Favoring Women". Strictly speaking, it is a "bias" that students rated authoritarian teachers less favorably than humanistic teachers, but filing Buck and Tiene 1989 under "Finding Bias" for that reason would stretch the definition of "bias" to include legitimate reasons such as teaching philosophy for students to rate one instructor more favorably than another instructor. (And the implication of the three main Holman et al. 2019 categories is that the "Finding Bias" category is limited to race bias or gender bias disfavoring women).
---
39.
Dukes and Victoria 1989 "The Effects of Gender, Status, and Effective Teaching on the Evaluation of College Instruction" reported results from 144 undergraduates from four sociology courses and two political science courses. Each student was given a description of four scenarios of college teaching, with experimental manipulations that included the professor's sex (e.g., Carl Pierce or Carla Pierce), whether the professor was a department chair, and the presence or absence of a certain characteristic of the professor (knowledgeable, enthusiastic, rapport, and organized).
For predicting teacher effectiveness, results indicated no main effect of professor sex by the knowledgeable scenario, no main effect of professor sex by the enthusiastic scenario, no main effect of professor sex by the rapport scenario, and no main effect of professor sex by the organized scenario. There were two reported interactions involving professor sex that, from what I can tell, were limited to one of the four scenarios, such as instructor sex and chair status interacting in the organized scenario.
Dukes and Victoria 1989 is another publication that I'm not sure should be classified under "Finding Bias". The Feldman 1992 review of the literature (pp. 356-357) summarizes results from Dukes and Victoria 1989, indicating only 2 of the 32 comparisons from Dukes and Victoria 1989 detected a statistically significant association and that neither of these 2 comparisons were of main effects of instructor sex.
---
Comments are open if you disagree, but I don't think that data from the 1980s or earlier are relevant for discussions of whether student evaluations of teaching should be used in employment decisions made in 2019 or beyond.
Leave a Reply