Month: February 2015

You might have seen a Tweet or Facebook post on a recent study about sex bias in teacher grading:

When graded anonymously, girls outperform boys at math. When names known, boys do better. http://t.co/waHkMEX9zu via @AdamMGrant @clairecm

— Vanessa Bohns, PhD (@profbohns) February 8, 2015

Here is the relevant section from Claire Cain Miller's Upshot article in the New York Times describing the study's research design:

Beginning in 2002, the researchers studied three groups of Israeli students from sixth grade through the end of high school. The students were given two exams, one graded by outsiders who did not know their identities and another by teachers who knew their names.

In math, the girls outscored the boys in the exam graded anonymously, but the boys outscored the girls when graded by teachers who knew their names. The effect was not the same for tests on other subjects, like English and Hebrew. The researchers concluded that in math and science, the teachers overestimated the boys' abilities and underestimated the girls', and that this had long-term effects on students' attitudes toward the subjects.

The Upshot article does not mention that the study's first author had previously published another study using the same methodology, but with the other study finding a teacher grading bias against boys:

The evidence presented in this study confirms that the previous belief that schoolteachers have a grading bias against female students may indeed be incorrect. On the contrary: on the basis of a natural experiment that compared two evaluations of student performance–a blind score and a non-blind score–the difference estimated strongly suggests a bias against boys. The direction of the bias was replicated in all nine subjects of study, in humanities and science subjects alike, at various level of curriculum of study, among underperforming and best-performing students, in schools where girls outperform boys on average, and in schools where boys outperform girls on average (p. 2103).

This earlier study was not mentioned in the Upshot article and does not appear to have been mentioned in the New York Times ever. The Upshot article appeared in the print version of the New York Times, so it appears that Dr. Lavy has also conducted a natural experiment in media bias: report two studies with the same methodology but opposite conclusions, to test whether the New York Times will report on only the study that agrees with liberal sensibilities. That hypothesis has been confirmed.