Comments on "Just seeing a Fox News logo prompts racial bias, new research suggests"
The Monkey Cage tweeted a link to post (Gift et al 2022), claiming that "Just seeing a Fox News logo prompts racial bias, new research suggests".
This new research is Bell et al 2022, which reported on an experiment that manipulated the logo on a news story provided to participants (no logo, CNN, and Fox News) and manipulated the name of the U.S. Army Ranger in the news story who was accused of killing a wounded Taliban detainee, with the name signaling race (e.g., no name, Tyrone Washington, Mustafa Husain, Santiago Gonzalez, Todd Becker).
The Appendix to Bell et al 2022 reports some results for all respondents, but Bell et al 2022 indicates (footnotes and citations omitted):
Research on racial attitudes in America largely theorizes about the proclivities and nuances of racial animus harbored by Whites, so we follow conventions in the literature by restricting our analysis to 1149 White respondents.
---
1.
From the Gift et al 2022 Monkey Cage post (emphasis added):
The result wasn't what we necessarily expected. We didn't anticipate that the Fox News logo might negatively affect attitudes toward the Black service member any more than soldiers of other races. So what could explain this outcome?
The regression results reported in Bell et al 2022 have the "no name" condition as the omitted category, so the 0.180 coefficient and 0.0705 standard error for the [Black X Fox News] interaction term for the "convicted" outcome indicates the effect of the Fox News logo in the Black Ranger condition relative to the effect of the Fox News logo in the no-name condition.
But, for assessing anti-Black bias among White participants, it seems preferable to compare the effect of the Fox News logo in the Black Ranger condition to the effect of the Fox News logo in the White Ranger condition. Otherwise, the Black name / no-name comparison might conflate the effect of a Black name for the Ranger with the general effect of naming the Ranger. Moreover, a Black name / White name comparison would better fit the claim about "any more than soldiers of other races".
---
The coefficient and standard error are 0.0917 and 0.0701 for the [White X Fox News] interaction term for the "convicted" outcome, and I don't think that there is sufficient evidence that the 0.180 [Black X Fox News] coefficient differs from the 0.0917 [White X Fox News] coefficient, given that the difference in coefficients for the interaction terms is only 0.09 and the standard errors are about 0.07 for each interaction term.
Similar concern about the "justified" outcome, which had respective coefficients (and standard errors) of −0.142 (0.0693) for [Black X Fox News] and −0.0841 (0.0692) for [White X Fox News]. I didn't see the replication materials for Bell et al 2022 in the journal's Dataverse, or I might have tried to get the p-values.
---
2.
From the Gift et al 2022 Monkey Cage post:
Of course one study is hardly definitive. Our analysis points to the need for more research into how Fox News and other media may or may not prime racial attitudes across a range of political and social issues.
Yes, one study is not definitive, so it might have been a good idea for the Gift et al 2022 Monkey Cage post to have mentioned the replication attempt *published in Bell et al 2022* in which the [Black X Fox News] interaction term did not replicate in statistical significance or even in the direction of the coefficients, with a −0.00371 coefficient for the "convicted" outcome and a 0.0199 coefficient for the "justified" outcome.
I can't see a good reason for the Gift et al 2022 Monkey Cage post to not report results for the preregistered replication attempt and for the Monkey Cage editors to have not known about the replication attempt or to permit publishing the post without mentioning the lack of replication for the [Black X Fox News] interaction term.
The preregistration suggests that the replication attempt was due to the journal (Research & Politics), so it seems that we can thank a peer reviewer or editor for the replication attempt.
---
3.
Below is the first sentence from the preregistration question about the main question for Study 2:
White Americans who see a story about a non-white soldier will be more likely to say the soldier should be punished for their alleged crime than either an unnamed soldier or a white soldier.
Bell et al 2022 Appendix Table A2 indicates that means for the "convicted" outcome in Study 2 were, from high to low and by condition:
No logo news source 0.725 White name 0.697 Latin name 0.692 MEast name 0.680 No name 0.655 Black name CNN logo 0.705 No name 0.698 Latin name 0.695 Black name 0.694 White name 0.688 MEast name Fox News logo 0.730 No name 0.703 White name 0.702 Black name 0.695 MEast name 0.688 Latin name
So, in the Fox News condition from this *preregistered* experiment, the highest point estimate for a named Ranger was for the White Ranger, for the "convicted" outcome, which seems like a better measure of punishment than the "justified" outcome.
The gap between the highest mean "convicted" outcome for a named Ranger (0.703) and the lowest mean "convicted" outcome for a named Ranger (0.688) was 0.015 units on a 0-to-1 scale. That seems small enough to be consistent with random assignment error and to be inconsistent with the title of the Monkey Cage post of "Just seeing a Fox News logo prompts racial bias, new research suggests".
---
NOTES
1. Tweet question to authors of Bell et al 2022.
2. The constant in the Bell et al 2022 OLS regressions represents the no-name Ranger in the no-logo news story.
In Study 1, this constant indicates that the Ranger in the no-name no-logo condition was rated on a 0-to-1 scale as 0.627 for the "convicted" outcome and as 0.389 for the "justified" outcome. This balance make sense: on net, participants in the no-name no-logo condition agreed that the Ranger should be convicted and disagreed that the Ranger's actions were justified. Appendix Table A1 indicates that the mean "convicted" rating was above 0.50 and the mean "justified" rating was below 0.50 for each of the 15 conditions for Study 1.
But the constants in Study 2 were 0.680 for the "convicted" outcome and 0.711 for the "justified" outcome, which means that, on net, participants in the no-name no-logo condition agreed that the Ranger should be convicted and agreed that the Ranger's actions were justified. Appendix Table A2 indicates that the mean for both outcomes was above 0.50 for each of the 15 conditions for Study 2.
3. I think that Bell et al 2022 Appendix A1 might report results for all respondents: the sample size in A1 is N=1554, but in the main text Table 2 sample sizes are N=1149 for the convicted outcome and 1140 for the justified outcome. Moreover, I think that the main text Figure 2 might plot these A1 results (presumably for all respondents) and not the Table 2 results that were limited to White respondents.
For example, A1 has the mean "convicted" rating as 0.630 for no-name no-logo, 0.590 for no-name CNN logo, and 0.636 for non-name Fox logo, which matches the CNN dip in the leftmost panel of Figure 2 and Fox News being a bit above the no-logo estimate in that panel. But the "convicted" constant in Table 1 is 0.630 (for the no-name no-logo condition), with a −0.0303 coefficient for CNN and a −0.0577 coefficient for Fox News, so based on this I think that the no-name Fox News mean should be lower than the no-name CNN mean.
The bumps in Figure 2 better match with Appendix Table A5 estimates, which are for all respondents.
4. This Bell et al 2022 passage about Study 2 seems misleading or at least easy to misinterpret (emphasis in the original, footnote omitted):
If the soldier was White and the media source was unnamed, respondents judged him to be significantly less justified in his actions, but when the same information was presented under the Fox News logo, respondents found him to be significantly more justified in his actions.
As indicated in the coefficients and Figure 3, the "more justified" isn't more justified relative to the no-name no-logo condition, but more justified relative to the bias against the White Ranger relative to the no-name Ranger in the no-logo condition. Relevant coefficients are −0.131 for "White", which indicates the reduction in the "justified" rating between the no-name no-logo condition and the White-name no-logo condition, and 0.169 for "White X Fox News", which indicates the White-name Fox-News advantage relative to the no-name Fox-News effect.
So the Fox News bias favoring the White Ranger in the Study 2 "justified" outcome only a little more than offset the bias against the White Ranger in the no-logo condition, with a net bias that I suspect might be small enough to be consistent with random assignment error.
Leave a Reply