My new publication is a technical comment on the Schneider and Gonzalez 2021 article "Racial resentment predicts eugenics support more robustly than genetic attributions".

The experience with the journal Personality and Individual Differences was great. The journal has a correspondence section that publishes technical comments and other types of correspondence, which seems like a great way to publicly discuss research and to hopefully improve research. The authors of the article that I commented on were also great.

---

My comment highlighted a few things about the article, and I think that two of the comments are particularly generalizable. One comment, which I discussed in prior blog posts [1, 2], concerns the practice of comparing the predictive power of factors that are not or might not be equally well measured. I don't think that is a good idea, because measurement error can bias estimates.

The other comment, which I discussed in prior blog posts [1, 2], concerns analyses that model an association as constant. I think that it is more informative to not model key associations as constant, and Figure 1 of the comment illustrates an example of how this can provide useful information.

There is more in the comment. Here is a 50-day share link for the comment.

Tagged with: , ,

This year, I have discussed several errors or flaws in recent journal articles (e.g., 1, 2, 3, 4). For some new examples, I think that Figure 2 of Cargile 2021 reported estimates for the feminine factor instead of, as labeled, the masculine factor, and Fenton and Stephens-Dougan 2021 described a "very small" 0.01 odds ratio as "not substantively meaningful":

Finally, the percent Black population in the state was also associated with a statistically significant decline in responsiveness. However, it is worth noting that this decline was not substantively meaningful, given that the odds ratio associated with this variable was very small (.01).

I'll discuss more errors or flaws in the notes below, with more blog posts planned.

---

Given that peer review and/or the editing process will miss errors that readers can catch, it seems like it would be a good idea for journal editors to get more feedback before an article is published.

For example, the Journal of Politics has been posting "Just Accepted" manuscripts before the final formatted version of the manuscript is published, which I think permits the journal to correct errors that readers catch in the posted manuscripts.

The Journal of Politics recently posted the manuscript for Baum et al. "Sensitive Questions, Spillover Effects, and Asking About Citizenship on the U.S. Census". I think that some of the results reported in the text do not match the corresponding results reported in Table 1. For example, the text (numbered p. 4) indicates that:

Consistent with expectations, we also find this effect was more pronounced for Hispanics, who skipped 4.21 points more of the questions after the Citizenship Treatment was introduced (t-statistic = 3.494, p-value is less than 0.001).

However, from what I can tell, the corresponding Table 1 result indicates a 4.49 difference, with a t-statistic of 3.674.

---

Another potential flaw in the above statement is that, from what I can tell, the t-statistic for the "more pronounced for Hispanics" claim is based on a test of whether the estimate among Hispanics differs from zero. However, the t-statistic for the "more pronounced for Hispanics" claim should instead be from a test of whether the estimate among Hispanics differs from the estimate among non-Hispanics or whatever comparison category the "more pronounced" refers to.

---

So, to the extent that these aforementioned issues are errors or flaws, maybe these can be addressed before the Journal of Politics publishes the final formatted version of the Baum et al. manuscript.

---

NOTES

1. I think that this is an error, from Lucas and Silber Mohamed 2021, with emphasis added:

Moreover, while racial sympathy may lead to some respondents viewing non-white candidates more favorably, Chudy finds no relationship between racial sympathy and gender sympathy, nor between racial sympathy and attitudes about gendered policies.

That seemed a bit unlikely to me when I read it, and, sure enough, Chudy 2020 footnote 20 indicates that:

The raw correlation of the gender sympathy index and racial sympathy index was .3 for the entire sample (n = 1,000) and .28 for whites alone (n = 751).

2. [sic] errors in Jardina and Stephens-Dougan 2021. Footnote 25:

The Stereotype items were note included on the 2020 ANES Time Series study.

...and the Section 4 heading:

Are Muslim's part of a "band of others?"

... and the Table 2 note:

2016 ANES Time Serie Study

Moreover, the note for Jardina and Stephens-Dougan 2021 Figure 1 describes the data source as: "ANES Cumulative File (face-to-face respondents only) & 2012 ANES Times Series (all modes)". But, based on the text and the other figure notes, I think that this might refer to 2020 instead of 2012.

These things happen, but I think that it's worth noting, at least as evidence against the idea that peer reviews shouldn't note grammar-type errors.

3. I discussed conditional-acceptance comments in my PS symposium entry "Left Unchecked".

Tagged with: , ,

The American Political Science Review recently published Mason et al. 2021 "Activating Animus: The Uniquely Social Roots of Trump Support".

Mason et al. 2021 measured "animus" based on respondents' feeling thermometer ratings about groups. Mason et al. 2021 reported results for a linear measure of animus, but seemed to indicate an awareness that a linear measure might not be ideal: "...it may be that positivity toward Trump stems from animus toward Democratic groups more than negativity toward Trump stems from warmth toward Democratic groups, or vice versa" (p. 7).

Mason et al. 2021 addressed this by using a quadratic term for animus. But this retains the problem that estimates for respondents at a high level of animus against a group are influenced by responses from respondents who reported less animus toward the group and from respondents who favored the group.

I think that a better strategy to measure animus is to instead compare negatively toward the groups (i.e., ratings below the midpoint on the thermometer or at a low level) to indifference (i.e., a rating at the midpoint on the thermometer). I'll provide an example below, with another example here.

---

The Mason et al. 2021 analysis used thermometer ratings of groups measured in the 2011 wave of a survey to predict outcomes measured years later. For example, one of the regressions used feeling thermometer ratings about Democratic-aligned groups as measured in 2011 to predict favorability toward Trump as measured in 2018, controlling for variables measured in 2011 such as gender, race, education, and partisanship.

That research design might be useful for assessing change net of controls between 2011 and 2018, but it's not useful for understanding animus in 2021, which I think some readers might infer from the "motivating the left" tweet from the first author of Mason et al. 2021, that:

And it's not happening for anyone on the Democratic side. Hating Christians and White people doesn't predict favorability toward any Democratic figures or the Democratic Party. So it isn't "anti-White racism" (whatever that means) motivating the left. It's not "both sides."

The 2019 wave of the survey used in Mason et al. 2021 has feeling thermometer ratings about White Christians, and, sure enough, the mean favorability rating about Hillary Clinton in 2019 differed between respondents who rated White Christians at or near the midpoint and respondents who rated White Christians under or well under the midpoint:

Even if the "motivating the left" tweet is interpreted to refer only to the post-2011 change controlling for partisanship, ideology, and other factors, it's not clear why that restricted analysis would be important for understanding what is motivating the left. It's not like the left started to get motivated only in or after 2011.

---

NOTES

1. I think that Mason et al. 2021 used "warmth" at least once discussing results from the linear measure of animus, in which "animus" or "animosity" could have been used, in the passage below from page 4, with emphasis added:

Rather, Trump support is uniquely predicted by animosity toward marginalized groups in the United States, who also happen to fall outside of the Republican Party's rank-and-file membership. For comparison, when we analyze warmth for whites and Christians, we find that it predicts support for Trump, the Republican Party, and other elites at similar levels.

It would be another flaw of a linear measure of animus if an association can be described as having been predicted by animosity or by warmth (e.g., animosity toward Whites and Christians predicts lower levels of support for Trump and other Republicans at similar levels)

2. Stata code. Dataset. R plot: data and code.

Tagged with: , , ,

See here for a discussion of the Rice et al. 2021 mock juror experiment.

My reading of the codebook for the Rice et al. 2021 experiment is that, among other items, the pre-election survey included at least one experiment (UMA303_rand), then a battery of items measuring racism and sexism, and then at least another experiment. Then, among other items, the post-election survey included the CCES Common Content racial resentment and FIRE items, and then the mock juror experiment.

The pre-election battery of items measuring racism and sexism included three racial resentment items, a sexism battery, three stereotypes about Blacks and Whites (laziness, intelligence, and violent), and 0-to-100 feeling thermometers about Whites and about Blacks. In this post, I'll report some analyses of how well these pre-election measures predicted discrimination in the Rice et al. 2021 mock juror experiment.

---

The first plot reports results among White participants who might be expected to have a pro-Black bias. For example, the first estimate is for White participants who had the lowest level of racial resentment. The dark error bars indicate 83.4% confidence intervals, to help compare estimates to each other. The lighter, longer error bars are 95% confidence intervals, which are more appropriate for comparing as estimate to a given number such as zero.

The plotted outcome is whether the participant indicated that the defendant was guilty or not guilty. The -29% for the top estimate indicates that, among White participants who had the lowest level of racial resentment on this index, the percentage that rated the Black defendant guilty was 29 percentage points lower than the percentage that rated the White defendant guilty.

The plot below reports results among White participants who might be expected to have a pro-White bias. The 26% for the top estimate indicates that, among White participants who had the highest level of racial resentment on this index, the percentage that rated the Black defendant guilty was 26 percentage points higher than the percentage that rated the White defendant guilty.

---

The Stata output reports additional results, for the sentence length outcome, and for other predictors: a four-item racial resentment index from the post-election survey, plus individual stereotype items (such as for White participants who rated Blacks higher than Whites on an intelligence scale). Results for the sentence length outcome are reported for all White respondents and, in later analyses, for only those White respondents who indicated that the defendant was guilty.

---

NOTE

1. Data for Rice et al. 2021 from the JOP Dataverse. Original 2018 CCES data for the UMass-A module, which I used in the aforementioned analyses. Stata code. Stata output. Pro-Black plot: dataset and code. Pro-White plot: dataset and code.

Tagged with: , , ,

Forthcoming at the Journal of Politics is Rice et al. 2021 "Same As It Ever Was? The Impact of Racial Resentment on White Juror Decision-Making".

---

See the prior post describing the mock juror experiment in Rice et al. 2021.

The Rice et al. 2021 team kindly cited my article questioning racial resentment as a valid measure of racial animus. But Rice et al. 2021 interpreted their results as evidence for the validity of racial resentment:

Our results also suggest that racial resentment is a valid measure of racial animus (Jardina and Piston 2019) as it performs exactly as expected in an experimental setting manipulating the race of the defendant.

However, my analyses of the Rice et al. 2021 data indicated that a measure of sexism sorted White participants by their propensity to discriminate for Bradley Schwartz or Jamal Gaines:

I don't think that the evidence in the above plot indicates that sexism is a valid measure of racial animus, so I'm not sure that racial resentment sorting White participants by their propensity to discriminate for Bradley or Jamal means that racial resentment is a valid measure of racial animus, either.

---

I think that the best two arguments against racial resentment as a measure of anti-Black animus are:

[1] Racial resentment on its face plausibly captures non-racial attitudes, and it is not clear that statistical control permits any post-statistical control residual association of racial resentment with an outcome to be interpreted as anti-Black animus, given that racial resentment net of statistical control often predicts outcomes that are not theoretically linked to racial attitudes.

[2] Persons at low levels of racial resentment often disfavor Whites relative to Blacks (as reported in this post and in the Rice et al. 2021 mock juror experiment), so the estimated effect for racial resentment cannot be interpreted as only the effect of anti-Black animus. Racial resentment in these cases appears to sort to low levels of racial resentment a sufficient percentage of respondents who dislike Whites in absolute or at least relative terms, so that indifference to Whites might plausibly be better represented at some location between the ends of the racial resentment measure. But the racial resentment measure does not have a clear indifference point such as 50 on a 0-to-100 feeling thermometer rating, so -- even if argument [1] is addressed so that statistical control isolates the effect of racial attitudes -- it's not clear how racial resentment could be used to accurately estimate the effect of only anti-Black animus.

---

NOTES

1. The sexism measure used responses to the items below, which loaded onto one factor among White participants in the data:

[UMA306bSB] We should do all we can to make sure that women have the same opportunities in society as men.

[UMA306c] We would have fewer problems if we treated men and women more equally.

[UMA306f] Many women are actually  seeking special favors, such as hiring policies that favor them over men, under the guise of asking for "equality."

[UMA306g] Women are too easily offended.

[UMA306h] Men are better suited for politics than are women.

[CC18_422c] When women lose to men in a fair competition, they typically complain about being discriminated against.

[CC18_422d] Feminists are making entirely reasonable demands of men.

Responses to these items loaded onto a different factor:

[UMA306d] Women should be cherished and protected by men.

[UMA306e] Many women have a quality of purity that few men possess.

2. Data for Rice et al. 2021 from the JOP Dataverse. Original 2018 CCES data for the UMass-A module, which I used in the aforementioned analyses. Stata code. Stata output. Data and code for the sexism plot.

3. I plan a follow-up post about how well different measures predicted racial bias in the experiment.

Tagged with: , , ,

Forthcoming at the Journal of Politics is Rice et al. 2021 "Same As It Ever Was? The Impact of Racial Resentment on White Juror Decision-Making". In contrast to the forthcoming Peyton and Huber 2021 article at the Journal of Politics that I recently blogged about, Rice et al. 2021 reported evidence that racial resentment predicted discrimination among Whites.

---

Rice et al. 2021 concerned a mock juror experiment regarding an 18-year-old starting point guard on his high school basketball team who was accused of criminal battery. Participants indicated whether the defendant was guilty or not guilty and suggested a prison sentence length from 0 to 60 months for the defendant. The experimental manipulation was that the target was randomly assigned to be named Bradley Schwartz or Jamal Gaines.

Section 10 of the Rice et al. 2021 supplementary material has nice plots of the estimated discrimination at given levels of racial resentment, indicating, for the guilty outcome, that White participants at low racial resentment were less likely to indicate that Jamal was guilty compared to Bradley, but that White participants at high racial resentment were more likely to indicate that Jamal was guilty compared to Bradley. Results were similar for the sentence length outcome, but the 95% confidence interval at high racial resentment overlaps zero a bit.

---

The experiment did not detect sufficient evidence of racial bias among White participants as a whole. But what about Black participants? Results indicated a relatively large favoring of Jamal over Bradley among Black participants, in unweighted data (N=41 per condition). For guilt, the bias was 29 percentage points in unweighted analyses, and 33 percentage points in weighted analyses. For sentence length, the bias was 8.7 months in unweighted analyses, and 9.4 months in weighted analyses, relative to a unweighted standard deviation of 16.1 months in sentence length among Black respondents.

Results for the guilty/not guilty outcome:

Results for the mean sentence length outcome:

The p-value was under p=0.05 for my unweighted tests of whether the size of the discrimination among Whites (about 7 percentage points for guilty, about 1.3 months for sentence length) differed from the size of the discrimination among Blacks (about 29 percentage points for guilty, about 8.7 months for sentence length); the inference is the same for weighted analyses. The evidence is even stronger considering that the point estimate of discrimination among Whites was in the pro-Jamal direction and not in the pro-ingroup direction.

---

NOTES

1. Data for Rice et al. 2021 from the JOP Dataverse. Original 2018 CCES data for the UMass-A module, which I used in the aforementioned analyses. Stata code. Stata output. "Guilty" plot: data and R code. "Sentence length" plot: data and R code.

2. I plan to publish a follow-up post about evidence for validity of racial resentment from the Rice et al. 2021 results, plus a follow-up post about how well different measures predicted racial bias in the experiment.

Tagged with: , , ,

Electoral Studies recently published Jardina and Stephens-Dougan 2021 "The electoral consequences of anti-Muslim prejudice". Jardina and Stephens-Dougan 2021 reported results from 2004 through 2020 ANES Time Series Studies, estimating the effect of anti-Muslim prejudice on vote choice, among White Americans, using feeling thermometer ratings and responses on stereotype scales.

Figure 1 of Jardina and Stephens-Dougan 2021 reports non-Hispanic Whites' mean feeling thermometer ratings about Muslims, Whites, Blacks, Hispanics, and Asians...but not about Christian fundamentalists, even though ANES data for each year in Figure 1 contain feeling thermometer ratings about Christian fundamentalists.

The code for Jardina and Stephens-Dougan 2021 includes a section for "*Robustness for anti christian fundamental affect", indicating an awareness of the thermometer ratings about Christian fundamentalists.

I drafted a quick report about how reported 2020 U.S. presidential vote choice associated with feeling thermometer ratings about Jews, Christians, Muslims, and Christian fundamentalists, using data from the ANES 2020 Time Series Study. Plots are below, with more detailed descriptions in the quick report.

This first plot is of the distributions of feeling thermometer ratings about the religious groups asked about, with categories such as [51/99] indicating the percentage that rated the indicated group at 51 through 99 on the thermometer:

This next plot is of how the ratings about a given religious group associated with 2020 two-party presidential vote choice for Trump, with demographic controls only, and a separate regression for ratings about each religious group:

This next plot added controls for partisanship, political ideology, and racial resentment, and put all ratings of religious groups into the same regression:

The above plot zooms in on y-axis percentages from 20 to 60. The plot in the quick report has a y-axis that runs from 0 to 100.

---

Based on a Google Scholar search, research is available about the political implications of attitudes about Christian fundamentalists, such as Bolce and De Maio 1999. I'll plan to add a discussion of this if I convert the quick report into a proper paper.

---

The technique in the quick report hopefully improves on the Jardina and Stephens-Dougan 2021 technique for estimating anti-Muslim prejudice. From Jardina and Stephens-Dougan 2021 (p. 5):

A one-unit change on the anti-Muslim affect measure results in a 16-point colder thermometer evaluation of Kerry in 2004, a 22-point less favorable evaluation of Obama in both 2008 and 2012, and a 17-point lower rating of Biden in 2020.

From what I can tell, this one-unit change is the difference between estimated support for a candidate, net of controls, comparing a 0 rating about Muslims on the feeling thermometers to a 100 rating about Muslims on the feeling thermometers, based on a regression in which the "Negative Muslim Affect" predictor was merely the set of feeling thermometer ratings about Muslims reversed and placed on a 0-to-1 scale.

If so, then the estimated effect size of anti-Muslim affect is identical to the estimated effect size of pro-Muslim affect. Or maybe Jardina and Stephens-Dougan 2021 considers rating Muslims at 100 to be indifference about Muslims, 99 indicates some anti-Muslim affect, 98 indicates a bit more anti-Muslim affect, and so on.

It seems more reasonable to me that some people are on net indifferent about Muslims, some people have on net positive absolute views about Muslims, and some people have on net negative absolute views about Muslims. So instead I coded feeling thermometer ratings for each religious group into six categories: zero (the coldest possible rating), 100 (the warmest possible rating), 1 through 49 (residual cold ratings), 50 (indifference), 51 through 99 (residual warm ratings), and non-responses.

The extreme categories of 0 and 100 are to estimate the outcome at the extremes, and the 50 category is to estimate the outcome at indifference. If the number of observations at the extremes is not sufficiently large for some predictors, it might make more sense to also collapse the extreme value into adjoining values on the same side of 50.

---

NOTES

1. Jardina and Stephens-Dougan 2021 footnote 24 has an unexpected-to-me criticism of Michael Tesler's work.

We note that our findings with respect to 2012 are not consistent with Tesler (2016a), who finds that anti-Muslim attitudes were predictive of voting for Obama in 2012. Tesler, however, does not control for economic evaluations in his vote choice models, despite the fact that attitudes toward the economy are notoriously important predictors of presidential vote choice (Vavreck 2009)...

I don't think that a regression should include a predictor merely because the predictor is known to be a good predictor of the outcome, so it's not clear to me that Tesler or anyone else should include participant economic evaluations when predicting vote choice merely because participant economic evaluations predict vote choice.

It seems plausible that a nontrivial part of participant economic evaluations are downstream from attitudes about the candidates. Tesler's co-authored Identity Crisis book has a plot (p. 208) illustrating the flip-flop by Republicans and Democrats on views of the economy from around November 2016, with a note that:

This is another reason to downplay the role of subjective economic dissatisfaction in the election: it was largely a consequence of partisan politics, not a cause of partisans' choices.

2. Jardina and Stephens-Dougan 2021 indicated that (p. 5):

The fact, however, that the effect size of anti-Muslim affect is often on par with the effect size of racial resentment is especially noteworthy, given that the construct is measured far less robustly than the multi-item measure of racial resentment.

The anti-Muslim affect measure is a reversed 0-to-100 feeling thermometer, which has 101 potential levels. Racial resentment is built from four items, with each item having five substantive options, so that would permit the creation of a measure that has 17 substantive levels, not counting any intermediate levels that might occur for participants with missing data for some but not all of the four items.

I'm not sure why it's particularly noteworthy that the estimated effect for the 101-measure scale is on par with the estimated effect for the 17-level measure. From what I can tell, these measures are not easily comparable, unless we know, for example, the percentage of participants that fell into the most extreme levels.

3. Jardina and Stephens-Dougan 2021 reviewed a lot of the research on the political implications about attitudes about Muslims. But no mention of Helbling and Traunmüller 2018, which, based on data from the UK, indicated that:

The results suggest that Muslim immigrants are not per se viewed more negatively than Christian immigrants. Instead, the study finds evidence that citizens' uneasiness with Muslim immigration is first and foremost the result of a rejection of fundamentalist forms of religiosity.

4. I have a prior post about selective reporting in the 2016 JOP article from Stephens-Dougan, the second author of Jardina and Stephens-Dougan 2021.

5. Quick report. Stata code. Stata output.

Tagged with: , ,