L.J Zigerell

By Posted on Posted in Public opinion No Comments

The ANES (American National Election Studies) has released the pre- and post-election questionnaires for its 2020 Time Series Study. I thought that it would be useful or at least interesting to review the survey for political bias. I think that the survey is remarkably well done on net, but I do think that ANES 2020 contains unnecessary political bias.

---

1

ANES 2020 has two gender resentment items on the pre-election survey and two modern sexism items on the post-election survey. These four items are phrased to measure negative attitudes about women, but ANES 2020 has no parallels to these four items regarding negative attitudes about men.

Even if researchers cared about only sexism against women, parallel measures of attitudes about men would still be necessary. Evidence indicates and theory suggests that participants sexist against men would cluster at the low end of a measure of sexism against women, so that sexism against women can't properly be estimated as the change from low level to high level of these measures.

This lack of parallel items about men will plausibly produce a political bias in research that uses these four items as measures of sexism, because, while a higher percentage of Republicans than of Democrats is biased against women, a higher percentage of Democrats than of Republicans is biased against men (evidence about partisanship is in in-progress research, but check here about patterns in the 2016 presidential vote).

ANES 2020 has a feeling thermometer for several racial groups, so hopefully future ANES surveys include feeling thermometers about men and women.

---

2

Another type of political bias involves inclusion of response options so that the item can detect only errors more common on the political right. Consider this post-election item labeled "misinfo":

1. Russia tried to interfere in the 2016 presidential election

2. Russia did not try to interfere in the 2016 presidential election

So the large percentage of Hillary Clinton voters who reported the belief that Russia tampered with vote tallies to help Donald Trump don't get coded as misinformed on this misinformation item about Russian interference. The only error that the item can detect is underestimating Russian interference.

Another "misinfo" example:

Which of these two statements do you think is most likely to be true?

1. World temperatures have risen on average over the last 100 years.

2. World temperatures have not risen on average over the last 100 years.

The item permits climate change "deniers" to be coded as misinformed, but does not permit coding as misinformed "alarmists" who drastically overestimate how much the climate has changed over the past 100 years.

Yet another "misinfo" example:

1. There is clear scientific evidence that the anti-malarial drug hydroxychloroquine is a safe and effective treatment for COVID-19.

2. There is not clear scientific evidence that the anti-malarial drug hydroxychloroquine is a safe and effective treatment for COVID-19.

In April 2020, the FDA indicated that "Hydroxychloroquine and chloroquine...have not been shown to be safe and effective for treating or preventing COVID-19", so the "deniers" who think that there is zero evidence available to support HCQ as a covid-19 treatment will presumably not be coded as "misinformed".

One more example (not labeled "misinfo"), from the pre-election survey:

During the past few months, would you say that most of the actions taken by protestors to get the things they want have been violent, or have most of these actions by protesters been peaceful, or have these actions been equally violent and peaceful?

[If the response is "mostly violent" or "mostly peaceful":]

Have the actions of protestors been a lot more or only a little more [violent/peaceful]?

I think that this item might refer to the well-publicized finding that "about 93% of racial justice protests in the US have been peaceful", so that the correct response combination is "mostly peaceful"/"a lot more peaceful" and, thus, the only error that the item permits is overestimating how violent the protests were.

For the above items, I think that the response options disfavor the political right, because I expect that a higher percentage of persons on the political right than the political left will deny Russian interference in the 2016 presidential election, deny climate change, overestimate the evidence for HCQ as a covid-19 treatment, and overestimate how violent recent pre-election protests were.

But I also think that persons on the political left will be more likely than persons on the political right to make the types of errors that the items do not permit to be measured, such as overestimating climate change over the past 100 years.

Other items marked "misinfo" involved vaccines causing autism, covid-19 being developed intentionally in a lab, and whether the Obama administration or the Trump administration deported more unauthorized immigrants during its first three years.

I didn't see an ANES 2020 item about whether the Obama administration or the Trump administration built the temporary holding enclosures ("cages") for migrant children, which I think would be similar to the deportations item, in that people not paying close attention to the news might get the item incorrect.

Maybe a convincing case could be made that ANES 2020 contains an equivalent number of items with limited response options disfavoring the political left as disfavoring the political right, but I don't think that it matters whether political bias in individual items cancels out, because any political bias in individual items is worth eliminating, if possible.

---

3

ANES 2020 has an item that I think alludes to President's Trump's phone call with the Ukrainian president. Here is a key passage from the transcript of the call:

The other thing, There's a lot of talk about Biden's son, that Biden stopped the prosecution and a lot of people want to find out about that so whatever you can do with the Attorney General would be great. Biden went around bragging that he stopped the prosecution so if you can look into it...It sounds horrible to me.

Here is an ANES 2020 item:

As far as you know, did President Trump ask the Ukrainian president to investigate President Trump's political rivals, did he not ask for an investigation, or are you not sure?

I'm presuming that the intent of the item is that a correct response is that Trump did ask for such an investigation. But, if this item refers to only Trump asking the Ukrainian president to look into a specific thing that Joe Biden did, it's inaccurate to phrase the item as if Trump asked the Ukrainian president to investigate Trump's political rivals *in general*, which is what the plural "rivals" indicates.

---

4

I think that the best available evidence indicates that immigrants do not increase the crime rate in the United States (pre-2020 citation) and that illegal immigration reduces the crime rate in the United States (pre-2020 citation). Here is an "agree strongly" to "disagree strongly" item from ANES 2020:

Immigrants increase crime rates in the United States.

Another ANES 2020 item:

Does illegal immigration increase, decrease, or have no effect on the crime rate in the U.S.?

I think that the correct responses to these items are the responses that a stereotypical liberal would be more likely to *want* to be true, compared to a stereotypical Trump supporter.

But I don't think that the U.S. violent crime statistics by race reflect the patterns that a stereotypical liberal would be more likely to want to be true, compared to a stereotypical Trump supporter.

Perhaps coincidentally, instead of an item about racial differences in violent crime rates for which responses could be correctly described as consistent or inconsistent with available mainstream research, ANES 2020 has stereotype items about how "violent" different racial groups are in general, which I think survey researchers will be much less likely to perceive to be addressed in mainstream research and will instead use to measure racism.

---

The above examples of what I think are political biases are relatively minor in comparison to the value that ANES 2020 looks like it will provide. For what it's worth, I think that the ANES is preferable to the CCES Common Content.

Tagged with: , , , ,

By Posted on Posted in Race No Comments

The Journal of Race, Ethnicity, and Politics published Buyuker et al 2020: "Race politics research and the American presidency: thinking about white attitudes, identities and vote choice in the Trump era and beyond".

Table 2 of Buyuker et al 2020 reported regressions predicting Whites' projected and recalled vote for Donald Trump over Hillary Clinton in the 2016 U.S. presidential election, using predictors such as White identity, racial resentment, xenophobia, and sexism. Xenophobia placed into the top tier of predictors, with an estimated maximum effect of 88 percentage points going from the lowest to the highest value of the predictor, and racial resentment placed into the second tier, with an estimated maximum effect of 58 percentage points.

I was interested in whether this difference is at least partly due to how well each predictor was measured. Here are characteristics of the predictors among Whites, which indicate that xenophobia was measured at a much more granular level than racial resentment was:

RACIAL RESENTMENT
4 items
White participants fell into 22 unique levels
4% of Whites at the lowest level of racial resentment
9% of Whites at the highest level of racial resentment

XENOPHOBIA
10 items
White participants fell into 1,096 unique levels
1% of Whites at the lowest level of xenophobia
1% of Whites at the highest level of xenophobia

So it's at least plausible from the above results that xenophobia might have outperformed racial resentment merely because the measurement of xenophobia was better than the measurement of racial resentment.

---

Racial resentment was measured with four items that each had five response options, so I created a reduced xenophobia predictor using the four xenophobia items that each had exactly five response options; these items were about desired immigration levels and agreement or disagreement with statements that "Immigrants are generally good for America's economy", "America's culture is generally harmed by immigrants", and "Immigrants increase crime rates in the United States".

I re-estimated the Buyuker et al 2020 Table 2 model replacing the original xenophobia predictor with the reduced xenophobia predictor: the maximum effect for xenophobia (66 percentage points) was similar to the maximum effect for racial resentment (66 percentage points).

---

Among Whites, vote choice correlated between r=0.50 and r=0.58 with each of the four racial resentment items and between r=0.39 and r=0.56 with nine of the ten xenophobia items. The exception was the seven-point item that measured attitudes about building a wall on the U.S. border with Mexico, which correlated with vote choice at r=0.72.

Replacing the desired immigration levels item in the reduced xenophobia predictor with the border wall item produced a larger estimated maximum effect for xenophobia (85 percentage points) than for racial resentment (60 percentage points). Removing all predictors from the model except for xenophobia and racial resentment, the reduced xenophobia predictor with the border wall item still produced a larger estimated maximum effect than did racial resentment: 90 percentage points, compared to 74 percentage points.

But the larger effect for xenophobia is not completely attributable to the border wall item: using a predictor that combined the other nine xenophobia items produced a maximum effect for xenophobia (80 percentage points) that was larger than the maximum effect for racial resentment (63 percentage points).

---

I think that the main takeaway from this post is that, when comparing the estimated effect of predictors, inferences can depend on how well each predictor is measured, so such analyses should discuss the quality of the predictors. Imbalances in which participants fall into 22 levels for one predictor and 1,096 levels for another predictor seem to be biased in favor of the more granular predictor, all else equal.

Moreover, I think that, for predicting 2016 U.S. presidential vote choice, it's at least debatable whether a xenophobia predictor should include an item about a border wall with Mexico, because including that item means that, instead of xenophobia measuring attitudes about immigrants per se, the xenophobia predictor conflates these attitudes with attitudes about a policy proposal that is very closely connected with Donald Trump.

---

It's not ideal to use regression to predict maximum effects, so I estimated a model using only the racial resentment predictor and the reduced four-item xenophobia predictor with the border wall item, but including a predictor for each level of the predictors. That model predicted failure perfectly for some levels of the predictors, so I recoded the predictors until those errors were eliminated, which involved combining the three lowest racial resentment levels (so that racial resentment ran from 2 through 16) and combining the 21st and 22nd levels of the xenophobia predictor (so that xenophobia ran from 0 through 23). In a model with only those two recoded predictors, the estimated maximum effects were 81 percentage points for xenophobia and 76 percentage points for racial resentment. Using all Buyuker et al 2020 predictors, the respective percentage points were 65 and 63.

---

I then predicted Trump/Clinton vote choice using only the 22-level racial resentment predictor and the full 1,096-level xenophobia predictor, but placing the values of the predictors into ten levels; the original scale for the predictors ran from 0 through 1, and, for the 10-level predictors, the first level for each predictor was from 0 to 0.1, a second level was from above 0.1 to 0.2, and a tenth level was from above 0.9 to 1. Using these predictors as regular predictors without "factor" notation, the gap in maximum effects was about 24 percentage points, favoring xenophobia. But using these predictors with "factor" notation, the gap favoring xenophobia fell to about 9.5 percentage points.

Plots below illustrate the difference in predictions for xenophobia: the left panel uses a regular 10-level xenophobia predictor, and the right panel uses each of the 10 levels of that predictor as a separate predictor.

---

So I'm not sure that these data support the inference that xenophobia is in a higher tier than racial resentment, for predicting Trump/Clinton vote in 2016. The above analyses seem to suggest that much or all of the advantage for xenophobia over racial resentment in the Buyuker et al 2020 analyses was due to model assumptions and/or better measurement of xenophobia.

---

Another concern about Buyuker et al 2020 is with the measurement of predictors such as xenophobia. The xenophobia predictor is more accurately described as something such as attitudes about immigrants. If some participants are more favorable toward immigrants than toward natives, and if these participants locate themselves at low levels of the xenophobia predictor, then the effect of xenophilia among these participants is possibly being added to the effect of xenophobia.

Concerns are similar for predictors such as racial resentment and sexism. See here and here for evidence that low levels of similar predictors associate with bias in the opposite direction.

---

NOTES

1. Thanks to Beyza Buyuker for sending me replication materials for Buyuker et al 2020.

2. Stata code for my analyses. Stata output for my analyses.

3. ANES 2016 citations:

The American National Election Studies (ANES). 2016. ANES 2012 Time Series Study. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2016-05-17. https://doi.org/10.3886/ICPSR35157.v1.

ANES. 2017. "User's Guide and Codebook for the ANES 2016 Time Series Study". Ann Arbor, MI, and Palo Alto, CA: The University of Michigan and Stanford University.

Tagged with: , , , ,

By Posted on Posted in Race No Comments

Political Research Quarterly published Garcia and Stout 2020 "Responding to Racial Resentment: How Racial Resentment Influences Legislative Behavior". The article abstract indicates (emphasis added):

Through an automated content analysis of more than fifty four thousand press releases from almost four hundred U.S. House members in the 114th Congress (2015–2017), we show that Republicans from districts with high levels of racial resentment are more likely to issue press releases that attack President Barack Obama. In contrast, we find no evidence of racial resentment being positively associated with another prominent Democratic white elected official, Hillary Clinton. Our results suggest that one reason Congress may remain racially conservative even as representatives' cycle out of office may be attributed to the electoral process.

---

Racial resentment conflates racial attitudes and political ideology, apparently even when controlling for factors such as partisanship and political ideology, so comparing how district racial resentment predicts the percentage of press releases attacking Barack Obama to how district racial resentment predicts the percentage of press releases attacking Hillary Clinton is a useful way to assess whether any association of racial resentment is due to the racial component of district racial resentment. But that comparison should involve a statistical test of whether the coefficient for district racial resentment in the Obama models differs from the coefficient for district racial resentment in the Clinton models. And my analyses indicate that, for results reported in the article, these coefficients don't differ at p<0.20.

My analyses indicated that the p-value is p=0.23 for a test of whether the coefficient on district racial resentment in an Obama model differs from the coefficient on district racial resentment in a Clinton model, using only a predictor of weighted district racial resentment and limiting the sample to Republican representatives. The p-value is about p=0.33 for a test comparing the key interaction coefficients in Models C and D in Garcia and Stout 2020 Table 1 (see the plot below).

---

The Garcia and Stout 2020 abstract's claim that "…we find no evidence of racial resentment being positively associated with another prominent Democratic white elected official, Hillary Clinton" (p. 812) is contradicted in the main text of Garcia and Stout 2020:

Pearson's R for the relationship between the unweighted (.16) and the weighted (.14) district's racial resentment score and Republicans issuing of negative-Clinton press releases are statistically significant at .05.

I think that the "no evidence" claim refers to the lack of statistical significance for the Clinton models in Table 1 when adding statistical control, but the coefficient/standard error ratio is about 1.3 for the key coefficient for Clinton in Table 1 Model D, so that's some evidence. Adding "cluster(robust)" to the regression specification increases this t-statistic to 1.78, which is not no evidence. And then removing the control for candidate margin of victory gets the p-value under p=0.05.

---

For the outcome variable codings of the percentage of press releases attacking Obama and the percentage of press releases attacking Clinton, 53% and 76% of the observations are zero, respectively. The outcome is a percentage, so I re-estimated the models using fractional logistic regression. As indicated in the output, the p-value for the interaction coefficient did not fall under p=0.15 in the Clinton models mentioned above or in the Obama models mentioned above.

---

The key Table 1 coefficient is an interaction term that involves district racial resentment and the political party of the representative, but the abstract claims are limited to Republican representatives. I estimated the Table 1 models limited to Republican representatives: the p-value for racial resentment did not fall under p=0.80 for the Obama models. The p-value for racial resentment did not fall under p=0.40 for the Obama models limited to non-Republican representatives.

---

So, in the fractional regression models discussed above, the key Table 1 interaction coefficient did not have a p-value under p=0.15; in the linear regression models discussed above with statistical control, district racial resentment did not predict at p<0.80 among Republican representatives the percentage of press releases attacking Obama; and in the linear regression models discussed above, the association of district racial resentment and the percentage of press releases attacking Obama did not differ at p<0.20 from the association of district racial resentment and the percentage of press releases attacking Clinton.

---

NOTES

1. Thanks to Jennifer R. Garcia for sending me data for the article.

2. Results reported in the post are for the weighted models, but the Stata output contains results for unweighted models, in which the inferences or a lack of inferences are the same or similar.

3. Stata code. Stata output. R code for the plot.

4. For what it's worth, Republican members of Congress from districts with relatively low levels of racial resentment were more likely to issue press releases that attacked Obama than to issue press releases that attacked Clinton, measuring low district racial resentment as the bottom 10% of GOP districts by racial resentment; the same pattern held for Republican members of Congress from districts with relatively high levels of racial resentment, measured as the top 10% of GOP districts by racial resentment. Stata code for this analysis. Stata output for this analysis.

Tagged with: , ,

By Posted on Posted in Race, Sex No Comments

Participants in studies reported on in Regina Bateson's 2020 Perspectives on Politics article "Strategic Discrimination" were asked to indicate the percentage of other Americans that the participant thought would not vote for a woman for president and the percentage of other Americans that the participant thought would not vote for a black person for president.

Bateson 2020 Figure 1 reports that, in the nationally representative Study 1 sample, mean participant estimates were that 47% of other Americans would not vote for a woman for president and that 42% of other Americans would not vote for a black person for president. I was interested in the distribution of responses, so I plotted in the histograms below participant estimates to these items, using the Bateson 2020 data for Study 1.

This first set of histograms is for all participants:

This second set of histograms is for only participants who passed the attention check:

---

I was also interested in estimates from participsnts with a graduate degree, given that so many people in political science have a graduate degree. Bateson 2020 Appendix Table 1.33 indicates that, among participants with a graduate degree, estimates were that 58.3% of other Americans would not vote for a woman for president and that 56.6% of other Americans would not vote for a black person for president.

But these estimates differ depending on whether the participant correctly responded to the attention check item: for the item about the percentage of other Americans who would not vote for a woman for president, the mean estimate was 47% [42, 52] for the 84 graduate degree participants who correctly responded to the attention check and was 68% [63, 73] for the 97 graduate degree participants who did not correctly respond to the attention check; for the item about the percentage of other Americans who would not vote for a black person for president, respective estimates were 44% [39, 49] and 67% [62, 73].

Participants who reported having a graduate degree were 20 percentage points more likely to fail the attention check than participants who did not report having a graduate degree, p<0.001.

---

These data were collected in May 2019, after Barack Obama had been elected president twice and after Hillary Clinton won the popular vote for president, and each aforementioned mean estimate seems to be a substantial overestimate of discrimination against women presidential candidates and Black presidential candidates, compared to point estimates from relevant list experiments reported in Carmines and Schmidt 2020 and compared to point estimates from list experiments and direct questions cited in Bateson 2020 footnote 8.

---

NOTES

1. Stata code for my analysis.

2. R code for the first histogram.

Tagged with: , ,

By Posted on Posted in Sex, Student evaluations of teaching, Teaching No Comments

In May 2020, PS published a correction to Mitchell and Martin 2018 "Gender Bias in Student Evaluations", which reflected concerns that I raised in a March 2019 blog post. That correction didn't mention me, and in May 2020 PS published another correction that didn't mention me but was due to my work, so I'll note below evidence that the corrections were due to my work, which might be useful in documenting my scholarly contributions for, say, an end-of-the-year review or promotion application.

---

In August 2018, I alerted the authors of Mitchell and Martin 2018 (hereafter MM) to concerns about potential errors in MM. I'll post one of my messages below. My sense at the time was that the MM authors were not going to correct MM (and the lead author of MM was defending MM as late as June 2019), so I published a March 2019 blog post about my concerns and in April 2019 I emailed PS a link to my blog post and a suggestion that MM "might have important errors in inferential statistics that warrant a correction".

In May 2019, a PS editor indicated to me that the MM authors have chosen to not issue a correction and that PS invited me to submit a comment on MM that would pass through the normal peer review process. I transformed my blog post into a manuscript comment, which involved, among other things, coding all open-ended student evaluation comments and calculating what I thought the correct results should be in the main three MM tables. Moreover, for completeness, I contacted Texas Tech University and eventually filed a Public Information Act request, because no one I communicated with at Texas Tech about this knew for certain why student evaluation data were not available online for certain sections of the course that MM Table 4 reported student evaluation results for.

I submitted a comment manuscript to PS in August 2019 and submitted a revision based on editor feedback in September 2019. Here is the revised submitted manuscript. In January 2020, I received an email from PS indicating that my manuscript was rejected after peer review and that PS would request a corrigendum from the authors of MM.

In May 2020, PS published a correction to MM, but I don't think that the correction is complete: for example, as I discussed in my blog post and manuscript comment, I think that the inferential statistics in MM Table 4 were incorrectly based on a calculation in which multiple ratings from the same student were treated as independent ratings.

---

For the Comic-Con correction that PS issued in May 2020, I'll quote from my manuscript documenting the error of inference in the article:

I communicated concerns about the Owens et al. 2020 "Comic-Con" article to the first two authors in November 2019. I did not hear of an attempt to publish a correction, and I did not receive a response to my most recent message, so I submitted this manuscript to PS: Political Science & Politics on Feb 4, 2020. PS published a correction to "Comic-Con" on May 11, 2020. PS then rejected my manuscript on May 18, 2020 "after an internal review".

Here is an archive of a tweet thread, documenting that in September 2019 I alerted the lead "Comic-Con" author to the error of inference, and the lead author did not appear to understand my point.

---

NOTES:

1. My PS symposium entry "Left Unchecked" (published online in June 2019) discussed elements of MM that ended up being addressed in the MM correction.

2. Here is an email that I sent the MM authors in August 2018:

Thanks for the data, Dr. Mitchell. I had a few questions, if you don't mind:

[1] The appendix indicates for the online course analysis that: "For this reason, we examined sections in the mid- to high- numerical order: sections 6, 7, 8, 9, and 10". But I think that Dr. Martin taught a section 11 course (D11) that was included in the data.

[2] I am not certain about how to reproduce the statistical significance levels for Tables 1 and 2. For example, for Table 1, I count 23 comments for Dr. Martin and 45 comments for Dr. Mitchell, for the N=68 in the table. But a proportion test in Stata for the "Referred to as 'Teacher'" proportions (prtesti 23 0.152 45 0.244) produces a z-score of -0.8768, which does not seem to match the table asterisks indicating a p-value of p<0.05.

[3] Dr. Martin's CV indicates that he was a visiting professor at Texas Tech in 2015 and 2016. For the student comments for POLS 3371 and POLS 3373, did Dr. Martin's official title include "professor"? If so, than that might influence inferences about any difference in the frequency of student use of the label "professor" between Dr. Martin and Dr. Mitchell. I didn't see "professor" as a title in Dr. Mitchell's CV, but the inferences could also be influenced if Dr. Mitchell had "professor" in her title for any of the courses in the student comments analysis, or for the Rate My Professors comments analysis.

[4] I was able to reproduce the results for the Technology analysis in Table 4, but, if I am correct, the statistical analysis seems to assume that the N=153 for Dr. Martin and the N=501 for Dr. Mitchell are for 153 and 501 independent observations. I do not think that this is correct, because my understanding of the data is that the 153 observations for Dr. Martin are 3 observations for 51 students and that the 501 observations for Dr. Mitchell are 3 observations for 167 students. I think that the analysis would need to adjust for the non-independence of some of the observations.

Sorry if any of my questions are due to a misunderstanding. Thank you for your time.

Best,

L.J

Tagged with: , ,

By Posted on Posted in Race No Comments

Study 1 of Strickler and Lawson 2020 "Racial conservatism, self-monitoring, and perceptions of police violence" in Politics, Groups, and Identities was an experiment in which participants rated how justified a police shooting was. The experiment had a control condition, a "stereotype" condition in which the officer was White and the suspect Black, and a "counterstereotype" condition in which the officer was Black and the suspect White.

The article indicates that:

And while racial resentment did not moderate how whites responded to treatment in the White Officer/Black Victim condition, it did impact response to treatment in the Black Officer/White Victim condition. As Table 3 and Figure 4 demonstrate, for whites, those with higher levels of racial resentment are significantly less likely to view shooting as justified if it involves a black officer and a white victim.

However, the 95% confidence interval in the aforementioned Figure 4 crosses zero at high levels of racial resentment. I emailed lead author Ryan Strickler for the data and code, which he provided.

---

Instead of using a regression to estimate the outcome at higher levels of racial resentment, I'll estimate the outcome for only participants at given ranges of racial resentment (see Hainmueller et al. 2019). This way, inferences about particular groups are based on data for only those groups.

Plots below report point estimates and 95% confidence intervals from tests comparing the outcome across conditions, at various ranges of racial resentment, among all White respondents or among Whites who responded correctly to manipulation checks about the officer's race and the suspect's race. Racial resentment was coded from 1 through 17.

The outcome for the first four plots was whether the participant indicated that the officer's actions were justified.

In the top left plot, the top estimate is for White participants at the highest observed level of racial resentment. The estimate is positive 0.06, which indicates that high racial resentment participants in the stereotypic condition were 6 percentage points more likely to rate the shooting as justified, compared to high racial resentment participants in the counterstereotypic condition; however, the 95% confidence interval crosses zero. The next lower estimate compared outcomes for White participants at a racial resentment of 16 and 17. The bottom estimate (RR>=1) is for all White participants, and the negative point estimate for this bottom estimate indicates that White participants in the counterstereotypic shooting condition were more likely to rate the shooting as justified, compared to White participants in the stereotypic shooting condition.

The evidence for bias among Whites high in racial resentment is a bit stronger in the right panels, which compared the counterstereotypic condition to the control condition, but the 95% confidence intervals still overlap zero. There is an exception among White participants who scored 14 or higher on the racial resentment scale, when excluding participants who did not pass the post-treatment manipulation check, but it's not a good idea to exclude participants after the treatment.

---

Tables in the main text of Strickler and Lawson 2020 reported results for a dichotomous outcome coded 1 if, for the first item of the branching, the respondent indicated that the officer's actions were justified. But tables in the appendix used ratings of the extent to which the shooting was justified, measured using branched items that placed respondents into nine levels, from "a great deal certain" that the shooting was not justified to "a great deal certain" that the shooting was justified.

The plots below report results from tests that compared conditions for this ordinal measure of justification, placed on a 0-to-1 scale. Evidence in the right panel is a bit stronger using this outcome, compared to the dichotomous outcome. Like before, the top estimate is for White participants at the highest observed level of racial resentment. Middle estimates (RR>=1 and RR<=17) are for all Whites; below that, estimates are for more extreme levels of low racial resentment, ending with RR==1, for White participants at the lowest observed level of racial resentment.

Results for Whites who passed the manipulation checks are in the output file.

---

NOTES

1. Thanks to Ryan Strickler for sending me data and code for the article.

2. Stata code and R code for my analyses. Data for the first four plots. Data for the final two plots.

Tagged with: ,

By Posted on Posted in Race No Comments

The American National Election Studies Time Series Cumulative Data File (1948-2016) contains data for feeling thermometer measures for Whites and for Blacks, collected in face-to-face or telephone interviews, for each U.S. presidential election year from 1964 to 2016.

Feeling thermometers range from 0 to 100, with higher values indicating warmer or more favorable feelings about a group. The ANES Cumulative Data File and some early individual year ANES Time Series files collapse responses of 97 through 100 into a response of 97. This means that a respondent who selected 97 for Whites and 100 for Blacks would have the same "difference" value as a respondent who selected 100 for Whites and 97 for Blacks. Therefore, I placed respondents with a substantive value for the feeling thermometer about Whites and the feeling thermometer about Blacks into one of three categories:

  • rated Whites more than 3 units above Blacks
  • rated Whites within 3 units of Blacks, and
  • rated Blacks more than 3 units above Whites.

Abrajano and Alvarez (2019) reported evidence from ANES Time Series Studies that responses to racial feeling thermometers differed between the non-internet mode and the internet mode, so my reported results do not include results from the internet mode, which do not go back to 1964.

Below is a plot of how Whites Americans (left) and Black Americans (right) fell into each of the three categories, not including the respondents in the cumulative data file who did not report a substantive response to the items, which ranged from 1% to 8% (see the Notes). Documentation for the cumulative data file indicated that in 1964 and 1968 a response was recorded as 50 for a "don't know" response or if the participant indicated that the participant did not know too much about a group.

---

The plot below indicates how these thermometer ratings associated with two-party vote choice, among White participants:

The right panel indicates a steep drop in two-party vote for the Republican presidential candidate among Whites who rated Blacks more than 3 units higher than Whites, which seems to be consistent with evidence of a "Great Awokening" (see, e.g., Yglesias 2019 and Goldberg 2019, and this image linked to in Goldberg 2019).

---

The plot below is the plot above, but with columns grouped by year:

---

NOTES

1. Percentage non-responses to one or both thermometer items, by year: 3% (1964), 4% (1968), 8% (1972), 5% (1976), 5% (1980), 7% (1984), 5% (1988), 4% (1992), 4% (1996), 8% (2000), 3% (2004), 3% (2008), 1% (2012), 2% (2016).

2. Code for my analyses and black-and-white plots.

3. Feeling thermometer ratings about Chicanos/Hispanics and about Asians are not available in ANES Time Series Cumulative Data File until 1976 and 1992, respectively.

4. A color version of the first plot, for comparison:

5. A color version with a black line divider:

Tagged with: