Home › Posts tagged public opinion

Tag: public opinion

By Posted on Posted in Race No Comments

The plot below is based on data from the ANES 2022 Pilot Study, plotting the percentage of particular populations that rated the in-general intelligence of Whites higher than the in-general intelligence of Blacks (black dots) and the percentage of these populations that rated the in-general intelligence of Asians higher than the in-general intelligence of Whites (white dots). For the item wording, see the notes below or page 44 of the questionnaire.

My understanding is that, based on a straightforward / naïve interpretation of educational data such as NAEP scores as good-enough measures of intelligence [*], there isn't much reason to be in the white dot and not in the black dot or vice versa. But, nonetheless, there is a gap between dots in the overall population and in certain populations.

In the plot above, estimated percentages are similar among very conservative Whites and among U.S. residents who attributed to biological differences at least some of the Black-American/Hispanic-American-vs-White-American difference in outcomes in things such as jobs and income. But similar percentages can mask inconsistencies.

For example, among U.S. residents who attributed to biological differences at least some of the Black-American/Hispanic-American-vs-White-American difference in outcomes in things such as jobs and income, about 37% rated Asians' intelligence higher than Whites' intelligence, about 34% rated Whites' intelligence higher than Blacks' intelligence, but only about 14% fell into both of these groups, as illustrated in the second panel below:

The plot below indicates corresponding comparisons for the estimated percentages that rated the in-general intelligence of Whites higher than the in-general intelligence of Blacks (black dots) and the percentage of these populations that rated the in-general intelligence of Asians higher than the in-general intelligence of Blacks (white dots).

---

[*] I can imagine reasons to not be in one or both dots, such as perceptions about the influence of past or present racial discrimination, the relative size of the gaps, flaws in the use of educational data as measures of intelligence, and imperfections in the wording of the ANES item. But I nonetheless thought that it would be interesting to check respondent ratings about racial group intelligence.

---

NOTES

1. Relevant item wording from the ANES 2022 Pilot Study:

Next, we're going to show you a seven-point scale on which the characteristics of the people in a group can be rated. In the first statement a score of '1' means that you think almost all of the people in that group tend to be intelligent.' A score of '7' means that you think most people in the group are 'unintelligent.' A score of '4' means that you think that most people in the group are not closer to one end or the other, and of course, you may choose any number in between. Where would you rate each group in general on this scale?

2. The ANES 2022 Pilot Study had a parallel item about Hispanic-Americans that I didn't analyze, to avoid complicating the presentation.

3. In the full sample, weighted, 13% rated in-general Black intelligence higher than in-general White intelligence (compared to 25% the other way), 8% rated in-general Black intelligence higher than in-general Asian intelligence (compared to 38% the other way), and 10% rated in-general White intelligence higher than in-general Asian intelligence (compared to 35% the other way). Respective equal ratings of in-general intelligence were 62% White/Black, 54% Asian/Black, and 55% Asian/White.

Respondents were coded into a separate category if the respondent didn't provide a rating of intelligence for at least one of the racial groups in a comparison, but almost all respondents provided a rating of intelligence for each racial group.

4. Plots created with R packages: tidyverse, waffle, and patchwork.

5. Data for the ANES 2022 Pilot Study. Stata code and output for my analysis.

6. An earlier draft of the first plot is below, which I didn't like as much, because I thought that it was too wide and not as visually attractive:

7. The shading in the plot below is intended to emphasize the size of the gaps between the estimates within a population, with red indicating reversal of the typical pattern:

8. Plot replacing the legend with direct labels:

9. Bonus plot, while I'm working on visualizations, with this plot comparing ratings about men and women on 0-to-100 feeling thermometers, with confidence intervals for each category, as if the category were plotted as its own percentage:

Tagged with: , , , , ,

By Posted on Posted in Race No Comments

1.

In May, I published a blog post about deviations from the pre-analysis plan for the Stephens-Dougan 2022 APSR letter, and I tweeted a link to the blog post that tagged @LaFleurPhD and asked her directly about the deviations from the pre-analysis plan. I don't recall receiving a response from Stephens-Dougan, and, a few days later, on May 31, I emailed the APSR about my post, listing three concerns:

* The Stephens-Dougan 2022 description of racially prejudiced Whites not matching how the code for Stephens-Dougan 2022 calculated estimates for racially prejudiced Whites.

* The substantial deviations from the pre-analysis plan.

* Figure 1 of the APSR letter reporting weighted estimates, but the evidence being much weaker in unweighted analyses.

Six months later (December 5), the APSR has published a correction to Stephens-Dougan 2022. The correction addresses each of my three concerns, but not perfectly, which I'll discuss below, along with other discussion about Stephens-Dougan 2022 and its correction. I'll refer to the original APSR letter as "Stephens-Dougan 2022" and the correction as "the correction".

---

2.

The pre-analysis plan associated with Stephens-Dougan 2022 listed four outcomes at the top of its page 4, but only one of these outcomes (referred to as "Individual rights and freedom threatened") was reported on in Stephens-Dougan 2022. However, Table 1 of Stephens-Dougan 2022 reported results for three outcomes that were not mentioned in the pre-analysis plan.

The t-statistics for the key interaction term for the three outcomes included in Table 1 of Stephens-Dougan 2022 but not mentioned in pre-analysis plan were 2.6, 2.0, and 2.1, all of which indicate sufficient evidence. The t-statistics for the key interaction term mentioned in pre-analysis plan but omitted from Stephens-Dougan 2022 were 0.6, 0.6, and 0.6, none of which indicate sufficient evidence.

I calculated the t-statistics of 2.6, 2.0, and 2.1 from Table 1 of Stephens-Dougan 2022, by dividing a coefficient by its standard error. I wasn't able to use the correction to calculate the t-statistics of 0.6, 0.6, and 0.6, because the relevant data for these three omitted pre-analysis plan outcomes are not in the correction but instead are in Table A12 of a "replication-final.pdf" file hosted at the Dataverse.

That's part of what I meant about an imperfect correction: a reader cannot use information published in the APSR itself to calculate the evidence provided by the outcomes that were planned to be reported on in the pre-analysis plan, or, for that matter, to see how there is substantially less evidence in the unweighted analysis. Instead, a reader needs to go to the Dataverse and dig through table after table of results.

The correction refers to deviations from the pre-analysis plan, but doesn't indicate the particular deviations and doesn't indicate what happens when these deviations are not made.  The "Supplementary Materials Correction-Final.docx" file at the Dataverse for Stephens-Dougan 2022 has a discussion of deviations from the pre-analysis plan, but, as far as I can tell, the discussion does not provide a reason why the results should not be reported for the three omitted outcomes, which were labeled in Table A12 as "Slow the Spread", "Stay Home", and "Too Long to Loosen Restrictions".

It seems to me to be a bad policy to permit researchers to deviate from a pre-analysis plan without justification and to merely report results from a planned analysis on, say, page 46 of a 68-page file on the Dataverse. But a bigger problem might be that, as far as I can tell, many journals don't even attempt to prevent misleading selective reporting for survey research for which there is no pre-analysis plan. Journals could require researchers reporting on surveys to submit or link to the full questionnaire for the surveys or at least to declare that the main text reports on results for all plausible measured outcomes and moderators.

---

3.

Next, let me discuss a method used in Stephens-Dougan 2022 and the correction, which I think is a bad method.

The code for Stephens-Dougan 2022 used measures of stereotypes about Whites and Blacks on the traits of hard working and intelligent, to create a variable called "negstereotype_endorsement". The code divided respondents into three categories, coded 0 for respondents who did not endorse a negative stereotype about Blacks relative to Whites, 0.5 for respondents who endorsed exactly one of the two negative stereotypes about Blacks relative to Whites, and 1 for respondents who endorsed both negative stereotypes about Blacks relative to Whites. For both Stephens-Dougan 2022 and the correction, Figure 3 reported for each reported outcome an estimate of how much the average treatment effect among prejudiced Whites (defined as those coded 1) differed from the average treatment effect among unprejudiced Whites (defined as those coded 0).

The most straightforward way to estimate this difference in treatment effects is to [1] calculate the treatment effect for prejudiced Whites coded 1, [2] calculate the treatment effect for unprejudiced Whites coded 0, and [3] calculate the difference between these treatment effects. The code for Stephens-Dougan 2022 instead estimated this difference using a logit regression that had three predictors: the treatment, the 0/0.5/1 measure of prejudice, and an interaction of the prior two predictors. But, by this method, the estimated difference in treatment effect between the 1 respondents and the 0 respondents depends on the 0.5 respondents. I can't think of a valid reason why responses from the 0.5 respondents should influence an estimated difference between the 0 respondents and the 1 respondents.

See my Stata output file for more on that. The influence of the 0.5 respondents might not be major in most or all cases, but an APSR reader won't know, based on Stephens-Dougan 2022 or its correction, the extent to which the 0.5 respondents influenced the estimates for the comparison of the 0 respondents to the 1 respondents.

Now about those 0.5 respondents…

---

4.

Remember that the Stephens-Dougan 2022 "negative stereotype endorsement" variable has three levels: 0 for the 74% of respondents who did not endorse a negative stereotype about Blacks relative to Whites, 0.5 for the 16% of respondents who endorsed exactly one of the two negative stereotypes about Blacks relative to Whites, and 1 for the 10% of respondents who endorsed both negative stereotypes about Blacks relative to Whites.

The correction indicates that "I discovered an error in the description of the variable, negative stereotype endorsement" and that "there was no error in the code used to create the variable". So was the intent for Stephens-Dougan 2022 to measure racial prejudice so that only the 1 respondents are considered prejudiced? Or was the intent to consider the 0.5 respondents and the 1 respondents to be prejudiced?

The pre-analysis plan seems to indicate a different method for measuring the moderator of negative stereotype endorsement:

The difference between the rating of Blacks and Whites is taken on both dimensions (intelligence and hard work) and then averaged.

But the pre-analysis plan also indicates that:

For racial predispositions, we will use two or three bins, depending on their distributions.

So, even ignoring the plan to average the stereotype ratings, the pre-analysis plan is inconclusive about whether the intent was to use two or three bins. Let's try this passage from Stephens-Dougan 2022:

A nontrivial fraction of the nationally representative sample—26%—endorsed either the stereotype that African Americans are less hardworking than whites or that African Americans are less intelligent than whites.

So that puts the 16% of respondents at the 0.5 level of negative stereotype endorsement into the same bin as the 10% at the 1 level of negative stereotype endorsement. Stephens-Dougan 2022 doesn't report the percentage that endorsed both negative stereotypes about Blacks. Reporting the percentage of 26% is what would be expected if the intent was to place into one bin any respondent who endorsed at least one of the negative stereotypes about Blacks, so I'm a bit skeptical of the claim in the correction that the description is in error and the code was correct. Maybe I'm missing something, but I don't see how someone who intends to have three bins reports the 26% and does not report the 10%.

For another thing, Stephens-Dougan 2022 has only three figures: Figure 1 reports results for racially prejudiced Whites, Figure 2 reports results for non-racially prejudiced Whites, and Figure 3 reports on the difference between racially prejudiced Whites and non-racially prejudiced Whites. Did Stephens-Dougan 2022 intend to not report results for the group of respondents who endorsed exactly one of the negative stereotypes about Blacks? Did Stephens-Dougan 2022 intend to suggest that respondents who rate Blacks as lazier in general than Whites aren't racially prejudiced as long as they rate Blacks equal to or higher than Whites in general on intelligence?

---

5.

Stephens-Dougan 2022 and the correction depict 84% confidence intervals in all figures. Stephens-Dougan 2022 indicated (footnote omitted) that:

For ease of interpretation, I plotted the predicted probability of agreeing with each pandemic measure in Figure 1, with 84% confidence intervals, the graphical equivalent to p < 0.05.

The 84% confidence interval is good for assessing a p=0.05 difference between estimates, but not for assessing at p=0.05 whether an estimate differs from a particular number such as zero. So 84% confidence intervals make sense for Figures 1 and 2, in which the key comparisons are of the control estimate to the treatment estimate. But 84% confidence intervals don't make as much sense for Figure 3, which plot only one estimate and for which the key assessment is whether the estimate differs from zero (Figure 3 in Stephens-Dougan 2022) or from 1 (the correction).

---

6.

I didn’t immediately realize why, in Figure 3 in Stephens-Dougan 2022, two of the four estimates cross zero, but in Figure 3 in the correction, none of the four estimates cross zero. Then I realized that the estimates plotted in Figure 3 of the correction (but not Figure 3 in Stephens-Dougan 2022) are odds ratios.

The y-axis for odds ratios for Figure 3 of the correction ranges from 0 to 30-something, using a linear scale. The odds ratio that indicates no effect is 1, and an odds ratio can't be negative, so that it why none of the four estimates cross zero in the corrected Figure 3.

It seems like a good idea for a plot of odds ratios to have a guideline for 1, so that readers can assess whether an odds ratio indicating no effect is a plausible value. And a log scale seems like a good idea for odds ratios, too. Relevant prior post that mentions that Fenton and Stephens-Dougan 2021 described a "very small" 0.01 odds ratio as "not substantively meaningful".

None of the 84% confidence intervals for Figure 3 capture an odds ratio that crosses 1, but an 84% confidence interval for Figure A3 in "Supplementary Materials Correction-Final.docx" does.

---

7.

Often, when I alert an author or journal to an error in a publication, the subsequent correction doesn't credit me for my work. Sometimes the correction even suggests that the authors themselves caught the error, like the correction to Stephens-Dougan 2022 seems to do:

After reviewing my code, I discovered an error in the description of the variable, negative stereotype endorsement.

I guess it's possible that Stephens-Dougan "discovered" the error. For instance, maybe after she submitted page proofs, for some reason she decided to review her code, and just happened to catch the error that she had missed before, and it's a big coincidence that this was the same error that I blogged about and alerted the APSR to.

And maybe Stephens-Dougan also discovered that her APSR letter misleadingly deviated from the relevant pre-analysis plan, so that I don't deserve credit for alerting the APSR to that.

Tagged with: , , , , , , , ,

By Posted on Posted in Methods No Comments

In a prior post, I criticized the questionnaire for the ANES 2020 Time Series Study, so I want to use this post to praise the questionnaire for the ANES 2022 Pilot Study, plus add some other comments.

---

1. The pilot questionnaire has items that ask participants to rate men and women on 0-to-100 feeling thermometers, which will permit assessment of the association for negative attitudes about women and men, presuming that some of the planned 1500 respondents express such negative attitudes.

2. The pilot questionnaire has items in which response options permit underestimation of the frequency of certain types of vote fraud, with a "Never" option for items about how often in the respondent's state [1] a voter casts more than one ballot and [2] votes are cast on behalf of dead people. That happened at least once recently in Arizona (see also https://www.heritage.org/voterfraud), and I suspect that this is currently a misperception that is more common on the political left.

But it doesn't seem like a good idea to phrase the vote fraud item about the respondent's state, so that coding a response as a misperception requires checking evidence in 50 states. And I don't think there is an obvious threshold for overestimating how often, say, a voter casts more than one ballot. "Rarely" seems like an appropriate response for Arizona residents, but is "Occasionally" incorrect?

3. The pilot questionnaire has an item about the genuineness of emails on Hunter Biden's laptop in which Hunter Biden "contacted representatives of foreign governments about business deals". So I guess that can be a misinformation item that liberals are more likely to be misinformed about.

4. The pilot questionnaire has items about whether being White/Black/Hispanic/Asian "comes with advantages, disadvantages, or doesn't it matter". Based on the follow up item, these items might not permit respondents to select both "advantages" and "disadvantages", and, if so, it might be better to differentiate respondents who think that, for instance, being White has only advantages from respondents who think that being White has on net more advantages than disadvantages.

5. The pilot questionnaire permits respondents to report the belief that Black and Hispanic Americans have lower socioeconomic status than White Americans because of biological differences, but respondents can't report the belief that particular less positive outcomes for White Americans relative to another group are due to biological differences (e.g., average White American K12 student math performance relative to average Asian American K12 student math performance).

---

Overall, the 2022 pilot seems like an improvement. For one thing, the pilot questionnaire, like is common for the ANES, has feeling thermometers about Whites, Blacks, Hispanics, and Asians, so that it's possible to construct a measure of negative attitudes about each included racial/ethnic group. And the feeling thermometers for men and women permit construction of a measure of negative attitudes about men and women. For another thing, respondents can report misperceptions that are presumably more common among persons on the political left. That's more than what is permitted by a lot of similar surveys.

Tagged with: , , , , ,

By Posted on Posted in Public opinion No Comments

Political Psychology recently published Chalmers et al 2022 "The rights of man: Libertarian concern for men's, but not women's, reproductive autonomy". The basis for this claim about libertarians' selective concern is indicated in the abstract as:

Libertarianism was associated with opposition to abortion rights and support for men's right both to prevent women from having abortions (male veto) and to withdraw financial support for a child when women refuse to terminate the pregnancy (financial abortion).

The above passage represents a flawed inferential method that I'll explain below.

---

The lead author of Chalmers et al 2022 quickly responded to my request about the availability of data, code, and codebooks, with replication materials now public at the OSF site. I'll use data from Study 2 and run a simple analysis to illustrate the inferential flaw.

The only predictor that I'll use is a 0-to-6 "Libert" variable that I renamed "Libertarianism" and recoded to range from 0 to 1 for responses to the item "To what extent would you describe your political persuasion as libertarian?", with 0 for "Not at all" to 1 "Very much".

---

In the OLS linear regression below, the abSINGLE outcome variable has eight levels, from 0 for "Not at all" to 1 for "Very much", for an item about whether the respondent thinks that a pregnant woman should be able to obtain a legal abortion if she is single and does not want to marry the man.

The linear regression output below (N=575) indicates that, on average, respondent libertarianism is negatively correlated with support for permitting a woman to have an abortion if she is single and does not want to marry the man.

. reg abSINGLE Libertarianism
---------------------------------
      abSINGLE |  Coef.  p-value
---------------+-----------------
Libertarianism | -0.30   0.000 
     intercept |  0.89   0.000 
---------------------------------

In the OLS linear regression below, the maleVETO outcome variable has six levels, from 0 for "Strongly disagree" to 1 for "Strongly agree", for an item about whether the respondent thinks that a woman should not be allowed to have an abortion if the man involved really wants to keep his unborn child.

The linear regression output below (N=575) indicates that, on average, respondent libertarianism is positively correlated with support for prohibiting a woman from having an abortion if the man involved really wants to keep his unborn child.

. reg maleVETO Libertarianism
--------------------------------
      maleVETO |  Coef. p-value
---------------+----------------
Libertarianism |  0.26  0.000 
     intercept |  0.13  0.000 
--------------------------------

So what's the flaw in combining results from these two regressions to infer that libertarians have a concern for men's reproductive autonomy but not for women's reproductive autonomy?

---

The flaw is that the linear regressions above include data from non-libertarians, and patterns among non-libertarians might account for the change in the sign of the coefficient on Libertarianism.

Note, for example, that, based on the OLS regression output, the predicted support among respondents highest in libertarianism will be 0.89 + -0.30, or 0.69, for women's right to an abortion on the 0-to-1 abSINGLE item, but will be 0.13 + 0.26, or 0.39, for men's right to an abortion veto on the 0-to-1 maleVETO item.

But let's forget these linear regression results, because the appropriate method for assessing whether a group is inconsistent is to analyze data only from that group. So here are respective means, for respondents at 6 on the 0-to-6 "Libert" variable (N=18):

0.45 on abSINGLE

0.49 on maleVETO

And here are respective means, for respondents at 5 or 6 on the 0-to-6 "Libert" variable (N=46):

0.53 on abSINGLE

0.42 on maleVETO

I wouldn't suggest interpreting these results to mean that libertarians are on net consistent about women's reproductive autonomy and men's reproductive autonomy or, for that matter, that libertarians favor women's reproductive autonomy over men's. But I think that the analyses illustrate the flaw in making inferences about a group based on a linear regression involving people who aren't in that group.

The Stata log file has output of my analyses above and additional analyses, but Chalmers et al 2022 had two datasets and multiple measures for key items, so the analyses aren't exhaustive.

Tagged with: , ,

By Posted on Posted in Sex No Comments

Politics & Gender published Deckman and Cassese 2021 "Gendered nationalism and the 2016 US presidential election", which, in 2022, shared an award for the best article published in Politics & Gender the prior year.

---

1.

So what is gendered nationalism? From Deckman and Cassese 2021 (p. 281):

Rather than focus on voters' sense of their own masculinity and femininity, we consider whether voters characterized American society as masculine or feminine and whether this macro-level gendering, or gendered nationalism as we call it, had political implications in the 2016 presidential election.

So how is this characterization of American society as masculine or feminine measured? The Deckman and Cassese 2021 online appendix indicates that gendered nationalism is...

Measured with a single survey item asking whether "Society as a whole has become too soft and feminine." Responses were provided on a four-point Likert scale ranging from strongly disagree to strongly agree.

So the measure of "whether voters characterized American society as masculine or feminine" (p. 281) ranged from the characterization that American society is (too) feminine to the characterization that American society is...not (too) feminine. The "(too)" is because I suspect that respondents might interpret the "too" in "too soft and feminine" as also applying to "feminine", but I'm not sure it matters much.

Regardless, there are at least three potential relevant characterizations: American society is feminine, masculine, or neither feminine nor masculine. It seems like a poor research design to combine two of these characterizations.

---

2.

Deckman and Cassese 2021 also described gendered nationalism as (p. 278):

Our project diverges from this work by focusing on beliefs about the gendered nature of American society as a whole—a sense of whether society is 'appropriately' masculine or has grown too soft and feminine.

But disagreement with the characterization that "Society as a whole has become too soft and feminine" doesn't necessarily indicate a characterization that society is "appropriately" masculine, because a respondent could believe that society is too masculine or that society is neither feminine nor masculine.

Omission of a response option indicating a belief that American society is (too) masculine might have made it easier for Deckman and Cassese 2021 to claim that "we suppose that those who rejected gendered nationalism were likely more inclined to vote for Hillary Clinton" (p. 282), as if only the measured "too soft and feminine" characterization is acceptance of "gendered nationalism" and not the unmeasured characterization that American society is (too) masculine.

---

3.

Regression results in Table 2 of Deckman and Cassese 2021 indicate that gendered nationalism predicts a vote for Trump over Clinton in 2016, net of controls for political party, a single measure of political ideology, and demographics such as class, race, and education.

Gendered nationalism is the only specific belief in the regression, and Deckman and Cassese 2021 reports no evidence about whether "beliefs about the gendered nature of American society as a whole" has any explanatory power above other beliefs about gender, such as gender roles and animus toward particular genders.

---

4.

Deckman and Cassese 2021 reported on four categories of class: lower class, working class, middle class, and upper class. Deckman and Cassese 2021 hypothesis H2 is that:

Gendered nationalism is more common among working-class men and women than among men and women with other socioeconomic class identifications.

For such situations, in which the hypothesis is that one of four categories is distinctive, the most straightforward approach is to omit from the regressions the hypothesized distinctive category, because then the p-values and coefficients for each of the three included categories will provide information about the evidence that that included category differs from the omitted category.

But the regressions in Deckman and Cassese 2021 omitted middle class, and, based on the middle model in Table 1, Deckman and Cassese 2021 concluded that:

Working-class Democrats were significantly more likely to agree that the United States has grown too soft and feminine, consistent with H2.

But the coefficients and standard errors were 0.57 and 0.26 for working class and 0.31 and 0.40 for lower class, so I'm not sure that the analysis in Table 1 contained enough evidence that the 0.57 estimate for working class differs from the 0.31 estimate for lower class.

---

5.

I think that Deckman and Cassese 2021 might have also misdescribed the class results in the Conclusions section, in the passage below, which doesn't seem limited to Democrat participants. From p. 295:

In particular, the finding that working-class voters held distinctive views on gendered nationalism is compelling given that many accounts of voting behavior in 2016 emphasized support for Donald Trump among the (white) working class.

For that "distinctive" claim, Deckman and Cassese 2021 seemed to reference differences in statistical significance (p. 289, footnote omitted):

The upper- and lower-class respondents did not differ from middle-class respondents in their endorsement of gendered nationalism beliefs. However, people who identified as working class were significantly more likely to agree that the United States has grown too soft and feminine, though the effect was marginally significant (p = .09) in a two-tailed test. This finding supports the idea that working-class voters hold a distinctive set of beliefs about gender and responded to the gender dynamics in the campaign with heightened support for Donald Trump’s candidacy, consistent with H2.

In the Table 1 baseline model predicting gendered nationalism without interactions, ologit coefficients are 0.25 for working class and 0.26 for lower class, so I'm not sure that there is sufficient evidence that working class views on gendered nationalism were distinctive from lower class views on gendered nationalism, even though the evidence is stronger that the 0.25 working class coefficient differs from zero than the 0.26 lower class coefficient differs from zero.

Looks like the survey's pre-election wave had at least twice as many working class respondents as lower class respondents. If that ratio was similar for the post-election wave, that would explain the difference in statistical significance and explain why the standard error was smaller for the working class (0.15) than for the lower class (0.23). Search for "class" at the PRRI site and use the PRRI/The Atlantic 2016 White Working Class Survey.

---

6.

At least Deckman and Cassese 2021 interpreted the positive coefficient on the interaction of college and Republican as an estimate of how the association of college and the outcome among Republicans differed from the association of college and the outcome among the omitted category.

But I'm not sure of the justification for "largely" in Deckman and Cassese 2021 (p. 293):

Thus, in accordance with our mediation hypothesis (H5), gender differences in beliefs that the United States has grown too soft and feminine largely account for the gender gap in support for Donald Trump in 2016.

Inclusion of the predictor for gendered nationalism pretty much only halves the logit coefficient for "female", from 0.80 to 0.42, and, in Figure 3, the gender gap in predicted probability of a Trump vote is pretty much only cut in half, too. I wouldn't call about half "largely", especially without addressing the obvious confound of attitudes about men and women that have nothing to do with "gendered nationalism".

---

7.

Deckman and Cassese 2021 was selected for a best article award by the editorial board of Politics & Gender. From my prior posts on publications in Politics & Gender: p < .000, misinterpreted interaction terms, and an example of the difference in statistical signifiance being used to infer an difference in effect.

---

NOTES

1. Prior post mentioning Deckman and Cassese 2021.

2. Prior post on deviations from a preregistration plan, for Cassese and Barnes 2017.

3. "Gendered nationalism" is an example of use of a general term when a better approach would be specificity, such as a measure that separates "masculine nationalism" from "feminine nationalism". Another example is racial resentment, in which a general term is used to describe only the type of racial resentment directed at Blacks. Feel free to read through participant comments in the Kam and Burge survey, in which plenty of comments from respondents who score low on the racial resentment scale indicate resentment directed at Whites.

Tagged with: , , ,

By Posted on Posted in Misperceptions No Comments

The Journal of Social and Political Psychology recently published Young et al 2022 "'I feel it in my gut:' Epistemic motivations, political beliefs, and misperceptions of COVID-19 and the 2020 U.S. presidential election", which reported in its abstract that:

Results from a US national survey from Nov-Dec 2020 illustrate that Republicans, conservatives, and those favorable towards President Trump held greater misperceptions about COVID and the 2020 election.

Young et al 2022 contains two shortcomings of too much social science: bias and error.

---

1.

In Young et al 2022, the selection of items measuring misperceptions is biased toward things that the political right is more likely than the political left to indicate a misperception about, so that the most that we can conclude from Young et al 2022 is that the political right more often reported misperceptions about things that the political right is more likely to report misperceptions about.

Young et al 2022 seems to acknowledge this research design flaw in the paragraph starting with:

Given the political valence of both COVID and election misinformation, these relationships might not apply to belief in liberal-serving misinformation.

But it's not clear to me why some misinformation about covid can't be liberal-serving. At least, there are misperceptions about covid that are presumably more common among the political left than among the political right.

For example, the eight-item Young et al 2022 covid misperceptions battery contains two items that permit respondents to underestimate the seriousness of covid-19: "Coronavirus (COVID-19 is a hoax" [sic for the unmatched parenthesis], and "The flu is more lethal than coronavirus (COVID-19)". But the battery doesn't contain corresponding items that permit respondents to overestimate the seriousness of covid-19.

Presumably, a higher percentage of the political left than the political right overestimated the seriousness of covid-19 at the time of the survey in late 2020, given that, in a different publication, a somewhat different Young et al team indicated that:

Results from a national survey of U.S. adults from Nov-Dec 2020 suggest that Trump favorability was...negatively associated with self-reported mask-wearing.

Another misperception measured in the survey is that "Asian American people are more likely to carry the virus than other people", which was not a true statement at the time. But, from what I can tell, at the time of the survey, covid rates in the United States were higher among Hispanics than among Whites, which presumably means that Hispanic Americans were more likely to carry the virus than White Americans. It's not clear to me why misinformation about the covid rate among Asians should be prioritized over misinformation about the covid rate among Hispanics, although, if someone wanted to bias the research design against the political right, that priority would make sense.

---

Similar flaw with the Young et al 2022 election 2020 misperceptions battery, which had an item that permits overestimation of the detected voter fraud ("There was widespread voter fraud in the 2020 Presidential election"), but had no item that would permit underestimation of voter fraud in 2020 (e.g., "There was no voter fraud in the 2020 Presidential election"), which is the type of error that the political left would presumably be more likely to make.

For another example, Young et al 2022 had a reverse-coded misperceptions item for "We can never be sure that Biden's win was legitimate", but had no item about whether we can be sure that Trump's 2016 win was legitimate, which would be an obvious item to pair with the Biden item to assess whether the political right and the political left are equally misinformed or at least equally likely to give insincere responses to surveys that have items such as "The coronavirus (COVID-19) vaccine will be used to implant people with microchips".

---

So I think it's less, as Young et al 2022 suggested, that "COVID misinformation and election misinformation both served Republican political goals", and more that the selection of misinformation items in Young et al 2022 was biased toward a liberal-serving conclusion.

Of course, it's entirely possible that the political right is more misinformed than the political left in general or on selected topics. But it's not clear to me how Young et al 2022 can provide a valid inference about that.

---

2.

For error, Young et al 2022 Table 3 has an unstandardized coefficient for Black race, indicating that, in the age 50 and older group, being Black corresponded to higher levels of Republicanism. I'm guessing that this coefficient is missing a negative sign, given that there is a negative sign on the standardized coefficient...The Table 2 income predictor for the age 18-49 group has an unstandardized coefficient of .04 and a standard error of .01, but no statistical significance asterisk, and has a standardized coefficient of .00, which I think might be too low...And the appendix indicates that "The analysis yielded two factors with Eigenvalues < 1.", but I think that should be a greater than symbol.

None of those potential errors are particularly important, except perhaps for inferences about phenomena such as the rigor of the peer and editorial review that Young et al 2022 went through.

---

NOTES

1. Footnotes 3 and 4 of Young et al 2022 indicate that:

Consistent with Vraga and Bode (2020), misperceptions were operationalized as COVID-related beliefs that contradicted the "best available evidence" and/or "expert consensus" at the time data were gathered.

If the purpose is to assess whether "I feel it in my gut" people are incorrect, then the perceptions should be shown to be incorrect and not merely in contradiction to expert consensus or, for that matter, in contradiction to the best available evidence.

2. The funding statement for Young et al 2022 indicates that the study was funded by the National Institute of Aging.

3. Prior posts on politically biased selection of misinformation items, in Abrajano and Lajevardi 2021 and in the American National Election Studies 2020 Time Series Study.

4. After I started drafting the above post, Social Science Quarterly published Benegal and Motta 2022 "Overconfident, resentful, and misinformed: How racial animus motivates confidence in false beliefs", which used the politically biased ANES misinformation items, in which, for example, respondents who agree that "World temperatures have not risen on average over the last 100 years" get coded as misinformed (an error presumably more common on the political right) but respondents who wildly overestimate the amount of climate change over the past 100 years don't get coded as misinformed (an error presumably more common on the political left).

5. I might be crazy, but I think that research about the correlates of misperceptions should identify respondents who have correct perceptions instead of merely identifying respondents who have particular misperceptions.

And I don't think that researchers should place particular misperceptions into the same category as the correct perception, such as by asking respondents merely whether world temperatures have risen on average over the last 100 years, any more than researchers should ask respondents merely whether world temperatures have risen on average by at least 3 degrees Celsius over the last 100 years, for which agreement would be the misperception.

Tagged with: , , ,

By Posted on Posted in Race No Comments

Some research includes measures of attitudes about certain groups but not about obvious comparison groups, such as research that includes attitudes about Blacks but not Whites or includes attitudes about women but not men. Feeling thermometers can help avoid this, which I'll illustrate with data from the Democracy Fund Voter Study Group's Views of the Electorate Research (VOTER) Survey.

---

The outcome is this item, from the 2018 wave of the VOTER survey:

Do you approve or disapprove of football players protesting by kneeling during the national anthem?

I coded responses 1 for strongly approve and somewhat approve and 0 for somewhat disapprove, strongly disapprove, don't know, and skipped. The key predictor was measured in 2017 and is based on 0-to-100 feeling thermometer ratings about Blacks and Whites, coded into six categories:

* Rated Whites equal to Blacks

---

* Rated Whites under 50 and Blacks at 50 or above

* Residual ratings of Whites lower than Blacks

---

* Rated Blacks under 50 and Whites at 50 or above

* Residual ratings of Blacks lower than Whites

---

* Did not rate Whites and/or Blacks

The plot below controls for only participant race measured in 2018, with error bars indicating 83.4% confidence intervals and survey weights applied.

The plot suggests that attitudes about anthem protests associated with negative attitudes about Blacks and with negative attitudes about Whites. These are presumably obvious results, but measures such as racial resentment probably won't be interpreted as suggesting both results.

---

NOTE

1. Stata code and output. The output reports results that had more extensive statistical control.

Tagged with: , , , ,