Home › 2014 › May

Month: May 2014

By Posted on Posted in Methods No Comments

The American National Elections Studies (ANES) has measured abortion attitudes since 1980 with an item that dramatically inflates the percentage of pro-choice absolutists:

There has been some discussion about abortion during
recent years. Which one of the opinions on this page best agrees with your view? You can just tell me the number of the opinion you choose.
1. By law, abortion should never be permitted.
2. The law should permit abortion only in case of rape, incest, or when the woman's life is in danger.
3. The law should permit abortion for reasons other than rape, incest, or danger to the woman's life, but only after the need for the abortion has been clearly established.
4. By law, a woman should always be able to obtain an abortion as a matter of personal choice.
5. Other {SPECIFY}

In a book chapter of Improving Public Opinion Surveys: Interdisciplinary Innovation and the American National Election Studies, Heather Marie Rice and I discussed this measure and results from a new abortion attitudes measure piloted in 2006 and included on the 2008 ANES Time Series Study. The 2006 and 2008 studies did not ask any respondents both abortion attitudes measures, but the 2012 study did. This post presents data from the 2012 study describing how persons selecting an absolute abortion policy option responded when asked about policies for specific abortion conditions.

---

Based on the five-part item above, and removing from the analysis the five persons who provided an Other response, 44 percent of the population agreed that "[b]y law, a woman should always be able to obtain an abortion as a matter of personal choice." The figure below indicates how these pro-choice absolutists later responded to items about specific abortion conditions.

Red bars indicate the percentage of persons who agreed on the 2012 pre-election survey that "[b]y law, a woman should always be able to obtain an abortion as a matter of personal choice" but reported opposition to abortion for the corresponding condition in the 2012 post-election survey.

2012abortionANESprochoice4

Sixty-six percent of these pro-choice absolutists on the 2012 pre-election survey later reported opposition to abortion if the reason for the abortion is that the child will not be the sex that the pregnant woman wanted. Eighteen percent of these pro-choice absolutists later reported neither favoring nor opposing abortion for that reason, and 16 percent later reported favoring abortion for that reason. Remember that this 16 percent favoring abortion for reasons of fetal sex selection is 16 percent of the pro-choice absolutist subsample.

In the overall US population, only 8 percent favor abortion for fetal sex selection; this 8 percent is a more accurate estimate of the percent of pro-choice absolutists in the population than the 44 percent estimate from the five-part item.

---

Based on the five-part item above, and removing from the analysis the five persons who provided an Other response, 12 percent of the population thinks that "[b]y law, abortion should never be permitted." The figure below indicates how these pro-life absolutists later  responded to items about specific abortion conditions.

Green bars indicate the percentage of persons who agreed on the 2012 pre-election survey that "[b]y law, abortion should never be permitted" but reported support for abortion for the corresponding condition in the 2012 post-election survey.

2012abortionANESprolife4

Twenty-nine percent of these pro-life absolutists on the 2012 pre-election survey later reported support for abortion if the reason for the abortion is that the woman might die from the pregnancy. Twenty-nine percent of these pro-choice absolutists later reported neither favoring nor opposing abortion for that reason, and 42 percent later reported opposing abortion for that reason. Remember that this 42 percent opposing abortion for reasons of protecting the pregnant woman's life is 42 percent of the pro-life absolutist subsample.

In the overall US population, only 11 percent oppose abortion if the woman might die from the pregnancy; this 11 percent is a more accurate estimate of the percent of pro-life absolutists in the US population than the 12 percent estimate from the five-part item.

---

There is a negligible difference in measured pro-life absolutism between the two methods, but the five-part item inflated pro-choice absolutism by a factor of 5. Our book chapter suggested that this inflated pro-choice absolutism might result because the typical person considers abortion in terms of the hard cases, especially since the five-part item mentions only the hard cases of rape, incest, and danger to the pregnant woman's life.

---

Notes

1. The percent of absolutists is slightly smaller if absolutism is measured as supporting or opposing abortion in each listed condition.

2. The percent of pro-life absolutists is likely overestimated in the "fatal" abortion condition item because the item asks about abortion if "staying pregnant could cause the woman to die"; presumably, there would be less opposition to abortion if the item stated with certainty that staying pregnant would cause the woman to die.

3. Data presented above are for persons who answered the five-part abortion item on the 2012 ANES pre-election survey and answered at least one abortion condition item on the 2012 ANES post-election survey. Don't know and refusal responses were listwise deleted for each cross-tabulation. Data were weighted with the Stata command svyset [pweight=weight_full], strata(strata_full); weighted cross-tabulations were calculated with the command svy: tabulate X Y if Y==Z, where X is the abortion condition item, Y is the five-part abortion item, and Z is one of the absolute policy options on the five-part item.

4. Here is the text for each abortion condition item that appeared on the 2012 ANES Time Series post-election survey:

>[First,/Next,] do you favor, oppose, or neither favor nor oppose abortion being legal if:
* staying pregnant could cause the woman to die
* the pregnancy was caused by the woman being raped
* the fetus will be born with a serious birth defect
* the pregnancy was caused by the woman having sex with a blood relative
* staying pregnant would hurt the woman's health but is very unlikely to cause her to die
* having the child would be extremely difficult for the woman financially
* the child will not be the sex the woman wants it to be

There was also a general item on the post-election survey:

Next, do you favor, oppose, or neither favor nor oppose abortion being legal if the woman chooses to have one?

5. Follow-up items to the post-election survey abortion items asked respondents to indicate intensity of preference, such as favor a great deal, favor moderately, or favor a little. These follow-up items were not included in the above analysis.

6. There were more than 5000 respondents for the pre-election and post-election surveys.

Tagged with: , ,

By Posted on Posted in Law and courts No Comments

For those of you coming from the Monkey Cage: welcome!

This is a blog on my research and other topics of interest. I'm in the middle of a series on incorrect survey weighting, which is part of a larger series on reproduction in social science. I'm a proponent of research transparency, such as preregistration of experimental studies to reduce researcher degrees of freedom, third-party data collection to reduce fraud, and public online archiving of data and code to increase the likelihood that error is discovered.

My main research areas right now are race, law, and their intersection. I plan to blog on those and other topics: I am expecting to post on list experiments, abortion attitudes, the file drawer problem, Supreme Court nominations, and curiosities in the archives at the Time-Sharing Experiments for the Social Sciences. I hope that you find something of interest.

---

UPDATE (May 21, 2014)

Links to the Monkey Cage post have been made at SCOTUSBlog, Jonathan Bernstein, and the American Constitution Society.

---

UPDATE (May 21, 2014)

Jonathan Bernstein commented on my Monkey Cage guest post, expressing skepticism about a real distinction between delayed and hastened retirements. The first part of my response was as follows:

Hi Jonathan,

Let me expand on the distinction between delayed and hastened retirements.

Imagine that Clarence Thomas reveals that he wants to retire this summer, but conservatives pressure him to delay his retirement until a Republican is elected president. Compare that to liberals pressuring Ruth Bader Ginsburg to retire before the 2016 election.

Note the distinctions: liberals are trying to change Ginsburg's mind about *whether* to retire, and conservatives are trying to change Thomas's mind about *when* to retire; moreover, conservatives are asking Thomas to sacrifice *extra* *personal* time that he would have had in retirement, and liberals are asking Ginsburg to sacrifice *all* the rest of her years as *one of the most powerful persons in the United States.*

Orin Kerr of the Volokh Conspiracy also commented on the post, at the Monkey Cage itself, asking why a model is necessary when the sample of justices is small enough to ask justices or use past interviews. My response:

Hi Orin,

Artemus Ward has a valuable book, Deciding to Leave, that offers more richness than statistical models offer for investigating the often idiosyncratic reasons for Supreme Court retirements. But for addressing whether justices retire strategically and, if so, when and under what conditions -- or for making quantitative predictions about whether a particular justice might retire at a given time -- there is complementary value in a statistical model.

1. For one thing, there is sometimes reason to be skeptical of the reasons that political actors provide for their behavior: there is a line of research suggesting that personal policy preferences inform Supreme Court justice voting on cases, though many justices might not admit this in direct questioning. Regarding retirements, many justices have been forthcoming about their strategic retirement planning, but some justices have downplayed or denied strategic planning: for example, Ward described press skepticism of Potter Stewart's assertion that he did not strategically delay retirement while Jimmy Carter was president (p. 194).

Statistical models permit us to test theories based on what Stewart and other justices *did* instead of what Stewart and other justices *said*, similar to the way that prosecutors might develop a theory of the crime based on forensic evidence instead of suspect statements.

2. But even if the justices were always honest and public about their reasons for retiring or not retiring, it is still necessary to apply some sort of statistical analysis to address our questions. By my count, from 1962 to 2010, 5 justices retired consistent with a delay strategy and 8 justices retired when the political environment was unfavorable. Observers using simple statistical tools might consider this evidence that justices are more likely to retire unstrategically than to delay retirement, but this overlooks the fact that justices have more opportunities to retire unstrategically than to delay retirement.

For example, assuming that no conservative retires during President Obama's eight years in office, the five conservative justices as a group will each have had eight years to retire unstrategically, for a total of 40 opportunities; but liberal justices have had fewer opportunities to delay retirement: Breyer, Ginsburg, Souter, and Stevens each had one opportunity to retire consistent with a delay strategy in 2009, and -- presuming that justices stay on another year to avoid a double summer vacancy -- Breyer, Ginsburg, Sotomayor, and Stevens each had one opportunity to retire consistent with a delay strategy in 2010, for a total of 8 opportunities.

In this particular period, the proper comparison is not 2 delayed retirements to 0 unstrategic retirements, but instead is 2 delayed retirements out of 8 opportunities (25%) to 0 unstrategic retirements out of 40 opportunities (0%).

3. Sotomayor's addition in the 2010 data highlights another value of statistical models: they permit us to control for other retirement pressures. Statistical models can help account -- in a way that qualitative studies or direct questioning cannot -- for the fact that the 2010 observation of Sotomayor is not equivalent to the 2010 observation of Ginsburg because these justices have different characteristics on other key variables, such as age. From 1962 to 2010, justices retired 14 percent of the time during delayed retirement opportunities, but retired only 4 percent of the time during unfavorable political environments. But these percentages should not be directly compared because there might be spurious correlations that have inflated or deflated the percentages: for example, perhaps older and infirm justices were more likely to experience a delayed opportunity and *that* is why the delayed percentage is relatively higher than the unstrategic percentage. Statistical models let us adjust summary statistics to address such spurious correlations.

---

Bill James is said to have said something to the effect that bad statistics are the alternative to good statistics. Relying only on justice statements instead of good statistics can introduce inferential error about justice retirement strategies in the aggregate in several ways: (1) justices might misrepresent their motives for retiring or not retiring; (2) we might not properly account for the fact that justices face more unstrategic opportunities than delayed opportunities or hasten opportunities; and (3) we might not properly account for variables such as age and illness that also influence decisions to retire.

Tagged with: ,

By Posted on Posted in Methods

My previous posts discussed the p-values that the base module of SPSS reports for statistical significance tests using weighted data; these weights are not correct for probability-weighted analyses. Jon Peck informed me of SPSS Complex Samples, which can provide correct p-values for statistical significance tests for probability-weighted analyses. Complex Samples does not have the most intuitive setup, so this post describes the procedure for analyzing data using probability weights in SPSS Statistics 21.

SPSS0

SPSS1

The dataset that I was working with had probability weights but no clustering or stratification, so the Stratify By and Clusters boxes remain empty in the image below.

SPSS4

The next dialog box has options for Simple Systematic and Simple Sequential. Either method will work if Proportions are set to 1 in the subsequent dialog box.

SPSS3

SPSS4

SPSS5

SPSS6

SPSS7

SPSS8

SPSS9

I conducted an independent samples t-test, so I selected the General Linear Model command below.

SPSS10

SPSS11

Click the Statistics button in the image above and then click the t-test box in the image below to tell SPSS to conduct a t-test.

SPSS12

SPSS13

Hit OK to get the output.

rattan2012outputSPSS

The SPSS output above has the same p-value as the probability-weighted Stata output below.

rattan2012outputStata

Tagged with: , ,

By Posted on Posted in Methods

My previous post discussed p-values in SPSS and Stata for probability-weighted data. This post provides more information on weighting in the base module of SPSS. Data in this post are from Craig and Richeson (2014), downloaded from the TESS archives; SPSS commands are from personal communication with Maureen Craig, who kindly and quickly shared her replication code.

Figure 2 in Craig and Richeson's 2014 Personality and Social Psychology Bulletin article depicts point estimates and standard errors for racial feeling thermometer ratings made by white non-Hispanic respondents. The article text confirms what the figure shows: whites in the racial shift condition (who were exposed to a news article titled, "In a Generation, Racial Minorities May Be the U.S. Majority") rated Blacks/African Americans, Latinos/Hispanics, and Asian-Americans lower on the feeling thermometers at a statistically significant level than whites in the control condition (who were exposed to a news article titled, "U.S. Census Bureau Reports Residents Now Move at a Higher Rate").

CraigRicheson2014PSPB

Craig and Richeson generated a weight variable that retained the original post-stratification weights for non-Hispanic white respondents but changed the weight to 0.001 for respondents who were not non-Hispanic white. Figure 2 results were drawn from the SPSS UNIANOVA command, which "provides regression analysis and analysis of variance for one dependent variable by one or more factors and/or variables," according to the SPSS web entry for the UNIANOVA command.

The SPSS output below represents a weighted analysis in the base SPSS module for the command UNIANOVA therm_bl BY dummyCond WITH cPPAGE cPPEDUCAT cPPGENDER, in which therm_bl, dummyCond, cPPAGE, cPPEDUCAT, and cPPGENDER respectively indicate numeric ratings on a 0-to-100 feeling thermometer scale for blacks, a dummy variable indicating whether the respondent received the control news article or the treatment news article, respondent age, respondent education on a four-level scale, and respondent sex. The 0.027 Sig. value for dummyCond indicates that the mean thermometer rating made by white non-Hispanics in the control condition was different at the 0.027 level of statistical significance from the mean thermometer rating made by white non-Hispanics in the treatment condition.

CR2014PSPB

The image below presents results for the same analysis conducted using probability weights in Stata, with weightCR indicating a weight variable mimicking the post-stratification weight created by Craig and Richeson: the corresponding p-value is 0.182, not 0.027, a difference due to the Stata p-value reflecting a probability-weighted analysis and the SPSS p-value reflecting a frequency-weighted analysis.

CR2014bl0

So why did SPSS return a p-value of 0.027 for dummyCond?

The image below is drawn from online documentation for the SPSS weight command. The second bullet point indicates that SPSS often rounds fractional weights to the nearest integer. The third bullet point indicates that SPSS statistical procedures ignore cases with a weight of zero, so cases with fractional weights that round to zero will be ignored. The first bullet point indicates that SPSS arithmetically replicates a case according to the weight variable: for instance, SPSS treats a case with a weight of 3 as if that case were 3 independent and identical cases.

 weightsSPSS

Let's see if this is what SPSS did. The command gen weightCRround = round(weightCR) in the Stata output below generates a variable with the values of weightCR rounded to the nearest integer. When the Stata command used the frequency weight option with this rounded weight variable, Stata reported p-values identical to the SPSS p-values.

CR2014bl2

The Stata output below illustrates what happened in the above frequency-weighted analysis. The expand weightCRround command replicated each dataset case n-1 times, in which n is the number in the weightCRround variable: for example, each case with a weightCRround value of 3 now appears three times in the dataset. Stata retained one instance of each case with a weightCRround value of zero, but SPSS ignores cases with a weight of zero for weighted analyses; therefore, the regression excluded cases with a zero value for weightCRround.

Stata p-values from a non-weighted regression on this adjusted dataset were identical to SPSS p-values reported using the Craig and Richeson commands.

CR2014bl3

So how much did SPSS alter the dataset? The output below is for the original dataset: the racial shift and control conditions respectively had 233 and 222 white non-Hispanic respondents with full data on therm_bl, cPPAGE, cPPEDUCAT, and cPPGENDER; the difference in mean therm_bl ratings across conditions was 3.13 units.

CR2014bl4before

The output below is for the dataset after executing the round and expand commands: the racial shift and control conditions respectively had 189 and 192 white non-Hispanic respondents with a non-zero weight and full data on therm_bl, cPPAGE, cPPEDUCAT, and cPPGENDER; the difference in mean therm_bl ratings across conditions was 4.67, a 49 percent increase over the original difference of 3.13 units.

CR2014bl4after

---

Certain weighted procedures in the SPSS base module report p-values identical to p-values reported in Stata when weights are rounded, cases are expanded by those weights, and cases with a zero weight are ignored; other weighted procedures in the SPSS base module report p-values identical to p-values reported in Stata when the importance weight option is selected or when the analytic weight option is selected and the sum of the weights is 1.

(Stata's analytic weight option treats each weight as an indication of the number of observations represented in a particular case; for instance, an analytic weight of 4 indicates that the values for the corresponding case reflect the mean values for four observations; see here.)

Test analyses that I conducted produced the following relationship between SPSS output and Stata output.

SPSS weighted base module procedures that reported p-values identical to Stata p-values when weights were rounded, cases were expanded by those weights, and cases with a zero weight were ignored:

  1. UNIANOVA with weights indicated in the WEIGHT BY command

SPSS weighted base module procedures that reported p-values identical to Stata p-values when the importance weight or analytic weight option was selected and the sum of the weights was 1:

  1. Independent samples t-test
  2. Linear regression with weights indicated in the WEIGHT BY command
  3. Linear regression with weights indicated in the REGWT subcommand in the regression menu (weighted least squares analysis)
  4. UNIANOVA with weights indicated in the REGWT subcommand in the regression menu (weighted least squares analysis)

---

SPSS has a procedure that correctly calculates p-values with survey weights, as Jon Peck noted in a comment to the previous post. The next post will describe that procedure.

---

UPDATE (June 20, 2015)

Craig and Richeson have issued a corrigendum to the "On the Precipice of a 'Majority-Minority' America" article that had used incorrect survey weights.

Tagged with: , , ,