Sunday, December 28, 2014

Washington Post: It is 'misleading to suggest that [the one in five stat] is representative of the experience of all college women'

From the Washington Post:

“We know the numbers: 1 in 5 of every one of those young women who is dropped off for that first day of school, before they finish school, will be assaulted in her college years.”

— Vice President Biden, remarks on the release of a White House report on sexual assault, April 29, 2014

“It is estimated that 1 in 5 women on college campuses has been sexually assaulted during their time there — 1 in 5.”

— President Obama, remarks at White House, Jan. 22, 2014

These quotes may seem old, but the issue is not, especially in the wake of the controversy over a Rolling Stone article alleging a gang rape at the University of Virginia.

Previous reports of sexual assault on college campuses spurred the White House early in 2014 to launch a task force to examine the issue. The group’s report was issued on April 29; the first sentence echoes what both the president and vice president have asserted in public: “One in five women is sexually assaulted in college.”

Where does this oft-repeated statistic come from? The Fact Checker dug into the data so you don’t have to.

The Facts

This statistic is derived from a 2007 study, the Campus Sexual Assault Study, which was conducted for the Justice Department’s National Institute of Justice. The researchers also surveyed men, but the statistic in question focuses on women, so we will look carefully at that part of the study.

In the winter of 2006, researchers used a Web-based survey to interview undergraduates at two large public universities, one in the Midwest and one in the South. A total of 5,446 undergraduate women, between the ages of 18 and 25, participated as part of a random sample. The survey was anonymous and took about 15 minutes to complete. (Participants received a $10 certificate for participating.)

So, first of all, it’s important to remember that this is a single survey, based on the experiences of students at two universities. As the researchers acknowledged, these results clearly can be generalized to those two large four-year universities, but not necessarily elsewhere. Moreover, researchers acknowledged the response rate was relatively low compared to face-to-face interviewing.

The survey found that 1,073 women, or 19 percent, said that they experienced attempted or completed sexual assault since entering college. The actual breakdown was that 12.6 percent experienced attempted sexual assault and 13.7 percent experienced actual sexual assault.

The sexual assault instances were further divided into sexual assault while incapacitated from drugs or alcohol or sexual assault through physical force. Most of the sexual assaults were identified as rapes, though the report said “sexual battery” could have included “sexual touching,” such as forced kissing or fondling.

Note that the percentage of sexual assaults — 13.7 percent — was lower than the 1-in-5 figure cited by administration officials. (It is more like 1 in 7.) That’s because the president and vice president used careful phrasing that covered a student’s entire time in college. The overall survey interviewed students that included freshmen, sophomores and juniors.

In a 2009 report, the researchers released a closer look at the data. This report showed that out of the subset of seniors surveyed (1,402 women), that 19 percent (about 287) had experienced sexual assault.

In other words, information that is localized to the seniors at two colleges has now been extrapolated by politicians to the universe of college experience.

To some extent, the results depend on how questions are phrased and answers interpreted.

On its Web site, NIJ notes that rapes and other forms of sexual assault are among the most underreported crimes, but that “researchers have been unable to determine the precise incidence of sexual assault on American campuses because the incidence found depends on how the questions are worded and the context of the survey.” It said that two parallel surveys of American college women were conducted in 1997 and came up with very different results, with one survey showing rapes were 11 times higher than the percentage in the other survey. The reason appears to be because of how the questions were worded.

Earlier this month, the Bureau of Justice Statistics released a study titled “Rape and Sexual Assault Victimization Among College-Age Females, 1995–2013,” which suggested much lower levels of sexual assault than indicated in the Campus Assault Study. But questions have been raised about whether the survey used for the report, the National Crime Victimization Survey, offers an adequate measure of the incidence of rape; a National Research Council report this year recommended ways to improve it.

Still, the BJS report did indicate interesting trends over time, including the fact that sexual assault on campus had been declining in recent years — and that the incidence of sexual assault was higher for women who were not in college. The report also offered a useful guide to the differences between various studies of sexual assault on campus, including this caveat about using the Campus Sexual Assault Study: “Because of the limited population included in the CSA, it should not be assumed that findings from the survey are representative of the population of persons ages 18 to 24 or even to college students specifically.”

Given that an arm of the Justice Department now has provided this warning about the 1-in-5 statistic, The Fact Checker asked the White House to explain why the president and the vice president expressed the data in such sweeping terms.

A White House spokesperson, who asked not to be quoted, said that the 1-in-5 statistic was consistent with other studies, pointing in particular to two surveys: a 2000 Justice Department study that reported that, over a seven-month period, 2.8 percent of college women faced rape victimization (completed or attempted) and a 2014 MIT survey that found that 17 percent of female undergraduates experienced one or more unwanted sexual behaviors.

But there are issues with both surveys. The 2000 report could be interpreted as suggesting that 1 in 5 women might face rape victimization over four or five years in college, but the authors conceded in 2010 that “admittedly, these projections are speculative and await longitudinal studies that follow women throughout their college careers.”

Meanwhile, the MIT sample was based on self-selected responses, so its data could not be used as any sort of confirmation. The university itself warned that “it would be a mistake to use these numbers to generalize about the prevalence of unwanted sexual behavior in the lives of all MIT students.”

The Pinocchio Test

The White House should be applauded for calling attention to this issue; as part of its effort, the White House task force is urging colleges to undertake climate surveys and pushing for more research on the prevalence of sexual assault.

But the BJS warning about the use of CSA data demonstrates the president and the vice president went too far in suggesting the 1-in-5 statistic applied to all college students. Instead, this oft-cited statistic comes from a Web-based survey of two large universities, making it misleading to suggest that it is representative of the experience of all college women.

Given the uncertainty of the research — and various surveys and anecdotal accounts indicating there likely is a serious problem of sexual assault on college campuses — we are going to limit our ruling to a single Pinocchio. But we will monitor how the president, the vice president and other administration officials use this statistic in the future.