Tuesday, August 29, 2006

"When You Have Children": presumptive wording just part of New York Times reporter Louise Story's pro-natalist debacle.

The New York Times last year printed an article describing how many professional women, including those educated at the Ivy League, plan to be stay at home mothers. In Many Women at Elite Colleges Set Career Path to Motherhood, Ms. Story reported that:

While the changing attitudes are difficult to quantify, the shift emerges repeatedly in interviews with Ivy League students, including 138 freshman and senior females at Yale who replied to e-mail questions sent to members of two residential colleges over the last school year.

The interviews found that 85 of the students, or roughly 60 percent, said that when they had children, they planned to cut back on work or stop working entirely. About half of those women said they planned to work part time, and about half wanted to stop work for at least a few years.

What I didn't know was that this wasn't just another pronatalist article. The methodology flaws and bias were extensive enough to stir up a minor controversy. As it turns out, the survey (posted by David Goldenberg of Gelf Magazine) included such gems as:

When you have children, do you plan to stay at home with them or do you plan to continue working? Why?
In day-to-day interactions, the presumption that all people will have children that is conveyed by the term "when" is irritating. In a survey, it amounts to more than that: it taints the results of the study.

. . .Emily Holleman, a current sophomore . . . says she declined to fill out the survey because she thought it was flawed. "I felt that it was very badly phrased and strongly suggested that ALL women at Yale planned to a) get married and b) have kids. It also assumed that all women at Yale were straight," she told Gelf in an email. "It was relatively clear to me and several of my friends that she was either unable to construct a suitable survey or had already decided what answers she wanted to receive and constructed her survey based on what questions would induce these responses."

(emphasis added)

These flaws prompted a flurry of reactions from publications such as Slate and Salon. An article in The Nation, Desperate Housewives of the Ivy League?, noted:
In fact, Story presents no evidence that more Ivy League undergrads today are planning to retire at 30 to the playground than ten, twenty or thirty years ago. Simultaneously, an armada of bloggers shredded her questionnaire as biased
(hint: If you begin with "When you have children," you've already skewed your results) and denounced her interpretation of the answers as hype. What she actually found, as the writer Robin Herman noted in a crisp letter to the Times, was that 70 percent of those who answered planned to keep working full or part time through motherhood. Even by Judith Miller standards, the Story story was pretty flimsy.

(emphasis added)

Fishbowl NY contacted some of those students quoted in the article. One student replied:

She in fact did interview my other suitemates who answered the survey as either not wanting to have children at all, or would continue working as a mother. I am somewhat shocked that she did not include ANY of their ideas or views in the article. . . Because of my own life experiences, I simply hold the personal attitude that I would be willing to stop my career for children if/when I have them.
Although that student remained anonymous, the evidence remains that one of the women cited as a future SAHM isn't even sure she'll have children!

Ms. Story did indeed post a response, albeit a weak one which does little to addressss the concerns with her methodology. Indeed, her citing of Sylvia Hewlett as a "top researcher on trends in women in the labor force" demonstrates just how little Ms. Strong understands.

While only a few noted the particularly relevant anti-childfree wording, it is still a good thing for us that this article is being scrutinized to this extent. Perhaps pro-natalist reporters will think twice before letting their bias run away with them.

Oh, and since Strong chose Yale because she was a recent grad, I can't resist:


Unknown said...

There's a litany of problems with this survey. I'll just throw out a few comments, some more serious than others.

1) "Was your mom a stay-at-home mom? Explain whether she worked, and how much she worked! Were you glad with her choice (to either work or stay-at-home or whatever combination she did)? "

Was the exclamation point really necessary? Even Elaine Benes would disapprove.

2) Format-wise, this survey sucked. I'm actually shocked that 138 people completed this survey. It appears to be all long- and short-answer. It's hard to picture anyone but the people who are most committed to the underlying bias of this study botherering to fill out the whole thing (and maybe perhaps a couple of very pissed off people on the other side). People generally don't have that kind of attention span.

3) The obvious bias in this study makes the results almost meaningless. They sure as hell aren't generalizable to anyone else. So in essence, we've got study that only relates to a miniscule subsample of the population, and even that subsample poorly examined.

4) The department of whichever school granted her a degree should be sent a case of Mad Dog 20/20. You've obviously done a complete shit job of educating Ms. Story in research methodology. Sometimes cheap booze helps to drown the sorrow and embarrassment.

5) I have a hard time believing that someone who put so little effort in creating this survey would have bothered to code the results of all of the open-ended questions. Which, you know, you kinda have to do if you're going to ask those questions to begin with.

Anonymous said...

"The department of whichever school granted her a degree should be sent a case of Mad Dog 20/20. You've obviously done a complete shit job of educating Ms. Story in research methodology. Sometimes cheap booze helps to drown the sorrow and embarrassment."

That's Yale all right. Go Crimson!

Anonymous said...

When you have children, will you stay at home or keep working?

Heh. Reminds me of the old joke that they use in logic classes and law school and so on: "Have you stopped beating your wife yet?"