Monday, October 11, 2010

Bad Research On Bad Mothers. Part I: Fox Popularizes It

DR sent me a link to Fox and friends discussion of a British study about how mothers treat their sons and daughters. Here is the relevant part of the show's rush (?) transcript:

02:05:27 >> Alisyn: An illustration of is british survey asked 2,000 women about their opinions towards their sons and their daughters and this is, by the way, via the internet and people could be more honest than they can be face-to-face and they found and I must say, this does ring true in my house and never noticed it until the study came out, 88% of mothers admit they are more critical of their daughters.
02:05:54 Than of their sons and use different adjectives, to describe them, the boys they think are funny, playful, .. their daughters, how do they describe them.
02:06:04 >> Clayton: Look, they describe their daughters as stroppy, british, argumentative and serious.
02:06:12 And they get into fights with them all the time and 55% of moms said they found it easier to bond with their sons.
02:06:19 >> Dave: I find that, it really comes down to who's oldest and who's youngest.
02:06:23 The examples I have seen the parents are tougher on the burst foreign child.
02:06:26 I mean, that is -- >> Clayton: That was my experience.
02:06:29 >> Dave: And mine, much easier on me, yeah.
02:06:31 I have seen -- >> Alisyn: That makes fee feel better, my first born are twin girls and our little boy, i would describe him as funnier, more playful and more loving and cuter and I thought it was girls are harder than boys.
02:06:47 >> Clayton: They are.
02:06:47 >> Alisyn: There you have it.
02:06:49 >> Clayton: Let me ask you -- >> Alisyn: We're blaming the kids, note parents.
02:06:52 >> Clayton: Let me ask you, hopefully your kids are not watching, which kids do you love more?
02:06:58 Go ahead, answer.
02:06:59 >> Alisyn: Look, all I'm saying is little boys are easier.
02:07:02 I think.
02:07:03 And that is why I'm possibly nicer to my little sons.
02:07:07 >> Clayton: My sisters got away with murder.
02:07:09 High sister what's the second born and my mom -- there was a lot of fire between them growing up, arguing back and forth.
02:07:14 But, you know, I had the earlier curfew, I'm sure, like you, FIRST BORNs AND SHE WAS OUT LIKE An hour-and-a-half later and i was like, wait.
02:07:21 >> Dave: I was the youngest.
02:07:23 >> Clayton: You were the youngest.
02:07:24 >> Dave: I stayed out all night!
02:07:27 >> Dave: We need another study for dave.
02:07:29 >> Dave: E-mail us, com, girls are more complicated.
02:07:37 >> Alisyn: I think so, too.
Now pay attention, my sweet readers:

Here we have a British study about mothers being harsher on their daughters! And the conclusion of the Fox And Friends discussion is?

Boys are easier!

What a relief for all mothers! Because it's OK to be harsher on your daughters than sons as it's the fault of the daughters. They are so hard to bring up.

That's what the popularizing media is telling us.

Now get your shovels out because we are going to start digging for this study.

Studying The "Study"


The first problem: It turns out that it's not a proper academic study at all! It's a survey carried out by a British website called Netmums, and, as far as I can see, participation in it is voluntary.

The second problem: If the people participating in that "study" are not a random sample of British mothers but a self-selection from the members/readers of that particular website, Netmums, none of the results can be statistically generalized to the overall population of British mothers, and they certainly cannot be generalized to American mothers. Except that Fox And Friends does exactly that.


Let's have a look at the survey questions themselves. That reveals the third problem: The survey is poorly designed and leads the recipient to certain answers, in particular in this question:

Of the following adjectives, which do you think best describes girls or boys? Funny, cute, cheeky, caring, naughty, industrious, playful, thoughtful, stroppy, loving, argumentative, eager to please, serious.
What is very bad about this question is that the answer cannot be applied to BOTH boys and girls. The respondents are FORCED to pick either boys or girls. Thus, if you think both your sons and daughters are funny/naughty/whatever, you must still pick either daughters or sons in the answer. Why is this a problem? Suppose that you think boys and girls are almost equally likely to be, say, naughty. This survey forces you to pick one or the other, and when the results are summarized you suddenly develop this tremendous difference by gender which may not be there in the first place. Also, forcing an either/or type of answer will certainly be affected by the societal gender norms. If you really think that boys and girls are equally stroppy but you must pick one, you are going to use that societal knowledge in your answer.

Finally, here's the fourth problem having to do with the popularization of this survey rather than the survey itself: Netmums also surveyed fathers! But the results about fathers appear uninteresting to Fox and Friends. Though I should add that the actual numbers of mothers and fathers responding to the survey are given in none of the places I searched (that would be the fifth problem), so I have no idea how many dads participated and how many mums participated. If the number of dads was, say, ten, then not discussing those results might make some sense, though only if the samples were random to begin with.

Are you having fun yet? I next printed out the survey results for mothers and fathers and collated them. Some of the figures, by the way, make no sense as added up they exceed 100% when the question doesn't allow that, and in other cases the figures may be typing errors.

What that little exercise showed me is that the mums and dads in this survey are not statistically similar groups as comes to their family composition. This is evident in the first two questions which ask how many children and of what sexes the respondent has. Among the fathers 18.4% had only sons but among the mothers 30.8% had only sons. Such a difference is large enough to make me believe that there is bias in the way people entered this study, and that bias is most likely coming from the fact that you could choose to participate. Note that the percentages of parents who had only daughters were fairly similar (22.6% of fathers and 23.2% of mothers) among the respondents.

Back To Fox And Friends' Take on the "Study"

Let's have a look at some of those assertions Fox And Friends made about the survey:

I must say, this does ring true in my house and never noticed it until the study came out, 88% of mothers admit they are more critical of their daughters.
Where did that come from? Let me check my papers*. Here it is:

For mothers:

11. If you have children of both genders, which of the following statements do you agree with?
* I am more critical of my son - 11.5% agree
* I am more critical of my daughter - 21% agree

For fathers:

11. If you have children of both genders, which of the following statements do you agree with?
* I am more critical of my son - 28.5% agree
* I am more critical of my daughter - 13.2% agree
Astonishing! Mindboggling! Where the f**k did Fox and News get that 88% figure for the mothers????

And note that the percentages for fathers and mothers seem to be mirror images, pretty much. Logic suggests that we should have had something about 88% of fathers being harsher with their sons on that show. Of course it would have been equally preposterously wrong but at least it would have been logical.

This is fun. Next comment from the Fox and Friends program:

And they get into fights with them all the time and 55% of moms said they found it easier to bond with their sons.
Get into fights with them all the time? However hard I comb the study I don't find that pearl of information anywhere. There is no question about fighting in that survey. The closest match I can find is in that biased question where you have to attach an adjective to either girls or boys but not both, and both mothers and fathers attach the adjective "argumentative" much more often to girls than boys. But that doesn't mean "getting into fights with them all the time", and if it does mean that then both fathers and mothers get into those.

What about that bonding bit? A disgusting misinterpretation of the data which goes like this:

Only for mothers who have both sons and daughters:
9. If you have children of both genders, did you find it easier to bond with one of your children than the other(s)?

* Yes - 27.8%
* No - 72.2%

10. If yes, please specify whether you found it easiest to bond with your:

* Son - 54.9%
* Daughter - 45.1%

And only for fathers who have both sons and daughters:
9. If you have children of both genders, did you find it easier to bond with one of your children than the other(s)?

* Yes - 33.3%
* No - 66.7%

10. If yes, please specify whether you found it easiest to bond with your:

* Son - 49%
* Daughter - 51%
So let's see how many lies Fox And Friends managed to get into that one: They stated that 55% of mothers found it easier to bond with sons. First, the question was only asked of parents who had both sons and daughters, though that's not a big problem, but it does change the overall percentages (note that they talk about all mothers). Second, out of the mothers who had both sons and daughters 72.2% did not find bonding any different by the gender of the child. Only 27.8% did, and out of those 54.9% found that the child they had the strongest bond with was a son.

Note also the data for fathers.

So what's the actual percentage of mothers in that survey who found it easier to bond with a son than a daughter? And what's the actual percentage of fathers who found it easier to bond with a son than a daughter?

The total answer doesn't make sense because roughly 30% of both mothers and fathers only had one child, and only 46% of the mothers (though 65.7% of the fathers) had children of both sexes. But it's possible to figure out the percentage of mothers and fathers within the group of parents which had both sons and daughters who found it easiest to bond with a child of the male sex. These percentages are 16.17% (49% of 33.3%) for fathers and 15.26% (54.9% of 27.8%) for mothers.

Now compare that to the 55% figure in Fox And Friends!


Last but not least: Boys are easier! Where did that conclusion come from? The survey asks nothing about the ease with which one can bring up a son or a daughter, nothing.

Perhaps this is all some sort of code for the survey results that both mothers (83.9%) and fathers (78.7%) find girls stroppier than boys in that question where they were forced to pick either boys or girls for each adjective. "Stroppy" is British slang for obstreperous, belligerent, easily insulted.

In the next part of this post I'm going to have a look at another popularization of this study. A calmer and gentler one than the Fox And Friends chatter.
----
These results can be found in the results for moms and the results for dads. I printed them out and looked at them side-by-side.

Added later: Netmums removed the detailed results from the net the day after I sent them my questions about their interpretation.