BlogCast: How Useful is Self-Report?

If you're a regular reader of Peter and Travis's blog, Obesity Panacea, you may have heard one of their semi-regular blogcasts. Well, since Peter is traveling the world (read about it here), Travis asked me to join him for a blogcast. While discussing topics that we could discuss, a sent a few links and papers his way, and he was like "ugh, self-report." And I was like, "dude, self-report makes the world go around."

Okay, so the conversation may not have gone exactly like that, but the outcome was we decided to go ahead and talk about the relative merits of self-report data in science.

As he wrote:

For the uninitiated, self-report data refers to information that people provide themselves - questionnaires and interviews are very common examples. This is in contrast to direct measurement, which is exactly what it sounds like - researchers measuring your height and weight themselves, etc. The podcast was inspired by a recent conversation where I bemoaned a press release which failed to acknowledge the limitations of self-report data, which I felt could lead to misleading conclusions.

The press release details a recent project which involved interviewing families on their cooking habits. The interesting finding was that most families said they were eating meals at home, which goes against the thinking that people are increasingly eating-out, which is thought to be a contributing factor in the obesity epidemic. Now that's intriguing, but it's hard to tell how much stock to put in that finding, since it's hard to remember how often you eat out, and since eating at home is a socially desirable behaviour - both common issues with self-report data.

So I sent him a review paper (PDF) from about 10 years ago concerning self-report data, and we used it as a launching point.

And, well, it appears we didn't disagree about the role of self-report data as much as we thought we did. But I still think we had an interesting conversation.

Enjoy the blogcast - it's only about 15 minutes long. And Travis even edited in a nice little jingle at the beginning. I've noticed we had an issue with the software, where my half is significantly lower in volume than his half. Any other technical problems? (I've also noticed that I say 'um' a lot. Crap.)

Be sure to let us know what worked, what didn't, how we might improve this in the future, etc. This is just one of what we hope to be many such blogcasts.

You can also directly download the MP3 here.

Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54 (2), 93-105 DOI: 10.1037//0003-066X.54.2.93