Self-Report vs Direct Measures - New Podcast!

Regular readers know that that Peter and I do a semi-regular podcast on obesity-related issues.  This week, I have a discussion with psychology researcher (and fellow ScienceBlogger) Jason Goldman (UPDATE: Jason's thoughts on the podcast here).  For the uninitiated, self-report data refers to information that people provide themselves - questionnaires and interviews are very common examples.  This is in contrast to direct measurement, which is exactly what it sounds like - researchers measuring your height and weight themselves, etc.  The podcast was inspired by a recent conversation where I bemoaned a press release which failed to acknowledge the limitations of self-report data, which I felt could lead to misleading conclusions.

The press release details a recent project which involved interviewing families on their cooking habits.  The interesting finding was that most families said they were eating meals at home, which goes against the thinking that people are increasingly eating-out, which is thought to be a contributing factor in the obesity epidemic.  Now that's intriguing, but it's hard to tell how much stock to put in that finding, since it's hard to remember how often you eat out, and since eating at home is a socially desirable behaviour - both common issues with self-report data (I would have found it really interesting to compare their responses to a direct-measurement of their eating behaviours).  Fortunately the MSc Thesis that the press release was based on actually spends a lot of time discussing limitations of their approach - including some of the issues related to self-report data - but the press release does not.

I think this will come across in the podcast, but I want to be on record as saying that I have no real issues with self-report data per se.  Almost all projects involve at least some element of self-report (e.g. even tightly controlled interventions rely on the participants to tell them about medical conditions, pre-study diet or physical activity, etc), and we frequently discuss self-report studies here on Obesity Panacea.  But it's still important to understand the limitations of self-report, especially when they could fundamentally change the way you interpret a paper.  Failing to do can lead to all sorts of trouble, all of which is nicely outlined in this review paper which Jason and I discuss in the podcast.

Enjoy the podcast, and don't forget to subscribe to future podcasts via iTunes!  This is the first of what will hopefully be many podcasts involving
other members of the ScienceBlogs community.  Thoughts and suggestions
would be greatly appreciated!

Travis

i-fdc9a4633acb6033f34872fcfe7fb67a-itunes logo.gif


ResearchBlogging.orgSchwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54 (2), 93-105 DOI: 10.1037//0003-066X.54.2.93

More like this

If you're a regular reader of Peter and Travis's blog, Obesity Panacea, you may have heard one of their semi-regular blogcasts. Well, since Peter is traveling the world (read about it here), Travis asked me to join him for a blogcast. While discussing topics that we could discuss, a sent a few…
When Jim was 12 or 13, he seemed to want to watch almost every R-rated movie that came out. PG-13 movies were okay, and PG and G-rated movies were beneath his dignity. Was he simply interested in these movies because they were forbidden (as we contended), or was he somehow more drawn to the content…
Over the next few months, Peter and I will be re-posting some of our favourite posts from our Obesity Panacea archives.  The following article was originally posted on December 2, 2009. Image by Mike Baird. There is a surprising amount of controversy about the ability of physical activity to…
In the eyes, of anti-vaccine advocates, vaccines bear the brunt of blame for a variety of conditions, including autism, asthma, neurodevelopmental disorders, autoimmune disorders and a wide variety others. Often this link is based on retrospective data, in which parents or patients recall and self-…

Where does the "thinking that people are increasingly eating out" data come from? I'm wondering what would make that the presumed default that the self-reported data are being measured against, rather than the other way around.

Good question. Here is one. It uses self-report data as well, but it lists the limitations in the discussion of the paper, and is also much larger, nationally representative, and examined the same issue over time.

Again, I don't really have a problem with this study or with self-report in general - I just think it's important to take a study's limitations into account, especially when those limitations may have a large impact on the conclusion of the study itself.

Travis

Oh, definitely. I was just thinking especially with the way the economy has been going lately that I would be surprised if people are eating out more, so I wondered where that trend came from.