Self-Report vs Direct Measures - New Podcast!

Regular readers know that that Peter and I do a semi-regular podcast on obesity-related issues.  This week, I have a discussion with psychology researcher (and fellow ScienceBlogger) Jason Goldman (UPDATE: Jason's thoughts on the podcast here).  For the uninitiated, self-report data refers to information that people provide themselves - questionnaires and interviews are very common examples.  This is in contrast to direct measurement, which is exactly what it sounds like - researchers measuring your height and weight themselves, etc.  The podcast was inspired by a recent conversation where I bemoaned a press release which failed to acknowledge the limitations of self-report data, which I felt could lead to misleading conclusions.

The press release details a recent project which involved interviewing families on their cooking habits.  The interesting finding was that most families said they were eating meals at home, which goes against the thinking that people are increasingly eating-out, which is thought to be a contributing factor in the obesity epidemic.  Now that's intriguing, but it's hard to tell how much stock to put in that finding, since it's hard to remember how often you eat out, and since eating at home is a socially desirable behaviour - both common issues with self-report data (I would have found it really interesting to compare their responses to a direct-measurement of their eating behaviours).  Fortunately the MSc Thesis that the press release was based on actually spends a lot of time discussing limitations of their approach - including some of the issues related to self-report data - but the press release does not.

I think this will come across in the podcast, but I want to be on record as saying that I have no real issues with self-report data per se.  Almost all projects involve at least some element of self-report (e.g. even tightly controlled interventions rely on the participants to tell them about medical conditions, pre-study diet or physical activity, etc), and we frequently discuss self-report studies here on Obesity Panacea.  But it's still important to understand the limitations of self-report, especially when they could fundamentally change the way you interpret a paper.  Failing to do can lead to all sorts of trouble, all of which is nicely outlined in this review paper which Jason and I discuss in the podcast.

Enjoy the podcast, and don't forget to subscribe to future podcasts via iTunes!  This is the first of what will hopefully be many podcasts involving
other members of the ScienceBlogs community.  Thoughts and suggestions
would be greatly appreciated!

Travis

i-fdc9a4633acb6033f34872fcfe7fb67a-itunes logo.gif


ResearchBlogging.orgSchwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54 (2), 93-105 DOI: 10.1037//0003-066X.54.2.93

More like this

If you're a regular reader of Peter and Travis's blog, Obesity Panacea, you may have heard one of their semi-regular blogcasts. Well, since Peter is traveling the world (read about it here), Travis asked me to join him for a blogcast. While discussing topics that we could discuss, a sent a few…
Today we have Scibling Showdown # 2 - the pros and cons of grad school.  This episode includes Jason Goldman of the Thoughtful Animal, Christie Wilcox of Observations of a Nerd and Scicurious of Neurotopia.  We talk about how we chose our current labs, things we wish we had known before we started…
Over the next few months, Peter and I will be re-posting some of our favourite posts from our Obesity Panacea archives.  The following article was originally posted on December 2, 2009. Image by Mike Baird. There is a surprising amount of controversy about the ability of physical activity to…
Image by thienzieyung Most people know that consuming too much fat, and especially saturated fat, is bad for your health.  That's why there has been a concerted push for several decades to get people to reduce the amount of saturated fat that they consume, and to replace it with complex…

Where does the "thinking that people are increasingly eating out" data come from? I'm wondering what would make that the presumed default that the self-reported data are being measured against, rather than the other way around.

Good question. Here is one. It uses self-report data as well, but it lists the limitations in the discussion of the paper, and is also much larger, nationally representative, and examined the same issue over time.

Again, I don't really have a problem with this study or with self-report in general - I just think it's important to take a study's limitations into account, especially when those limitations may have a large impact on the conclusion of the study itself.

Travis

Oh, definitely. I was just thinking especially with the way the economy has been going lately that I would be surprised if people are eating out more, so I wondered where that trend came from.