There’s been an abundance of PowerPoint advice in the science blogosphere lately. Based on my personal experience, I’d say Chad and Amy give some good advice — and it’s advice that probably serves them well in their own presentations.
But I was curious about something different. There are plenty of places where you can find tips about PowerPoint. The bigger question is, do these tips actually help poor speakers improve their presentations? In other words, can offering basic presentation advice without actually showing someone how to be a more engaging speaker help them give better presentations?
We’ve all all seen or slept through god-awful talks when the speaker droned along in a somnambulant monotone. We’ve even seen other, more engaging speakers give similar talks — and not surprisingly, the talks were more interesting. But if you can’t (or won’t) change your personality or speaking style, will listening to Chad’s tips help?
To try to answer the question, I created five different presentations. Each presentation had the identical oral soundtrack (me reading from a perception textbook), but the accompanying PowerPoint slides were different. Participants were randomly assigned to one of the five presentations, and then told to watch carefully because they would be tested at the end. Before the test, they also read an unrelated passage from the same book, which they were also told they’d be tested on. In reality, we were only interested in the PowerPoint talk, so we never tested people on the reading passage.
The different slides were designed to follow (or ignore) specific bits of Chad’s advice. In just a two-minute presentation, I couldn’t work in every piece of advice, so I focused in on three:
* Text is death
* Explain your graphics, and
* Keep the background simple
In one version of the slides, I included only text: just long, bulleted lists summarizing what I was reading. In another version, I used two graphics, one of which was explained in detail, and one which was not. I also created a version of this presentation with no text at all, only the two graphs. Finally, I made a version of the text-only slides with a horrendous “pain” background graphic I downloaded from a chiropractor’s web site. The fifth presentation was audio-only, with no slides at all.
If Chad’s advice works, people should remember the information presented in the presentation with graphics that I explained the best. The text-only presentation with the distracting background should result in the poorest recall. So, how did our viewers do? Here are their average scores, broken down by the type of presentation they watched:
Holy psychedelic tie-dye, Batman! The worst presentation fared the best! The people watching the presentation with text-only slides and a repugnant background image had the highest scores. They scored significantly higher than any of the other groups. What could possibly be going on here?
One thing I had noticed while the results were coming in was that a surprisingly large number of respondents didn’t answer any of the questions. One possibility is that they didn’t know the answer, but perhaps a more reasonable explanation is that they were bored by the presentation and simply exited the survey before finishing. Let’s re-analyze the data with these participants removed from the results:
Ah, that’s better. Not only did the scores significantly improve, but now there is no significant difference between the scores for any of the presentations.
Wait a minute! No significant difference? But that would mean that the presentation format doesn’t matter at all. You might as well not even bother with PowerPoint.
But there’s one more possible explanation: all these averages may be distorting the larger picture. Let’s take a closer look at the data:
Now we’re looking at question-by-question averages for two different presentations: the presentation with charts and text, and the presentation with text only (both with a plain background). Even though there was no significant difference between the overall test scores, there were significant differences on a per-question basis. Let’s focus in further, on question 1, which asked “When holding your hand in hot water, at what temperature (°C) does your body begin to NOT adapt to the pain?”
The chart that was supposed to help answer that question looked like this:
The correct answer was 47, but the graph depicts a richer vision of the data, which shows that even at 47 degrees, some adaptation may be possible, and hints that eventually we might even adapt to 48-degree water. Perhaps people who viewed the graph didn’t have time to process all that information, or perhaps they were focusing on a different aspect of the data.
The text-only slide looked like this:
The text oversimplifies the data, but also leads viewers to the “correct” answer. Though 97 percent of them got the right answer, people viewing only this slide would probably have difficulty sketching a graph that shows how pain adaptation works, and might not be able to explain the basic concept verbally.
Overall, I’d say this study doesn’t shed a whole lot of light on the question of how best to present information, other than to say “it depends on what you want your audience to get out of it.” Perhaps the most interesting result is that the crummy “pain” graphic did appear to cause more readers to hang around and listen to the whole presentation, so Chad’s advice to never use distracting background graphics may not actually be correct.
On the other hand, this was a 2-minute presentation for a disinterested web audience, not a 50-minute lecture. Different advice will almost certainly apply in a different context. This, of course, was Chad’s first piece of advice:
* Know your audience