Film
April 3, 2007
Category: Film • Language • Perception • Research
If you're like me, when you read a book, you form a picture of what's happening in your mind. I try to imagine not only what the characters look like, but also their surroundings. Eventually, I've created a picture of an entire world in my head. Then, when the book gets made into a film, I'm often disappointed when things don't look the way I imagined them. Could it be that the visual environment I've created in my head is just as appealing as the one created by the filmmaker?
It makes some sense: if we've gone to the trouble of creating a visual environment, why not use the part of our mind that's designed to handle visual information -- the visual perception system itself? Wouldn't it be more convenient to do it that way instead of clumsily describing visual representations using words?
But how can we know whether readers really process language in this way -- processing the visual images the same way we process images that we sensed directly with our eyes? One way is to see if these two types visual representations are treated in the same way. For example, when you see a real scene, you don't keep a precise representation of it in your visual memory; you rely on the fact that the items are present in front of you. If an item disappears when you're not looking at it, you're not likely to remember it was there -- this is change blindness. While it's impossible to make an item "disappear" in a story, a room can go dark, a character can be blinded by bright light, or something can move in front of her so that she can't see some or all of the objects she once saw. If we really represent stories using the visual system, it should be more difficult to recall items that the main character can't see.
Consider this brief story:
1. Russ was in the hospital recovering from minor surgery.
2. In the bed next to him was an older man named Marty.
3. A television was attached to the ceiling between them.
4. At Marty's side was a tall vase of flowers.
5. Russ's friends had given him only a get well card.
6. A nurse came in and drew the curtain around Marty's bed.
7. She wanted privacy while she did an examination.
Now, try to answer this question about the story:
Read on »
Posted by Dave Munger at 3:10 PM • Comments (14)
March 22, 2007
Category: Emotion • Face perception • Film • Research
The Kuleshov Effect, discovered nearly a century ago by Soviet filmmaker Lev Kuleshov, posits that the context in which we see an image of an actor's face will determine the emotion the face portrays. For example, take a look at this short little clip I made (QuickTime required). First you'll see a gray screen, then a photo, then a second gray screen, and another photo of a face, taken just after that person looked at the first photo:
What emotion would you say characterizes the second face? Is it neutral, subtly happy, or subtly sad?
Kuleshov's work suggests that most viewers will see that second face as happier in the context of the happy photo preceding it, compared to if they had seen it on its own. If the identical photo had been preceded by a negative image such as an aggressive dog, people would rate it more negatively.
Research during the 1980s and 1990s confirmed the effect, but it has been exploited by filmmakers countless times during the intervening decades -- the neutral face of a heroine is seen as sad if she's just witnessed her lover's death, but happy if she's anticipating his arrival on the next train.
Most recently, a team led by Dean Mobbs replicated the Kuleshov Effect while viewers' brain activity was monitored via fMRI. Mobbs' team showed movies similar to the example above to 14 volunteers. Sometimes the first (context) image was neutral, sometimes it was positive (happy), and sometimes negative (fearful). While the fMRI monitored brain activity, viewers rated the second (neutral) image as positive or negative. This chart shows the results:
Read on »
Posted by Dave Munger at 5:10 PM • Comments (9)
March 20, 2007
Category: Emotion • Film • Research
On the opening episode of the Colbert Report, faux conservative Stephen Colbert expressed his preference for "guts" over facts:
That's where truth comes from -- the gut. Facts come from the brain -- and some people think that makes facts better. But did you know you have more nerve endings in your stomach than in your brain? You can look it up....
Anyone can tell the news to you. I'm going to feel the news at you.
In Colbert's signature parodic style, he appeals to the "common-sense" notion that "guts" matter more than "brains." Even his dubious claim about nerve endings has some merit -- though the brain clearly has many more neurons, it can't sense tactile input the same way the organs in the human gut can.
The idea that "guts" literally serve some cognitive function isn't as far-fetched as it may seem. Some research has found that different visceral states (e.g. indigestion, heartburn) map on to specific brain areas associated with emotion. The relationship between the gastrointestinal system and the brain is particularly complex, but little research has explored whether there is a direct link between our physical "guts" and our emotional responses.
A team led by Eduardo Vianna found a novel way to explore the relationship between guts and emotion. The key was patients with Crohn's disease, an condition that increases feedback from the nerves in the gastrointestinal system to the brain. Crohn's disease can have active and silent phases, so otherwise similar individuals can be easily compared.
Read on »
Posted by Dave Munger at 4:09 PM • Comments (14)
January 25, 2007
Category: Film • News
I'm currently reading Hugo Münsterberg's fascinating 1916 book, The Photoplay (I'm reading a paper copy, but the link takes you to the complete online text). It's one of the earliest serious works on film, which was unfortunately not well received at the time it was published due to the start of World War I and Münsterberg's strong German nationalism (he was a professor at Harvard at the time).
Anyway, I wanted to direct your attention to a couple descriptions of amazing research conducted with the extremely limited tools available at that time:
Read on »
Posted by Dave Munger at 2:07 PM • Comments (0)
January 9, 2007
Category: Film • Perception • Research
In movie fight scenes, punches often miss by a foot or more, but when sound effects are added, and the punchee adds an effective-looking recoil, we're convinced that the punch is "real." We've posted on this phenomenon before: when a "click" sound is played as two animated balls pass by each other, it's perceived as a "bounce."
This type of sound effect, where the sound occurs at a critical moment in an animation or movie, has been explored quite extensively. But is it possible that other types of sound might affect how we see motion? Adam Ecker and Laurie Heller realized that they could apply sound to the research stimuli of Daniel Kersten. In Kersten's research, a ball moves around a virtual 3-D box, and the appearance of a shadow causes viewers to view the identical ball movement in different ways. What if sound is added to the moving ball? I've modified a couple of Kersten's demos, adding sound to suggest different types of movement.
In the first two examples, the motion of the ball and shadow is consistent with with the sound effects. In the third example, the shadow suggests that the ball is moving along the ground, but the airplane sound effects might suggest a lift-off and landing. In the final example, the tire skidding suggests motion along the ground, but the shadow again suggests flight. Which input matters most in motion perception? Ecker and Heller designed a series of comprehensive experiments to find out.
Read on »
Posted by Dave Munger at 10:26 AM • Comments (4)
June 14, 2006
Category: Attention • Film • Memory • Research
In many ways, my career has been dominated by efforts to make "work" bear as much resemblance to "having fun" as possible. Today's article only confirms that rule. Yesterday afternoon, I spent an hour watching a World Cup soccer match, and for once I could claim that it was completely relevant to work. I could still remember Diego Maradona's stunning 1986 run through the center of the English defense as vividly as if it happened yesterday, and was hoping to form a similarly vivid memory. Unfortunately, yesterday's lackluster 0-0 tie between France and Switzerland didn't provide me with anything remotely approaching that experience.
So what does all this have to do with cognitive psychology? If you study visual memory, quite a bit. One problem that's interested researchers for decades is the question of how visual memories are represented. Do we encode the 3-D event in memory, so that if we saw it from a different angle our recollection would be just as good? Or do we encode it only from the vantage point we saw? Soccer matches offer a perfect means of studying this phenomenon, since most of us watch on TV, with a single camera view.
A group led by Bärbel Garsoffky developed a method of testing these questions. They showed German college students videos of key goal-scoring plays from soccer matches in the 1988 World Cup. They chose only games during which one team scored at least two goals; the students were only shown a video of one of the goals. Then they were tested with still photos. Eight of the still shots were were from the play they had watched, and eight were from the other goal-scoring play during the same match. The students had to indicate whether each still image was part of the play they had watched. Two of the eight "correct" photos were taken from the same vantage point, but the other six were taking from different angles -- either 45, 90, or 135 degrees away from the initial view. Here are the results:
Read on »
Posted by Dave Munger at 11:54 AM • Comments (3)
May 4, 2006
Category: Film • Memory • Research
Over at Uncertain Principles, Chad Orzel's hosting a discussion on who should be on a hypothetical Mount Rushmore of science. There's a fairly broad consensus that Darwin, Einstein, and Newton make the cut, but rather heated debate on who should be the fourth member.
Many of Chad's readers suggest Sigmund Freud. I found that surprising, since the field of psychology has largely moved away from the work of Freud. Freud is still very influential in literary and cultural studies, but not so much in the world of science. Indeed, one of Freud's lasting influences was the attempt to undertake a rational analysis of literature; to analyze the motivations of characters as if they were real people (and vice versa).
Though Freud's analysis of the human mind has little basis in science, scientists do still attempt to understand literature and other forms of art, but their methods have changed considerably. For one thing, researchers acknowledge that "high art" is perhaps too nuanced to withstand scientific analysis, and so have focused on simpler works. Instead of Oedipus Rex, think Moonraker.
Read on »
Posted by Dave Munger at 3:24 PM • Comments (2)
March 23, 2006
Category: Film • Memory
What's it like to have all your memories erased? Well, not all your memories, because if that happened, you'd simply be like a newborn infant, and you'd have to relearn everything. The more interesting scenario is to lose only certain memories -- the memories that most people think of as "true" memories: episodic memory.
Memories can be divided into three rough categories: episodic, semantic, and procedural (there are actually many more categories). Procedural memories are the memories of how to do things: driving a car, walking, sewing, and so on. Semantic memories are bits of factual knowledge: names, places, meanings. Episodic memories consist of the explicit recollection of specific events: prom night, my trip to Monument Valley, and so on.
A few nights ago, Greta and I watched the movie Unknown White Male, which purports to be a true case history of a 33-year-old man, Doug Bruce, who suddenly and inexplicably lost his entire episodic memory and much of his semantic memory. In fact, the movie demonstrates the inadequacy of such categories -- what, for example, is the memory of the concept "ocean"? In one sense, it's purely semantic: an ocean is a large body of salt water. But when Bruce visits the ocean for the first time following his amnesia, he is overwhelmed with a vast array of new sensations, from the sound and power of the surf, to the water filtering the soft sand through his feet. This particular ocean visit could become an episodic memory, but isn't there a semantic aspect to the power of the surf? Bruce can't remember whether or not he can swim, but when he dives into the water, he quickly realizes he can stroke effortlessly through the waves. Is all knowledge of swimming procedural, or are there finer points of such knowledge, such as "keep your elbows up" better characterized as semantic?
Read on »
Posted by Dave Munger at 1:09 PM • Comments (10)
March 21, 2006
Category: Film • Research • Social
What's the best way to ensure that law enforcement officers don't abuse their authority and coerce innocent suspects into confessing? Yesterday we discussed research suggesting that a side-view videotape of a confession was more likely than a head-on view to result in an accurate assessment of whether that confession was voluntary or coerced.
But the Lassiter team's study was still open to some key criticisms. First, the study participants were all college students -- certainly not a typical jury demographic. Second, jurors don't see videotaped confessions in isolation -- when a confession is disputed, the prosecution and defense both offer additional arguments. Finally, the study only considered two video angles. Perhaps a different angle can produce even better results.
The team doesn't dispute these concerns, but instead points out that less-expensive studies using college student participants can be conducted to see if a line of research is valid. If those results are promising, then they can move on to more realistic scenarios.
Read on »
Posted by Dave Munger at 11:16 AM • Comments (0)
March 20, 2006
Category: Film • Research • Social
College student Bradley Page dropped his girlfriend off in a park one evening, only to learn later that she had been murdered and buried in a shallow grave. Police investigating the death interviewed him about the incident, repeatedly asking him why he could have left her alone in that park. "It was the biggest mistake of my life," he responded in anguish. Eventually, the officers told him that witnesses had seen him near where the body was buried and that his fingerprints had been found on the murder weapon. These statements astonished Page, who hadn't even remembered leaving his apartment. There was good reason for his astonishment: both statements were lies.
Page, still credulous, asked the officers if it was possible for him to forget these details, to "blank it out." They told him such occurrences were common, and it might help to assuage his guilt if he told them how he might have killed his girlfriend. He obliged them, giving a hypothetical description of how he would have committed the crime. Two hours later, the police told him that they considered his statement to be a confession. Page, thoroughly surprised by this turn, immediately retracted the confession.
The confession was, however, admitted into court, and ultimately led to Page's conviction for manslaughter. Page's confession was recorded on audiotape and a complete transcript of the interrogation was made, but cases like this one have led to calls for videotaping of all police interrogations, so that juries can judge for themselves whether a confession is coerced. Videotaping has already been adopted in some U.S. states. But the research of G. Daniel Lassiter and his colleagues has suggested that even videotaped confessions can bias the jury: study participants who viewed reenactments of confessions from a face-on perspective were more likely to view the confession as legitimate than those who watched a side view.
Read on »
Posted by Dave Munger at 12:05 PM • Comments (5) • TrackBacks (1)
March 10, 2006
Category: Casual Fridays • Film
Does being a movie expert make you a better predictor of the Oscar winners? Comedy Central pundit parodist Stephen Colbert claims that he made his oscar predictions without having seen any of the movies, but then went 5 for 5, even predicting the upset of the year, Crash, to win best picture. If you take Colbert's case as an example, it appears that no expertise is necessary to predict the Oscar winners. Of course, since Colbert's TV character is a parody of conservative political commentators, we can never be quite sure if he's being on the level about his level of movie expertise.
At Cognitive Daily, by contrast, we've now got some hard (though nonscientific) numbers on the question of whether it pays to be a film expert before making your Oscar picks. Ninety Cognitive Daily readers responded to our survey asking them to give predictions in four categories and then report their level of expertise, measured by how many of the nominated films they had actually seen. Here's how the level of expertise was distributed among our respondents:
By far most of our respondents had seen fewer than three movies, with the largest number of repondents answering with the Colbertian "0" movies seen. But did the viewers who had seen more movies do better in predicting the winners?
Read on »
Posted by Dave Munger at 11:39 AM • Comments (4)
February 8, 2006
Category: Development / Aging • Film • Research • Social
One of the first questions our son Jim asks when a new movie comes out is "what's it rated?" The more "adult" the rating, the more appealing the movie is to him: PG is the lowest rating he'll even consider, PG-13 is better, and R is best. Since he's only 14, we don't take him to many R-rated movies, which is possibly what adds to their appeal.
But even PG-rated movies and TV shows still display an abundance of violence, and plenty of parents are happy to let their kids watch violent programming, especially if there isn't any sex involved. We've reported on a lot of media violence studies on Cognitive Daily, (for a summary of the effects of violent media on youth, see here, here, and here), but it still came as a surprise to us that until now, there hadn't been a brain activation study of children exposed to media violence.
A team led by John Murray has addressed that deficiency with a new experiment on 8- to 12-year-olds. They began by pre-screening 40 children by monitoring physiological responses to violent videos. They selected 15 children who showed similar responses to the videos: they were "accelerators," whose heart rate increased when viewing violence. This would allow them to more easily compare brain responses in the second phase of the study.
Read on »
Posted by Dave Munger at 1:01 PM • Comments (13)
February 24, 2005
Category: Film • Language • Research
Literary theory is being influenced more and more by research in cognitive psychology, and as the previous article I discussed showed, psychology research is also influenced by theory. Today's article, "Generating Predictive Inferences While Viewing a Movie" (Joseph P. Magliano, Northern Illinois University, and Katinka Dijkstra and Rolf A. Zwaan, Florida State University, in Discourse Processes, 1996), is another example.
In much film theory, the key techniques used to tell a story are mise en scène, montage, cinematography, and sound. Mise en scène is everything used to create the set, including costuming and stage directions. Montage is simply the techniques used to edit film shots together. Cinematography is the technical craft of shooting a film: the camera, lens, film, filters, framing, etc. Sound, of course, includes music, sound effects, and dialog. Film theorists like David Bordwell have suggested that a combination of several of these devices is more likely to have a significant impact on film viewers than one device employed on its own.
Magliano and his colleagues wanted to test Bordwell's theory, so they devised two experiments. In the first one, college students watched the over-the-top James Bond flick Moonraker on a VCR, and were encouraged to stop the tape and write down predictions as to what would happen next, whenever such a thought occurred to them. For example, when "Jaws" fell out of a plane without a parachute, and then the film cut to a circus big top, many viewers "predicted" that he would fall into the tent. Magliano et al. eliminated predictions made by only one viewer, and still counted 98 different predictions. They then analyzed the predictions to find the most common ones and see what preceded them in the movie. They redefined Bordwell's categories into 5 "types of support" for predictions: Mise en scène, Montage, Framing, Music, and Dialog. 57 of the 98 were supported by more than one category (e.g. Montage and Framing). Montage was the most common support for a prediction, with all the other categories accounting for roughly the same number of predictions.
Then the looked at each individual prediction and analyzed what categories were most likely to generate a prediction. Perhaps most surprisingly, Music was unrelated to prediction generation. For all the hype about the importance of music in film, it seems unrelated to whether a viewer will make a prediction about what will come next. What did matter, perhaps less surprisingly, was the number of cinematic devices supporting a given prediction. The more devices, the more likely a viewer was to arrive at that prediction.
In their second experiment, Magliano et al. wanted to know if viewers would make the same predictions without being asked specifically to make predictions. They took another group of college students and showed them the film again. This time, instead of asking them to make predictions, they simply stopped the film at moments when predictions were made by the first group, and asked the participants to write down whatever came to mind. They also stopped the film at several random points where no predictions were made before.
They found that viewers were indeed still making predictions, and they were more likely to make the predictions that the first group had made when the prediction was supported by more than one cinematic device. The effect was dramatic:
| Percent of viewers making the same prediction as in Experiment 1 |
74 |
44 |
14 |
| |
Multiple sources of support |
One source of support |
No source of support |
Magliano et al. realize that these devices are used intentionally by the filmmakers—so therefore the filmmakers must want viewers to make predictions. Why? They suggest that making predictions helps keep viewers involved in following the film. Taking the example of Jaws falling into the big top, filmmakers were so certain that viewers would make the prediction that they didn't even show Jaws actually landing. They showed him falling without a parachute, then began playing circus music, then cut to the big top, and back to Jaws, and finally back to the inside of the tent, but then moved on to another scene. But viewers were not surprised to see Jaws show up again later in the film.
In Bond films, I think there might be another reason the filmmakers want viewers to make predictions: they want viewers to feel sophisticated. Part of the appeal of the Bond film is that viewers are supposed to be excited by the lifestyles of the wealthy socialites depicted in the film. By encouraging them to make predictions, the filmmakers invite viewers to be a part of that lifestyle; to live it vicariously through Bond.
While some may believe this research to be somewhat anticlimactic since it only supports Bordwell's theory of film narrative, I should point out that in Monika Suckfull's work that I discussed previously, the film theorist's view was not supported. "Unsurprising" results in psychological research often tell us as much as surprising ones do.
Posted by Dave Munger at 6:02 AM • Comments (0)
February 17, 2005
Category: Film • Perception • Research
Most schools of literary criticism suggest that it's fruitless to attempt to consider what the intentions of the author are; we can only examine the "text" itself: it is the only solid evidence we have. Similarly, critics toss up their hands when trying to comprehend the experience of the reader of a text. While the notion of "author" becomes even more complex when we consider film, examining the experience of the film viewer does seem to be attainable: since films are experienced in real time, we can compare the experiences of different viewers while they watch the same portions of a film. We can even chart and compare their body and brain functions across the course of an entire movie.
Monika Suckfüll of University Jena tries to do that in the research she reports in her article "Film Analysis and Psychophysiology: Effects of Moments of Impact and Protagonists" (Media Psychology, 2000). Suckfüll chose to study Jane Campion's film The Piano, showing it to 62 observers while charting their movements and heart rate. She divided the film into 30-second segments and analyzed it according to Peter Wuss' theory of film narrative, identifying seven key "moments of impact" in the film, such as when Sam Neill cuts off Holly Hunter's finger. Suckfüll hypothesized that these moments would be periods of increased attention by film viewers. Previous research has noted that participants' heart rates tend to decline when they are trying to focus their attention, so monitoring viewers' heart rates should help confirm or disprove Suckfüll's hypothesis.
Suckfüll was surprised to find that participants' heart rates behaved quite unpredictably at these key moments. She did find a correspondence between heart rates and events on the screen, but rather than corresponding to "moments of impact," they coincided with "topic lines" or motifs. Heart rates rose (signaling anxiety or the "defense reaction") when the "angel motif" occurred—typically when Anna Paquin was about to betray Hunter. Heart rates declined during the naughty parts (the "sex motif").
Suckfüll's discussion seems to indicate disappointment in her results, but they do offer some interesting insight into viewers' response to film. It seems to me that our physiological reactions to film are occuring over a much shorter time frame than our intellectual reactions. We can determine "moments of impact" in a film only after viewing the whole thing, so real-time analysis of viewers' responses to the film can never offer a complete explanation of the whole. It is interesting to recall that much of the critique of the film, when it was released, centered on the idea that it was thinly-veiled soft-core porn. Perhaps this reaction could have been predicted, had the filmmakers performed an assessment of the film similar to Suckfüll's study.
Posted by Dave Munger at 10:02 AM • Comments (1)
January 20, 2005
Category: Film • Memory • Music • Research
Today's reading is "Musical Soundtracks as a Schematic Influence on the Cognitive Processing of Filmed Events," by Marilyn G. Boltz of Haverford College (Music Perception, Summer 2001).
All film is illusion. The illusion of motion is created by a sequence of static frames. The illusion of a three-dimensional world is created by a two-dimensional photograph. What role does music play in maintaining that illusion? A big one, it turns out. Marshall and Cohen (1988) found that by showing cartoons where the only "characters" were a triangle, a circle, and a square, changing the music could change viewer's perceptions about the "characters" (it turns out, triangles are thugs). Other researchers have found that appropriate music can help you remember details about a scene in a film.
What Boltz wanted to know was whether music alone could change the way viewers thought about a scene in a film, and furthermore, whether it could actually affect viewers' memory later on. She showed viewers three ambiguous scenes, from "Cat People," "Vertigo," and the TV show "The Hitchhiker," and played either "positive," "negative," or no music to accompany the scenes.
Boltz found that when viewers watched Malcolm McDowell and Nastassia Kinski talk in a benign scene accompanied by "positive" music, they saw McDowell primarily as "kind/caring," "loving," or "playful." When the "negative" music was played, he became "crazy/deranged," "evil," manipulative," "controlling/possessive," and "mysterious" (and this is without seeing him turn into a black leopard and rip someone's arm off). When asked to predict what would happen next, viewers who had never seen the film and who saw the version with "positive" music ("Blossom Meadow" by George Winston) thought that McDowell and Kinski would have a happy life together and possibly fall in love. Viewers who instead saw a version with "negative" music (from Rubycon by Tangerine Dream—of "Risky Business" fame) thought McDowell would "harm," "kill," or "do supernatural harm" to Kinski. The results were similar for scenes from "Vertigo" and "The Hitchhicker." So music matters, whether we're watching a bad '80s HBO series or a Hitchcock classic.
A separate group of viewers was not asked to give predictions. Instead, they were invited back to Boltz's lab a week later to be given a pop quiz. It turns out, when shown a list of words, some of which were names of objects that appeared in the film clip and some of which weren't, they correctly remembered about two-thirds of the objects. But people who listened to "positive" music remembered about 80 percent of the "positive" objects in the scene, as opposed to only half of the "negative" objects. For people who heard the "negative" music, the results were correspondingly reversed. Furthermore, people hearing "negative" music were more likely to remember negative objects that weren't even present in the scene they were shown, such as an open grave and an ice pick in "Vertigo."
This suggests a new way moviemakers can mess with viewers' heads. By playing "positive" music in the presence of a "negative" object they don't want viewers to remember, they can fool their audience into all sorts of misperceptions. Or they could use consistent music in a seemingly innocuous scene to force viewers to pay more attention. Of course, filmmakers didn't need Boltz to tell them most of this stuff—they've known it for years. But Boltz has finally quantified it for us in a way that shows how significant it really is.
Posted by Dave Munger at 2:01 PM • Comments (0)
Cognitive Daily reports nearly every day on fascinating peer-reviewed developments in cognition from the most respected scientists in the field.
