The Frontal Cortex

Musical Predictions

Posted by Jonah Lehrer on January 20, 2010
(47)
More »

There’s an interesting new paper on how the brain makes sense of music by constructing detailed models in real time. The act of listening, it turns out, is really an act of neural prediction. Here are the scientists, from the University of London:

The ability to anticipate forthcoming events has clear evolutionary advantages, and predictive successes or failures often entail significant psychological and physiological consequences. In music perception, the confirmation and violation of expectations are critical to the communication of emotion and aesthetic effects of a composition.

The paper consists of a computational model and and an experiment. The model essentially demonstrated that statistical predictions based on our personal listening experience – because I listen to Bruce Springsteen, I’m able to predict the melodies of John Mellencamp – was much better at simulating the mind than a rule-based model, in which our expectations are fixed and inflexible.

The experiment was more compelling. The scientists measured the brain waves of a twenty subjects while they listened to various hymns. It turned out that unexpected notes – pitches that violated the previous melodic pattern – triggered an interesting sequence of neural events and a spike in brain activity:

Our electrophysiological results showed that low-probability notes, as compared to high-probability notes, elicited a larger (i) negative ERP component at a late time period (400-450 ms), (ii) beta band (14-30 Hz) oscillation over the parietal lobe, and (iii) long-range phase synchronization between multiple brain regions.

There are two interesting takeaways from this experiment. The first is that music hijacks some very fundamental neural mechanisms. The brain is designed to learn by association: if this, then that. Music works by subtly toying with our expected associations, enticing us to make predictions about what note will come next, and then confronting us with our prediction errors. In other words, every melody manipulates the same essential mechanisms we use to make sense of reality.

The second takeaway is that music requires surprise, the dissonance of “low-probability notes”. While most people think about music in terms of aesthetic beauty – we like pretty consonant pitches arranged in pretty patterns – that’s exactly backwards. The point of the prettiness is to set up the surprise, to frame the deviance. (That’s why the unexpected pitches triggered the most brain activity, synchronizing the activity of brain regions involved in motor movement and emotion.) I wrote about this concept in Proust Was A Neuroscientist:

Before a pattern can be desired by the brain, it must play hard to get. Music only excites us when it makes our auditory cortex struggle to uncover its order. If the music is too obvious, if its patterns are always present, it is annoyingly boring. This is why composers introduce the tonic note in the beginning of the song and then studiously avoid it until the end. The longer we are denied the pattern we expect, the greater the emotional release when the pattern returns, safe and sound. Our auditory cortex rejoices. It has found the order it has been looking for.

To demonstrate this psychological principle, the musicologist Leonard Meyer, in his classic book Emotion and Meaning in Music (1956), analyzed the 5th movement of Beethoven’s String Quartet in C-sharp minor, Op. 131. Meyer wanted to show how music is defined by its flirtation with–but not submission to–our expectations of order. He dissected fifty measures of Beethoven’s masterpiece, showing how Beethoven begins with the clear statement of a rhythmic and harmonic pattern and then, in an intricate tonal dance, carefully avoids repeating it. What Beethoven does instead is suggest variations of the pattern. He is its evasive shadow. If E major is the tonic, Beethoven will play incomplete versions of the E major chord, always careful to avoid its straight expression. He wants to preserve an element of uncertainty in his music, making our brains beg for the one chord he refuses to give us. Beethoven saves that chord for the end.

According to Meyer, it is the suspenseful tension of music (arising out of our unfulfilled expectations) that is the source of the music’s feeling. While earlier theories of music focused on the way a noise can refer to the real world of images and experiences (its “connotative” meaning), Meyer argued that the emotions we find in music come from the unfolding events of the music itself. This “embodied meaning” arises from the patterns the symphony invokes and then ignores, from the ambiguity it creates inside its own form. “For the human mind,” Meyer writes, “such states of doubt and confusion are abhorrent. When confronted with them, the mind attempts to resolve them into clarity and certainty.” And so we wait, expectantly, for the resolution of E major, for Beethoven’s established pattern to be completed. This nervous anticipation, says Meyer, “is the whole raison d’etre of the passage, for its purpose is precisely to delay the cadence in the tonic.” The uncertainty makes the feeling. Music is a form whose meaning depends upon its violation.

(47)
More »

Comments

  1. #1 Dick
    January 20, 2010

    On a less intricate level, does this perhaps explain why a crossword puzzle is no fun unless it poses a challenge; and an academic lecture is most engaging when one feels as though one is sitting on the edge of one’s intellectual seat, just barely grasping the concepts under duscussion?

  2. #2 VJBinCT
    January 20, 2010

    I recall from way long ago a useful rule of thumb about the difference between Haydn and Mozart: if you can whistle the next 8 bars of a piece after hearing 4, it’s Haydn. Also, I recall that Einstein famously said that music is noise. Meaning the unpredictable component holds the music.

  3. #3 Stan Taylor
    January 20, 2010

    As a choral musician, these findings reinforce my observations of music appreciation. Another observation is: a lot of people like completely predictable music. How do we explain their enjoyment?

  4. #4 Ray in Seattle
    January 20, 2010

    Cool article. As a musician I’d add that rhythm and sequence is an under-appreciated dimension of music where such prediction and unfulfilled expectation is possibly even more significant. I suspect scientists focus more on pitch in these experiments because pitch is more accessible to cognition. Rhythm either moves you or it doesn’t and you can’t logically quantify it as easily as cycles per second or beats per minute. It’s even difficult to put it down in sheet music. You have to feel it.

    Syncopation, as in Reggae for example, infuses a piece with a recurring pattern of delayed and on-time notes that provide a background of temporal expectations, violated in a regular way that therefore becomes pleasant when our brain realizes that the on-time note will always resolve the perceived “error” in every measure. I’ve noted that not many people stop to enjoy drum circles in the park where the drummers can not maintain a minimally recognizable regularity to their highly syncopated percussion.

    Notes in music (melody and harmony) can be appreciated as voices floating along on a dependable bed of rhythm. If the rhythm becomes incoherent the notes lose their significance.

    Western music has usually had a strong preference for expression in melody beginning with the Gregorian chants – notes with almost no recognizable rhythm and evolving into the more complex polyphony of the Renaissance whose repeating melodies on regular simple rhythm still forms the basic structure of most Western music. Rap music today takes this rhythm / melody relationship in the opposite direction.

  5. #5 William
    January 20, 2010

    Wagner used this trick all the time – he would strongly point to that tonal resolution, that “relief” everyone was anticipating… and then *BOOM*, the antagonist makes his/her presence, accompanied by a corruption of the music far from any tonal resolution. As a result, that final resolution is that much sweeter (though Wagner had a way of dragging things out a bit…)

  6. #6 Jezebel Jones
    January 20, 2010

    Especially like your posts on music–this was very insightful– thanks, Jonah!

  7. #7 Kristof
    January 20, 2010

    So if music is all about fulfilling and violating expectation, why do we like listening to the same pieces over and over again?

  8. #8 Zoasterboy
    January 20, 2010

    Read about this in This Is Your Brain On Music (a book I recommend to anyone who wants to learn more on this subject).

    @Stan:
    Some people like “simple” music because to them it is not simple. You being a musician have an advanced understanding and expertise in music, therefore you require a more complex musical style to find enjoyment. Others can listen to music that is seemingly simple to you, but thrilling to them.

    @Ray
    In the book I mentioned the author goes into great detail on many of the different aspects of music that can cause enjoyment through surprise, rhythm, timbre, etc, it’s really interesting.

  9. #9 Zoasterboy
    January 20, 2010

    @Kristof

    It gets more complex than what this article covers, this is just a basic reductionist look on how music is enjoyable, but there are other factors such as nostalgia, personal emotional connection to the song/artist(s) and it’s emotional message etc

  10. #10 royniles
    January 20, 2010

    Delayed anticipation makes the event of an hoped for ending the equivalent of a surprise. So where is the “meaning [that] depends upon its violation.” If it’s supposed to involve some alleged “fixed expectations” rule, there is no such rule, as it’s the nature and purpose of biological expectations to contend with an element of doubt.

  11. #11 Ray in Seattle
    January 20, 2010

    @Zoasterboy – I’ve had that book recommended before. Maybe it’s time to buy it. Thanks for the kick.

    Regarding nostalgia and emotional connections to music – could you say in that case the music is a pointer in memory to the emotional experience? Perhaps the music is what lets the brain index the memory more effectively – questions of musical quality aside.

    Scents can do that for example. I walked past a woman at Costco a few weeks ago whose perfume immediately brought memories to my mind of a very sweet girl I dated in high school back in 1959. What a delightful moment that was.

  12. #12 Kevin Vogelsang
    January 20, 2010

    This post could just as easily been about wooing members of the desired sex.

  13. #13 Swift Loris
    January 21, 2010

    One of the best examples of this I know of is the harpsichord cadenza in Bach’s Fifth Brandenburg. When you’re very familiar with it, paradoxically it’s always a continuous series of surprises, each one of which you anticipate. The pleasure of the fulfillment of those anticipated surprises brings tears to my eyes every time I hear it.

  14. #14 wartybliggins
    January 21, 2010

    Interesting, except this does not explain why James Taylor was able to sell 100+ million records.

  15. #15 Phil
    January 21, 2010

    Comedy works the same way: your brain has to quickly link two disparate things (instead of notes that don’t follow each other, words that aren’t normally used together, say) – it’s the journey we make from one word to the next that provides the pleasure.

    You could argue that the harder it is to make the journey, the higher the art is. Unless the comedian’s just really unfunny.

  16. #16 eanmdphd
    January 21, 2010

    What about children, perhaps at the height of learning about the world (not just music) and all its dissonance, who seem to delight in repetitious tunes?

  17. #17 raven
    January 21, 2010

    @eanmdphd I think children like the repetitive stuff because their brains are still forming a model of the world, so they are rewarded when their model-in-progress successfully predicts something. Adults don’t get that same reward.

  18. #18 Front Row Lead
    January 22, 2010

    As a member of the reigning International Champion Men’s Barbershop Chorus, I can tell you that when we surprise the audience with beautifully unexpected (to them) dissonance and harmony, their reaction certainly supports the point of this article. I guess the chorus is changing lives in more ways than we thought.

  19. #19 Ponder Stibbons
    January 22, 2010

    VJBinCT,

    That’s an unfair slight on Haydn. Haydn’s symphonies and string quartets (I’m not familiar with the rest of his oeuvre) are full of surprises and suspense. I actually find Mozart more predictable. Haydn is wittier than Mozart, and one of the chief ways that wit is manifested is by his setting up listener’s expectations and then failing to meet them.

  20. #20 2_Stand
    January 23, 2010

    Just last week my mind brought together ideas that formed the postulation that music is enjoyable because of the conscious mind’s recognition of the mathematically correct order of the sequence of events within the music and that the completion of the piece is the fulfillment of the expected.

    But I came to my end via the proposal that auditory perceptions are sensed and processed before the conscious mind has perceived the results.

    As the music proceeds, the brain processes the perceived auditory sensations, while the mind connects this information to previous experiences. Within that lag time between the computational QED and the conscious mind’s realization of the music’s journey, lies the spark that fires our emotional response to the music.

    Thank you for giving me a chance to express this idea. And thank you for sharing your knowledge.

  21. #21 wendee george
    January 23, 2010

    I heard Jonah last night on the radio for the first time and I’m very intrigued by his research. I’m also quite surprised at how good looking he is. Makes him all the more interesting. Reading his books will be my new endeavor.

  22. #22 Michael
    January 24, 2010

    Thank you for citing Leonard Meyer’s Emotion and Meaning in Music- still one of the most enlightening books on the how and why of music.

  23. #23 bnjmn
    January 24, 2010

    The experiment was more compelling. The scientists measured the brain waves of a twenty subjects while they listened to various hymns. It turned out that unexpected notes – pitches that violated the previous melodic pattern – triggered an interesting sequence of neural events and a spike in brain activity:

    thanks nice post

  24. #24 Alexandra Khouri
    January 25, 2010

    You are a joke>

  25. #25 Rachael
    January 25, 2010

    To Stan Taylor’s comment:

    Perhaps those brains do not wish to be challenged…

  26. #26 EraseBoard
    January 25, 2010

    Learn something new everyday. Been hanging out around on your site for a couple weeks now…
    Figured I would finally reply on one of your posts to let you know I exist :)

  27. #27 Dr H
    January 25, 2010

    As fascinating as this is, it suffers from the same drawback as nearly all of the research I’ve come across in this area. The experiment only looks at a single musical /culture/, in this case, western Euro/American, twelve-tone equal temperment, common-practice harmony variants.

    What would be really interesting would be to see this experiment repeated with, for example, Indoneseans listening to Javanese gamelan music; Indians listening to classical raga; or Japanese listening to traditional Kabuki music.

    Only with that kind of broad-based replication would the experimenters really be justified in claiming that they have uncovered a fundamental -human- trait regarding music, as opposed to simply a -cultural- trait.

  28. #28 Josquin
    January 26, 2010

    Glad to have found your site. I think this phenomenon is an integral part of the enjoyment of music (although not the complete extent of it. ) Dr H: I recently went to a concert of Indian raga music, and experienced the same exhilarating feeling of being “flirted” with. Although their system doesn’t involve the same tonic tones, leading tones, cadences etc. as does Western music, and I am essentially ignorant of Indian music theory, I was still led emotionally and intellectually on a very satisfying ride because of the way the music led me hither and yon, tempted me with resolution, pulled away, led me around again, and finally came back to a very solid accordance. {whew! and now I think I’d better go off and take a cold shower!} Interestingly , the mostly Indian audience expressed appreciation for particular parts of the music by their head nodding, body swaying, and vocal affirmations often at the same time I was most thrilled.

  29. #29 Rozalina Gutman
    January 28, 2010

    I really appreciate this excellent and timely spotlight for the issue. What makes me very sad though is the fact that so very few people would be able to relate (even remotely) to the musical examples he quotes… This persistently forming trend of extinction for the sophistication in music (and other arts to some degree) must be addressed ASAP. What will allow to overcome that steep learning curve that leads to classical music aficionados rank? Music educators know well, that in addition to some innate (natural and evolutionally acquired) capacity to make those mentioned predictions in the process of music perception (and therefore – successfully interact with a complex music canvas), nurturing (as in “nature vs. nurture”) through education is the way to create the music literacy. In other words, the ability to be fully aware and to listen for the mentioned nuances of music is a complex mental skill and is based on having knowledge and practice (as a skill in any other subject that should be taught systematically). Ancient Greeks knew enough to teach it to their young, although today’s education policy creators “evolved” so much to be unable to deem music (and the arts) to be “essential” subjects. (Are remnants of those roaming Neandethals’ genes still fighting to be resurrected..?)
    [By the way, I am also very fond of coverage of this (and many other fascinating music/brain matters) in the excellent new book "The World in 6 Songs: How the Music Brain Created Human Nature" by Daniel Levitin (who is truly a Renaissance man - both research neuroscientist and accomplished musician/producer/composer). Levitin’s publications (incl. earlier book “Your Brain on Music” and video “Music Instinct”) explain very well and in plain language the unique capacity of music practice to engage our brain fully - unlike engagement in any other activity…]

  30. #30 Jenny Bartleson
    January 28, 2010

    I feel like this article describes perfectly why I love music so much and why I seem to have such an emotional connection to it. What’s interesting is that I’m losing hearing in one ear (not related to the love of music) and now different notes and instruments stand out to me in different ways – and these can be songs I’ve been listening to for 15 years. My brain doesn’t know what to do with it – it would be interesting to see how it reacts when I hear a song that I’ve heard a thousand times and suddenly hear a note or an instrument differently.

  31. #31 Chad Kleitsch
    January 29, 2010

    Just for fun, there is a nice demonstration of this idea by George to Ringo in the collaboration of composing Octopus’s Garden. You’ll see it happen at about 1min into the video.

    http://www.youtube.com/watch?v=92NPVgdz6Fw

  32. #32 Kim Brandt
    January 30, 2010

    This reminds me of an old song, “My Romance”: “My romance doesn’t need a castle rising in Spain/Or a dance to a constantly surprising refrain….”. Evidently this principle has been recognized for, well, a long time.

  33. #33 David T.
    January 31, 2010

    My daughter played “send me on my way” by Rusted Root several times for me and i enjoyed its fusion of African drums, flutes and complex rhythms. After a few months away from most music, I was amazed to wake up in the night, the song waking me up. I woke up highly aware of the sounds I’d heard just a few hours past, with astonishingly clear memory of the music, almost as if my brain had developed a deeper awareness of the song’s complexity and I was hearing it more clearly while asleep, so clear that it woke me up. It was as if my brain, overindulged in non-fiction audiobooks (I loved How We Decide BTW) craved some musical “food” and analyzed and played with this song as I slept. It was an amazing example of the way our body rests during sleep while our brain sort of “reformats” itself. I sometimes feel that audiobooks are more beneficial and practical than music, but, through this experience, realized that my brain does not live by words alone, and a little “play,” through music, is equally necessary.

  34. #34 MikeReins
    January 31, 2010

    LISZT is the King of Unpredictable.
    Delicious, irreverent, holy, militaristic, childish, deep, gypsy, courtly, etc. – Sometimes ALL AT ONCE!
    Liszt’s piano pieces are the easiest way to experience the WHOLE romantic era at once. And in an intuitive moment, grasp a feeling for what is was like to want to WANT to be overwhelmed with every kind of feeling! (In my humble opinion!)

  35. #35 Richard Whittall
    January 31, 2010

    So explain to me the lasting popularity of Philip Glass, Steve Reich etcetera, repeating the same predictable patterns over and over again?

  36. #36 MadameFLY
    January 31, 2010

    Delighted to find this article (thanks to the Dish!) In 1997, Robert Jourdain’s book “Music, the Brain and Ecstasy” came to essentially the same conclusion about music: that (among other things) it is another instance of pattern recognition and that part of our pleasure in it is the pleasure in a survival skill exercised … he also writes: “By providing the brain with an artificial environment, and forcing it through that environment in controlled ways, music imparts the means of experiencing relations far deeper than we encounter in our everyday lives…. In this perfect world, our brains are able to piece together larger understandings than they can in the workaday external world…. It’s for this reason that music can be transcendent. For a few moments it makes us larger than we really are, and the world more orderly than it really is. We respond not just to the beauty of the sustained deep relations that are revealed, but also to the fact of our perceiving them. As our brains are thrown into overdrive, we feel our very existence expand….”

    And as for the second takeaway you mention — “The point of the prettiness is to set up the surprise, to frame the deviance. (That’s why the unexpected pitches triggered the most brain activity…” I’d just note that the recognition of the pleasure this affords has been captioned for many years in jazz as a reference to ‘them changes’ … the bedrock of the improvisational experience.

    I enjoy your comments on this subject and would like to reference this article on the BeatConscious page devoted to the brain and music: http://www.beatconscious.org/brain.htm

  37. #37 Chris Darrouzet
    January 31, 2010

    This line of argument, to which I subscribe enthusiastically, derives from the information and systems theorists of the 1950s and 1960s. F. J. Crossen, a philosopher and systems theorist wrote a fine article on this very subject in a 1966 (or thereabouts) collection on systems theory. He argued that what he labels ‘retrospectively recognized redundancy’ (RRR) is that which delights us in any domain, providing us with what we call a meaningful experience; he uses music as one key example. RRR is the occurrence not only of the unexpected from the side of our anticipation of what’s coming next, which interests and delights us, it is the that which we then see in retrospect of its surprising turning up that we could or should have anticipated it but did not—the phenomenon that fulfills an improved algorithm for our anticipation. Great music does this all the time. Crossen was arguing that meaning is indeed something that arises in inverse proportion to “information” as the information theorists defined it (the greater the improbability the greater amount of information a message contains.) So, indeed, it is this dynamic of pattern recognition over time that outlines what could be called a formal definition of meaningfulness.

  38. #38 BM
    January 31, 2010

    Poetry, the closest the arts of language get to music, is like this too.

  39. #39 Dan Sickles
    January 31, 2010

    There’s nothing here that wasn’t taught in my high school music classes in the ’70s or that isn’t intuitive to musicians, composers and most music lovers. Tension/release and harmonic, melodic and rhythmic suspension and anticipation are some of the most fundamental principles in music theory. If a suspension fools me by not meeting an expectation, I, um…expected something else. The fact that “expected something else” is a function of a musical model in my brain is hardly insightful. Learning to love a piece of music by anticipating the unexpected elements is also well known.

    Did any of those earlier “connotative meaning” theorists bother to ask a musician or a composer if music necessarily relates to externalities? Music is its own thing and it’s not about something. It either moves you or it doesn’t.

  41. #41 bnjmn
    February 4, 2010

    The pleasure of the fulfillment of those anticipated surprises brings tears to my eyes every time I hear it.

  42. #42 Your Beeswax
    February 12, 2010

    James Taylor’s sales figure is based on Sound Scan data, which can be manipulated by any accountant. I refuse to believe he sold 100,000,000 plus records. Same for Mariah Carey. They didn’t do it. Those are just numbers. People lie with numbers all the time. Thats what numbers are for, unless you believe in quaint things like Truth.

  43. #43 Free Gift Cards
    April 6, 2010

    Yeah, I strongly agree with you. It seems there are really talented writers who are willing to share a very good articles online.

  44. #44 Daniel Eichenbaum
    July 31, 2010

    I would like to know more about the subjects used in this experiment. What was their musical background like? What were their pre-study musical preference? While I find the implications fascinating and inline with other studies I believe there is a BIG gaping hole.

    Previous exposure to musical/aural stimulus helps train the brain to recognize patterns. What may seem “random” to one person may seem like the logically expected outcome to another. For instance, I’ve used a short Schoenberg piano piece (Op. 11 No. 1) with undergraduate theory students to demonstrate a similar point:

    I play a short excerpt and ask them to talk about the piece. “Chaos,” “unmusical,” and “boring” are three common responses. I then play the sample several more times and then ask them to sing the opening phrase (which nonvocalists hate!). After several attempts I play the example one more time and their answers become a grudging acceptance of the music. And it doesn’t stop there! This acceptance seemed to translate into other works of Schoenberg we look at. This is what it boils down to: Familiarity breeds acceptance! Repeated exposure to any kind of music gives a wider array of aural tools to unlock meaning and enjoyment from other pieces of music.

    Therefor I think tests like those mentioned in the blog post above bring an immense amount of bias even though the researchers most likely didn’t intend for there to be bias. This is what happens, I believe, when researchers unstudied in performing arts attempt to design experiments without consulting musicians I think a synergy between musical and scientific research is necessary for any experiment along these lines to have validity across a wide scope of people from different cultures, educational, socio-economic backgrounds.

  45. #45 peter -
    October 8, 2010

    Wow! great article.

    I have always been fascinated by musical experiences and how they affect our lives. As a speaker box designer the relationship between musical playback and the reality of the true recorded event lies a constant struggle.

    As musicians I think we look for ways to surprise the listener. But at the same time the music must have a pleasing sound. Sounds that are in your face, harsh, or out of rhythm easily draw winces from nearly any crowd. The expressions that some people make when a note is missed can be quite entertaining.

    But what does this say about musical accuracy? Do we expect musicians to be accurate. What about when a note is clearly missed. How does this detour from what is expected from the player? Or from the music? Are we still entertained and is the event pleasing?

    If we experience some violation from what is true (if it is a known musical passage) then we wait and expect the right notes to arrive.

    But what happens when the sound is off? When the violin is out of tune? I think there may be more factors here. To me this theory assumes that each note is played perfectly by an instrument/s that are on key.

    In musical reproduction though, many more things could go wrong. All types of inaccuracies could be introduced. I think a good question would be how does the live musical event differ from a reproduced one?

    And then ask how these events (as discussed in the study above) have an effect on our psyche.

  46. #46 rawimanuraMom
    October 14, 2010

    the Federal-State Unemployment Insurance Program provides unemployment benefits to eligible workers . http://insuranceinstates.com/arizona/Chandler/Insurance%20Low%20Cost%20Health%20Insurance/85226/

  47. #47 ZepInidaped
    May 24, 2011

    Hi man.