Ever since Pac Man, video games have obeyed a few basic principles: A player sits down in front of a screen and presses a few buttons with his or her thumbs. Perhaps there's a joystick involved, or maybe the index finger has to do some work, too. But the body is essentially still. The only moving parts are the eyes and the fingers.
The Wii changed everything. Unlike every other game console, the Wii controller isn't built around a confusing alphabet of buttons. Instead, Nintendo uses some nifty bluetooth technology to translate our body movements directly onto the screen. When we swing our arms, a baseball bat moves. When we make a jabbing motion, Super Mario lands a punch. It doesn't matter if we're bowling or golfing or imitating Jimi on Guitar Hero: the video game console requires that our body is always moving. We might even break a sweat.
This physicality is the Wii's real innovation. It's also the reason why Microsoft is so heavily invested in Project Natal/Kinect which, like the Wii, requires users to move about the living room. While Nintendo and Microsoft argue that their wireless controllers make game play more intuitive - you no longer have to remember arcane sequences of buttons - their interfaces actually do something much more powerful: By involving our limbs in the on-screen action, the Wii and Kinect make video games much more emotional.
To understand how the Wii turns stupid arcade games into a passionate experience, we have to revisit an old theory of emotion, first proposed by William James. In his 1884 article "What is an emotion?" James argued that all of our mental feelings actually begin in the body. Although our emotions feel ephemeral, they are rooted in the movements of our muscles and the palpitations of our flesh. Typical of his work, James' evidence consisted of vivid examples stolen straight from real life, such as a person encountering a bear in the woods.
"What kind of an emotion of fear would be left," James wondered, "if the feeling of quickened heart beats nor of shallow breathing, neither of trembling lips nor of weakened limbs, neither of goose bumps nor of visceral stirrings, were present?" James' answer was simple: without the body there would be no fear, for an emotion begins as the perception of a bodily change. When it comes to the drama of feelings, our flesh is the main stage.
For most of the 20th century, James' theory of bodily emotions was ignored. It just seemed too implausible. But in the early 1980s, the neuroscientist Antonio Damasio realized that James was mostly right: Many of our emotions are preceded by changes in our physical body. Damasio came to this conclusion after studying neurological patients who, after suffering damage in their orbitoprefrontal cortex or somatosensory cortex, were unable to experience any emotion at all. Why not? The tight connection between the mind and body had been broken. Even though these patients could still feel their flesh--they weren't paraplegic--they could no longer use their body to generate feelings. And if you can't produce the bodily symptoms of an emotion--the swelling tear ducts of sadness, or the elevated heart rate of fear--then you can't feel the emotion. As Damasio notes, "The mind is embodied, not just embrained."
How might such a neurological process unfold? Let's say we are playing Super Mario Galaxy 2 on the Wii. Unlike other game consoles, which leave us stranded on the couch, the Wii actually makes us move. If we want to kill off the Goomba, we need to run around, twirl the remote, and, once we've maneuvered close to the evil character, jump on top of him. We are no longer just twiddling our thumbs.
In order to prepare for all this combat, the brain automatically triggers a wave of changes in our "physical viscera," such as quickening our pulses, flooding our bloodstream with adrenaline, and contracting our intestines. While even stationary entertainment can lead to corporeal changes - that's why the pulse quickens when watching a Hitchcock movie - the physical activity involved in fighting off the Goomba, exaggerates these effects, because our active muscles need oxygenated blood. Although we might look a little foolish, the game has managed to excite our flesh, and that means our emotions aren't far behind.
All these bodily changes are then detected by the orbitofrontal cortex and somatosensory cortex, which connect them to the scary sensation (the Goomba) that is causing us to move in the first place. As Damasio puts it, "the essence of feeling an emotion is the experience of such [bodily] changes in juxtaposition to the mental images that initiated the cycle." The resulting state of consciousness--an emulsion of thought and flesh, body, and mind--is our feeling of fear. It is an idea that has passed through the vessel of the body. We are suddenly terrified of a cartoon Shitake mushroom.
For decades, video game designers have been obsessed with visual realism, as if the eyeball was the key to our emotional brain. But accurate graphics have diminishing returns. At a certain point, we don't need more pixels - we need more physicality. For the first time, video games are taking advantage of their specific medium, exploiting the features that other entertainments (such as movies and novels) are missing. No other form of culture, after all, depends on the verb "to play". (We play video games - we don't watch or read them.) But here's the thing about playing: it's much more captivating when the play itself is a physical act, when we play not just with the mind but with the body.
- Log in to post comments
We'll ultimately end up with a star trek type holodeck where we wil have fooled our brain entirely. Can't wait!
Thanks - now my son HAS to get one of these blessed things. They do change the dynamics, I have to admit. I have my grandson punching me out every time. He has no idea how this works, but he moves a lot and it gets me.
Thanks for your column / blog. Always enjoy it.
You've never actually played any of these games have you? Or even seen someone else play them who is being paid to do so.
Here for instance, is someone playing mario galaxy, sitting on a couch, controlling it with small motions of the wrist and button presses:
http://www.youtube.com/watch?v=H4wnkLKgfgQ
"But the body is essentially still. The only moving parts are the eyes and the fingers."
You haven't seen me playing Descent or Gran Turismo or Halo.
I think you're missing the mark here. As a couple of commentators above point out, many Wii games actually require very little motion. In fact, most play better when you don't flail around. Even though it might feel more natural to try to take a big swing in Wii Tennis, the game responds much better to a flick of the wrist (note: this may be different with the Wii Motion Plus add-on, which I haven't yet tried). I have some friends who loathe the Wii's motion controls, saying, "They're just more buttons!" Which is completely true for most games.
Now, it might be a failure on the part of the player, that we're too conditioned to old style video games to really get into the wii in the same way. But I personally don't have any problems getting emotionally invested in games that don't require much movement. (I will grant, however, that my favorite games are sports.)
Secondly, how on earth can you say this, "But here's the thing about playing: it's much more captivating when the play itself is a physical act, when we play not just with the mind but with the body." Umm, chess? Poker? Go? Some of the most compelling and enduring games are completely non-physical. People can play games of chess in their heads, simply saying the moves out loud. It's hard to get less physical than that!
I think this article took a reasonable conclusion a bit too far.
Never heard of DDR?
Ever hear of Dance Dance Revolution? It wasn't Wii that started movement games.
And the Playstation Eye with movement games also preceded the Wii, but wasn't well marketed.
After reading this, I have to admit that I too am left wondering whether or not you play many games.
..."a confusing alphabet of buttons"
You sound like somebody's grand-dad (or else a Wii salesman!) describing a pre-Wii video controllers like this. On even the most complex controllers, there is a handful of buttons, and most games typically only use a few of them. Children can master them with little difficulty.
Game developers are right on the money in focussing on visual realism for emotional impact. I recall playing the game Half Life 2 (now about 6 years old) and then afterwards, wandering the alleys near my home and being startled at how my mind was now so focussed on all the textures and patterns on the surfaces - broken concrete, dirty puddles of water, peeling paint, a breeze blowing rubbish about. These are all visual features within the post-apocalyptic game I had been playing, and increasingly, such attention to detail is being captured in contemporary games.
In the games I have played recently, emotional content and immersion has been most successfully created with the clever combination of music, and sound effects with the action, and with visual and sound effects to mimic bodily sensations - e.g. having the screen go blurry or be covered in a red haze when your avatar is near death, hearing a high-pitched tone or buzzing sound effect after an explosion, mimicking ringing ears, shaking the screen when an exposion goes off, slowing down time in game during a crucial moment e.g. a heavy blow or a headshot, to mimic being lost in the moment etc.
In contemporary first-person army type games, atmosphere is created by having AI squad mates who conjure emotion with the tone of their voices in game, screaming out orders, pleading for help, uttering warnings etc.
The game F.E.A.R which is now more than a few years old, was incredibly successful for using clever visual effects and sound to conjure up real fear, to the point that I could rarely play this game for more than 15mins at a time, my heart would beat so rapidly.
The Wii pales in comparison.
Especially in a well-designed first-person game which is the type of game that seems the most immersive, once you have been playing for a while, you almost seem to forget you are sitting in front of a screen and holding a controller. Your movements become the movements of your avatar. you are no longer clicking a button, or pushing a joystick, you are running through trees, and jumping.
I am reminded of the experiments detailed in "The Brain that Changes Itself" (e.g. blind people 'seeing' using a matrix of imprints on the back) which seem to simulate one sense with another - once you are immersed in the game and the unreality, you feel more like you are performing the actions in the game, not simply clicking buttons.
I also don't consider the Wii to involve a reversal of brain and body influence. We were physically exhausting ourselves at the Pac Man machine in the 1980's, which was just as near a physical experience as you describe. That said, the basic point on true digital interaction, or a more modern virtual reality is an important one. I think it will likely not be through Wii style gaming though that this will happen. More likely it will be from making something like Second Life more physically interactive. Where slow and thoughtful movements could be recognized as they are intended.
more input on âembodied cognitionâ check out
http://www.core77.com/blog/columns/to_change_hearts_and_mindschange_the…
there are some links at the bottom to some scientific studies
interesting ramifications for visual art ...
Well, I tried to link to a demo of the Playstation's version of this tech, called "Move". Apparently it's hung up in moderation. Search youtube for "sorcery demo e3" if you want to see it.
My point was that this game combines the controllers of the Wii - a physical object you hold - with the motion-recognition of the Xbox's Kinect. I kinda thought it illustrated the notion of immersion and physical engagement that Jonah was talking about. Oh, well.
Steve makes a good point that even 'chair bound' games can inspire real emotion - but Jonah does have a point, in that most people don't invest themselves that much in a game. 'Hardcore' gamers are a minority.
I think the Wii succeeds not by heightening emotion - though that's a factor - so much as by using simple motions that people are already used to. Making the learning curve as gentle as possible. The vast majority of Wii games involve simple actions and objectives. A lot of games - the "Party" series, or Wii Sports - are nothing but minigames - thus being appealing (or at least, not intimidating) to people beyond the 'hardcore'.
Gamers are a surprisingly passionate lot, for many of them its like the chess experience (they look like they're dispassionately sitting there but there's a lot more going on). The Wii and Natal/Kinect really are for those who were turned off by the lack of physical interaction. They will possibly bring more life to the gameplay of gamers as well but I'd say they are targeted at showing the other people who haven't quite gelled to the experience that games are for everyone.
Gaming has evolved into something that brings people together across distance and bringing physical bodies into it is the next step to bringing another dimension into play. The problem has been that the average non-gamer doesn't see it as the answer to the physical distance we've gotten as a result of our mobility. Gaming provides a space in which people can share meaningful events even when they are not in the same local area. Adding a physical element will add something more but I think it is wrong to think that what has been gaming is not already a fairly rich experience.
People pretty much accept the movie going experience as one that can be social and pleasurable but don't tend to apply that to gaming. Gaming adds a richness to the screen that movies/tv do not. You can play alone and challenge yourself. You can play multi-player with the novice who isn't just having fun playing the game but also because this is a shared experience. Being able to share this as a fully physical experience will add another layer but in no way does that negate all the layers which came before for what they are and what they represent.
"No other form of culture, after all, depends on the verb 'to play'."
Really??? Sports come to mind as something that have been around a lot longer than the Wii to say the least, and they are very physical (some more than others) and as a result impart a lot of the same psychological benefits purported to be exclusive to the Wii.
In my experience, having played video games as far back as pong on my Atari 2600; I *do* think that the Wii is revolutionary. Certainly, other systems explored the concept before. And absolutely - I sweated out some of the spookier levels of Halo on the xBox with my son - who just didn't get why I had to "twitch my head" or "twist my whole upper body" when I got into a tight spot. (I blame lack of peripheral vision; and game-lag).
My son compensated, of course, by orchestrating "chords" and "arpeggios" of button and joystick presses, that came from his brain in a timed-series, programmed to anticipate lag, and deal with known-situations that were outside of range-of-vision. Faster youthful reflexes, and a tremendous library of canned muscle-memory responses to the game; where I tried to rely on pure situational awareness and response. (I also found the changes in how the button controls worked from Halo2 to Halo3 very disorienting, and as an adult with a full-time job, I never put in the "hours" needed to learn the new interface.)
That's what was engaging my fear response. (and why my body was attempting to move stuff, unrelated to game-controller hardware). I'm sure of that.
My problem with the Wii was that it's not accurate or quick enough. I can always come up with an external excuse for my own failures. :)
What's going on with Wii; sure some games are going to be throwbacks on that system. But now other systems are pushing this style of interface forward, as Jonah says. More immersion will necessarily involve those emotional reflexes more, and gamers will have to rely on the repetitive rehearsal mechanism less, and the primitive situational awareness and response mechanism more. (of course, we're also going to need better 3d, that can pan and focus as quickly as we can glance with our eyeballs, so we're not forced to fight through the postage-stamp of a 40" video monitor - lol.)
jonah: are you familiar with the japanese anime "ghost in the shell"? I guess the writer of the series read William James ideas and play a little with them.
The main character of the series, a cyborg, asks herself every now and then if she's still human because in an accident she lost everything but the brain and the spinal cord. once in a while she feels empty, devoid of emotions. One of the assumptions in the script is that some kind of technology had to be developed in order to allow a "human mind" inhabit a mechanical body. This technology is precisely the emulation of body sensations like hunger, cold, heart pulse, pain, touch, etc. The premise is: the mind needs that continuous inflow of body information to to keep being a human mind.
it was only an interesting premise when i watched the series, but now that i read about experimental research....oh, it's mindfuck stuff! great post!
As a...somewhat older...person who owns a Wii, it is fun to watch children who visit play the games. They have such an economy of movements, breaking down the required motions to the most basic level possible. Even with DDR, for which I certainly gain a workout hopping about madly, they can simply reach the tip of their feet to the required spot with virtually no torso movement.
On the other hand, with I first had the NES, sitting within three feet to the right of my mom when she was trying to make Mario jump the mushrooms was a recipe for a black eye...her whole body when into the midget's jumps, even with the tiny rectangle controller.
Despite the advanced technology in play, I think games that successfully bring out emotion follow the same popularity pattern that makes Pixar characters so compelling...there's a good story behind them. Whether it is Mario Kart or Up, I get emotionally involved because they are fun to play/watch (tho I have no research to prove this...).
Actually we use the verb, "to play," in theatre and films.
or more obviously, people "play" music all the time... DUH!
"No other form of culture, after all, depends on the verb 'to play'. "
What an astonishing thing to say, especially in the context of "culture"!
Mario's punching people, eh?
You've never played a Wii game.
I don't know about this argument because most of my generation, generation Y, much prefers the PS3 and XBOX to the Wii, which would suggest that we care more about visual stimulation than kinesthetic stimulation.
This was really interesting. It got me to read both William James and some more about the neurology. I love using games to teach math and wrote about my connections with this post on my blog.
This is an interesting theory but the arguments fail on so many practical levels, and the use of Super Mario Galaxy 2 in itself pretty much invalidates your argument. I wound up discussing this at length on my Wii blog: http://nintendo.about.com/b/2010/07/05/science-writer-claims-wii-turns-…
I think we are simply pleased with the degree of control the Wii gives us. Humans seem to like control. In scary video games they try to arouse fear by removing control from the player (strange camera angles, "realistic" controls). Maybe more control makes us more likely to be happy, and less control moves us in the opposite direction. I think that the relationship is more between control and emotion rather than movement and emotion.
Further testing would be needed, as there are many varying stimulus in these games (music, color, animation speed, context).
The author of this article has obviously never played (or even seen someone playing) the Wii. You stand there, and waggle your controller - and hope something happens.
I basically share the comments of Steve and Stuart Coleman. Playing in your dark room with the only light of the screen and listening to the voices of your team mates looking for a monster of whatever to kill is quite emotional.
We might consider the Wii as a revolution in its conception, but the truth is that there is not revolution when you try it. The motion that you need to play Wii is almost none. We will have to wait to see the Natal project live to see what happens but you are completely unfocused this time Jonah.
okey oyna okeyburda okeyoyna
Hi, ever listen of Dance Dance Revolution? It wasn't Wii that began movement games.
Well, I tried to link to a demo of the Playstation's version of this tech, called "Move". Apparently it's hung up in moderation. Search youtube for "sorcery demo e3" if you want to see it.
My point was that this game combines the controllers of the Wii - a physical object you hold - with the motion-recognition of the Xbox's Kinect. I kinda thought it illustrated the notion of immersion and physical engagement that Jonah was talking about. Oh, well.
saÄol adminim kralsın sen valla süper