Perception in Legoland

Rummaging through your bag in search of keys, it's clear that you can recognize objects using just your fingers. But is it easier to recognize the keys if you feel them as if you were going to open to door, or if you encounter the key's teeth in some odd orientation--like pointed straight up? Consider the following pictures. Which object looks more familiar?

Indeed, it is easier to recognize objects visually when you see them in familiar orientations. Could this orientation effect extend to objects we touch? Fiona Newell and her colleagues used little LEGO towers to ask this question, and (with the help of my local LEGO expert, Nora) we've recreated a couple of their stimuli. Notice that these objects are only slightly different from one another, making this a pretty tough task.

Newell et al mounted their towers on stands, so they could not be moved, and presented four towers for participants to learn. The participants were instructed to either visually inspect the towers, or used their hands to explore the objects presented behind a curtain. When participants thought they could identify each object, the researchers presented eight towers that included the original set plus four new ones. Half of the time the towers were in the same orientation that participants had learned, but half of the time towers had been rotated about the horizontal axis (180°, like doing a headstand). In addition to the orientation change, half of the time participants identified the towers using the same modality they had used at study--if you had visually inspected, you would still be visually identifying the objects. For the other half of the trials, modality was switched--if you had visually studied the towers, you would now be feeling the towers behind the curtain. With this design, Newel et al. get to answer two questions: does orientation matter for touch (if yes, the 180° rotation should be harder), and how readily can we transfer visual and touch information (changing modalities from study to test)?

You can see that when the towers have done a headstand (a 180° rotation about the x-axis), it's significantly harder to recognize them (p < .05). Orientation matters for touch, as well as vision. Remember that there are 8 towers to choose from, so participants are well above chance.

But something cool is going on when you switch modalities; now participants are better when objects are presented in the 180° rotation. In other words, if you studied an object with your fingers, you have an easier time identifying it by sight if it's upside down! What is going on?

Newell et al suggest that we think about what part of these towers would be easiest to feel. As you reach behind the curtain to learn the towers, the most natural grasp would involve you thumb on the front of the tower with fingers behind, perhaps leading to a much more detailed representation of the back of the object. With visual inspection, however, you'd know a lot more about the front of the object. So, if exploration by touch emphasizes the back of the object, and you are then trying visually identify it, a view of the back, thanks to a 180° horizontal rotation/headstand, would be most helpful.

This suggests that any rotation that switched the front and back views of the towers would help cross-modally (switching from vision to touch, or vice versa). And, a rotation that did not switch back and front would not help. Specifically, a rotation about the z-axis, or a cartwheel, should not lead to improved identification at 180°, because the front is the same at study and test (and so is the back). Here are the results for this transformation:

Again, percent correct are presented for each study-test pattern and orientation, but now it doesn't help to have the 180° rotation when you switch modalities. In fact, here it's easier to identify the object with either modality when it hasn't been rotated at all.

Looking for your keys, it's going to be easier to identify them from a familiar angle both by sight and by touch. However, visually you know a lot about the front of an object, while you know more about it's back if you've been grasping it. So if you need to identify which of two apparently identical paper coffee cups is yours, it would probably help to look at its back to see just where that seam you've been fiddling with is located.

Newell, F. N., Ernst, M. O., Tjan, B. S. & Bülthoff, H. H. (2001). Viewpoint dependence in visual and haptic object recognition. Psychological Science, 12, 37-42.