Cognitive Daily

This is a guest post by Laura Younger, one of Greta’s top student writers from Spring 2007

Everyone has heard of the concept of reinforcement. You reinforce your child with dessert after finishing his or her vegetables; you praise your dog with ear scratches for not barking at the mailman; or you give yourself a little TV time for cleaning the bathroom. It’s a system that often works, but what types of behaviors can be reinforced? We know that learning can be improved with reinforcement, but is external reinforcement required for learning to occur? A team of researchers led by Aaron Seitz were interested in this very question as it relates to perceptual learning. Perceptual learning in this case can be defined by improvements in our sensory abilities. Suppose I asked you to determine in what direction a cluster of faintly shaded dots are moving, as in this video (QuickTime required). Can you see them move?

Would you get better at this exercise without any reward or feedback? It might help to think of a task like this in the context of the real world. Think of it as having to drive in the rain and perceive road signs, when everything around you is dark or as creeping through a green fantasy world in a video game and having to shoot an evil plant. Does improvement in these types of activities develop automatically or is there some external force that aids our improvement?

Some research on this area has shown that perceptual learning can occur after participants simply perform a task, but others have shown that external reinforcement is needed for perceptual learning. The role of external reinforcement also seems somewhat dependent on the type of task that is used.

Aaron Sietz and colleagues used two optical motion tasks to see if participants could in fact learn without external reinforcement. In their first experiment, two groups of participants were asked to figure out the direction of motion of 200 dots (the movie above is one example); the dots were very similar in contrast (varied above and below the average threshold for detection) and moving in many different and random directions; one subset of dots moved in a coherent direction. The participants then had to choose from one of eight directional arrows to indicate in what direction the subset of dots were moving. Those subjects who received external reinforcement got it in the form of feedback to their responses: a green + with a high pitched tone indicted a correct response and a red x with a low pitched tone indicated an incorrect response. The external reinforcement was only given on the first of two training days.

After one day of trials, the participants were tested on the second day. Those who had received external reinforcement had a significant improvement in their performance on the task. You might think that the group with no external reinforcement would improve just a bit. But no, the group with no external reinforcement showed no significant changes in their performance from the first day to the second day. The researchers commented that learning based on mere exposure to a task might take more than a single day of training. So, they did a second experiment.

Experiment two was basically the same except participants received training for ten straight days before they were tested on days eleven and twelve. They also performed a different task; this time the task was to report the direction of a white bar on a dark-grey background masked in either white or dark-grey pixels. But, even after ten days, the participants who did not get external reinforcement still showed no improvement in their performance on the task, while the external reinforcement group did. Take a look at the no reinforcement group in the graph below; their performance on day twelve was actually worse than on day one! The external reinforcement group improved from about 75% correct on day one to a high of 85% correct on day eleven. The gap between the two groups is quite wide considering that the only difference between the two was feedback on right or wrong answers.

So, from a general standpoint, it seems that internal reinforcement is not enough for our brains to learn how to perform motion and orientation related tasks. But how do we know that these tasks even offered internal reinforcement? The discrimination tasks were quite tricky in that many of the dots and bars presented were below the threshold of detection of the participants. When asked about their performance, participants reported low confidence; perhaps they were so frustrated with the difficulty of the task that they lost interest and simply were not reinforced by the task itself. Stimuli consisting of signals of dots and bars that are hard to detect, like the ones in these experiments, are characterized has having low-signal strength. In similar contrast detection tasks with low-signal strength, learning effects have been difficult to find even when using external reinforcement. It seems that detecting low signals is quite difficult and once again reminds me of having to distinguish between road signs and stop lights in the fog, rain, or even snow.

Without a deeper exploration of the tasks used in these types of experiments and a comparison of how the results vary based on the characteristics of each task, no great conclusions can be made about whether or not external reinforcement is absolutely necessary for perceptual learning to occur. Regardless, it seems that the role of reinforcement in certain perceptual tasks, such as this one, can be very important. So, maybe the next time you are driving through the rain on a dark night or tying to shoot an alien on a video game, a little feedback from a friend might just improve your performance.

Still wondering which way the dots were moving in that first movie? Why not take a look at this version, where the contrast has been increased significantly — now do you see it?

Seitz, A. R., Nanez, J. E., Sr., Holloway, S., Tsushima, Y., & Watanabe, T. (2006). Two cases requiring external reinforcement in perceptual learning. Journal of Vision, 6, 966-973.

Comments

#1 Aaron Couch
June 21, 2007

At first, I thought the video clip on top wasn’t playing, but it is just very dark as intended. It will likely show up differently depending on how strong your display’s contrast and black levels are.
#2 roseindigo
June 21, 2007

There are people who are inwardly directed and people who are outwardly directed. Those who need reinforcement from outside are outwardly directed, and reward and punishment works for them. That’s about 80% of the population. Then there are people who are inwardly directed, who challenge themselves and need no pat on the back from anything outside themselves. I think most good managers in the work environment know these two types of people.

As I was reading the study above, this sentence stood out to me: “perhaps they were so frustrated with the difficulty of the task that they lost interest and simply were not reinforced by the task itself.”

Perhaps they saw that the difficulty of the task did not justify the sort of effort they needed to put into it, since there was no real reason for the task in the first place. When most people see a good reason for effort, whether for an outer reward or a self-directed reward, they do not lose interest. Give the same people a REAL task that is useful, and I bet they will perform much better.
#3 mbs
June 21, 2007

It’s hard for me to draw a conclusion from this experiment. It seems to prove that reinforcement aids task performance when vision is stretched to its contrast-resolving limit… but I can’t understand how any conclusion follows about learning/reinforcement in general.

Reading back over the blog, I guess I have just expressed agreement with the author (Aaron).
#4 Ann Nunnally
June 21, 2007

I don’t think that learning is a direct result of reinforcement; it is a direct result of effort and repetition. Reinforcement is the motivation for a person to provide the effort and repetition in the face of difficulty or boredom. I am a speech language pathologist, so I may be using these terms in a different way than the author; however, I have noticed this to be true with my clients. For example, a child may learn to pronounce a difficult family name sooner than an easier word because he or she has a strong motivation to do so.
#5 Mark Frank
June 21, 2007

Feedback may be a reward but it also just helps you know “what good looks like”. Maybe the guys who had no feedback were just that little bit unsure in marginal cases – while the guys with feedback had some help recognising marginal cases.
#6 Thomas Leahey
June 21, 2007

I’m a 60 year old psychologist who does both history (A history of psychology, Prentice-Hall) and learning/cognition (Learning and cognition, P-H, with Richard Harris). The “is learning necessary” issue is really old, going back to the 19th century. I hope we don’t have to rehash the whole thing again (see our L & C book).
#7 c0t0s0d0
June 22, 2007

“hope we don’t have to rehash the whole thing again” – well, we do need to do it all again… yes, it better to learn from the past and move on… but often its very important to question the learnings of the past – otherwise, we might be progressing on false knowledge.
#8 Gary
June 23, 2007

I wonder if it is more likely that psychologists, lacking any kind of coherent, integrative framework, and being spread across diverse areas that represent different viewpoints, simply have not agreed on such basic issues even after the early research. Is there any consensus on what constitutes an answer or progress in this area?
#9 Tony Jeremiah
June 24, 2007

From my understanding, the results of studies associated with this very question laid the foundation for the transition from behavioral to cognitive psychology. Interestingly, the conclusion from this study is opposite to the conclusion drawn from earlier studies.

Tolman and Honzik (1930), studying rats, derived the principle of latent learning, showing learning in the absence of reinforcement. In this instance, learning was demonstrated by evidence indicating that rats created mental maps of mazes without requiring (immediate) reinforcement.

Bandura’s (1965) Bobo doll experiment extended Tolman and Honzik’s research via the construct of observational learning. That research showed that children can learn aggressive behaviors by watching others, and such behaviors did not require immediate reinformcent for this type of learning to take place. However, he noted that four factors were involved in observational learning: attention, memory, motor skills, and motivation. Perhaps the absence of one or more of these factors explains the failure to show learning in the absence of reinforcement. Or, perhaps perceptual learning is fundamentally different from observational learning.
#10 Nicolas Wilkins
July 4, 2007

It seems that the simple answer here is that humans/animals need external reinforcement to learn. The question is how does one operationaly define reinforcement. In otherwords, if one can not tell which way a dot is moving on a screen because the stimulus of movement is below their sensory threshold, than no information can be encoded in memory (sensory, working-memory, or long-term memory). Because there is no memory trace to be recalled and compared to the current state of the stimuli, there is nothing about the task that can be internally or externally reinforced. The control group, the group that was externally reinforced by verbal conformation, was not reinforced directly. That is if the stimulus of movement was the same for both groups, than they both did not encode movement information in memory. It is much more likely that the external reinforcement reinforced some sort of field mapping behavior. For an example thought: I (participant) answered correctly but was not able to see the movement of the partical. I did this by following this procedure: look at the particle in the field, then close my eyes for 1 second, then look again and compare the two dots field position, and finally assume the direction of the movement based on the difference between the two mapped fields. So why doesn’t the no-reinforcement group do this? Wouldn’t they think of doing this if they were internally motivated? These are questions that should be studied. If the participants were given the choice to study material (ie a book on field dependence) that may inform them on how to answer the test question, then would it not prove an internal reinforcement if they chose to study that material and then improved on the test?
#11 mantissa
May 17, 2008

I need a simple analysis of the importance of reinforcement in learning