You all know the score. A train leaves one city travelling at 35 miles per hour and another races toward it at 25 miles an hour from a city 60 miles away. How long do they take to meet in the middle? Leaving aside the actual answer of 4 hours (factoring in signalling problems, leaves on the line and a pile-up outside Clapham Junction), these sorts of real-world scenarios are often used as teaching tools to make dreary maths "come alive" in the classroom.
Except they don't really work. A new study shows that far from easily grasping mathematical concepts, students who are fed a diet of real-world problems fail to apply their knowledge to new situations. Instead, and against all expectations, they were much more likely to transfer their skills if they were taught with abstract rules and symbols.
The use of concrete, real-world examples is a deeply ingrained part of the maths classroom. Its advantages have never really been tested properly, for they appear to be straightforward. Maths is difficult because it is a largely abstract field and is both difficult to learn and to apply in new situations. The solution seems obvious: present students with many familiar examples that illustrate the concepts in question and they can make connections between their existing knowledge and the more difficult concepts they are trying to pick up.
The train problem is a classic example. Another is the teaching of probability with rolls of a die, or by asking people to pick red marbles from a bag containing both blue and red ones. The idea is that, armed with these examples, students will recognise similar problems and apply what they have learned. It's a technique deeply rooted in common sense, which is probably as good an indicator as any that it might be totally wrong.
Jugs, pizza, balls and symbols
Jennifer Kaminski from Ohio State University demonstrated this by recruiting 80 undergraduate students and teaching them about a simple mathematical concept that involved adding three separate elements together. The concept included very basic mathematical ideas, like the concept of zero, or the idea of commutativity - that the order in which things are added doesn't change the result (1+2 and 2+1 both equal 3).
Three groups were taught using familiar, concrete examples. The first group were told to imagine measuring cups containing varying levels of liquid and asked to work out how much liquid remained when the two were combined. So for example, combining a jug that was a third full with another that was two-thirds full would give a full jug. Uniting two jugs both two-thirds full would give one jug that was a third full as a remainder.
The second group was taught using another similar example involving pizza slices alongside the jugs, and the third group learned both of these, along with a third system involving tennis balls. They were told that each new system worked in the same ways as the old ones, and obeyed similar rules.
The fourth group was taught in a more generic way. The vivid jugs, pizzas and balls were replaced with generic using a meaningless and arbitrary symbols, and the students simply had to learn how they could be combined. For example, a circle and a diamond combined to form a wavy rectangle, while the sum of two circles was a diamond.
After the training, feedback and 24-question multiple-choice tests, Kaminski was satisfied that the vast majority of students had picked up the principles successfully. She then asked to students to apply their knowledge to a fresh setting, described as a foreign children's game involving three objects. Children pointed to two of the objects and one child, who was "it", had to point to the correct final one to win.
The students were told that the game's rules were very much like those of the systems they had just learned. They were shown some examples so that they could deduce these rules and were tested with 24 multiple-choice questions. These were, in fact, the same as the questions they had previously answered, but "translated" to the new setting.
Generic beats concrete
Contrary to all expectations, the group who were taught with generic symbols fared best, answering 76% of the questions correctly. They completely outperformed the three groups who were taught with real-world examples, who all scored between 44-51%, no better than a chance result. The group that was taught using three concrete examples fared just as poorly as than the one which only learned one system (see Experiment 1 below).
So the common sense idea that students are capable of picking out the similar threads from related examples seems to fall short. In fact, quite the opposite was true - they were better able to apply their knowledge to a new setting if they were taught using generic abstract examples.
The result seems so at odds with the traditional view of education that Kaminski tested it further. She recruited another 20 volunteers and taught them the jug and pizza systems, but this time, she explicitly spelled out the similarities between the two. Astonishingly, this didn't help matters and the students' scores still reflected random guesswork more than learned problem-solving.
In a third experiment, the students themselves were asked to work out the similarities between the two systems themselves. This time, nine of the 20 students scored very high marks of 95%, but the others still performed no better than chance. So this type of teaching method benefits some high-performing students achieve a top grade, but it also fails to help others. And on average, the 'class' still scored less than the group who learned the generic symbols.
In a final experiment, Kaminski wanted to see if a combination of real-world and generic examples would have a stronger effect than either alone. Clearly, they both have their advantages - generic examples seem easier to apply, while real-world ones are easier to pick up at the start. Even so, students who only learned the abstract symbol system still outperformed a second group who learned the jugs method, followed by the symbols (see Experiment 4 in the graph above).
Kaminski's work will no doubt come as a shock to those in the education sector. While it's certainly true that students engage with mathematical concepts more easily when faced with real-world examples, these striking experiments suggest that they aren't actually picking up any real insights about the underlying principles involved. And without those insights, they are unable to apply their knowledge from one real-world example to another, exactly the opposite of what maths teachers want to achieve!
Kaminski isn't calling for an end to all real-world examples in classrooms, but she suggests that they should only be used when the basic abstract principles have been introduced. Deeply grounding an abstract concept in a real-world example could actually do more harm than good, by constraining the knowledge that students gain and hindering their ability to recognise the same concept elsewhere. Questions about bags of marbles and speeds of trains really are just about bags of marbles and speeds of trains.
Update: Chad has a considered analysis of these results over at Uncertain Principles. Have a look.
Reference: Kaminski, J.A., Sloutsky, V.M., Heckler, A.F. (2008). LEARNING THEORY: The Advantage of Abstract Examples in Learning Math. Science, 320(5875), 454-455. DOI: 10.1126/science.1154659
Images: Train photo by Nachoman-au; drawing and graph from Science.
- Log in to post comments
Maybe the jugs example just sucks and is confusing?
One thing that always got to me about 'real world' examples in math is that they are rarely realistic and sometimes downright silly.
I think the difference between abstract/concrete would be better tested by an explaination of deriviatives that involved only numbers and a discussion of rates, and an explaination that involved examples based on the velocity of moving vehicles... and then see how each group performs when solving each type of question (maybe take some questions from a math textbook and others from a physics textbook).
It'll take me a while to digest all of this, but I don't think it's too shocking. I would suggest that one reason that "real world" examples don't work well may be because students have been trained by math teachers to ignore their real-world knowledge. I remember an elementary school or junior high problem involving the number of painters painting the surface area of a room. My argument was that there's a certain point at which more painters would make it take LONGER. Of course, I had to ignore this kind of logic to do the problem, and that was true throughout my math education. So "real world" examples may be treated by students as confusing symbolic concepts that look like real things they know about but act like abstract notions that are defined by the teacher.
The study also reflects what most thoughtful teachers already know--that students can often handle basic math concepts outside of class, but don't generalize well to in-class exercises. I think the whole notion of generalization needs to be given a good hard look--particularly those claims we constantly had inflicted on us as kids, that learning math would magically make us more logical, critical, and scientifically-minded.
Anyway, I think a lot more research needs to be done, but I'm glad people are looking at this kind of thing.
Interesting! This ties into several ideas, all of which tell us something about why Everything Is Difficult.
The first is how much we under-estimate how difficult new concepts are. When we learn Newton's Laws of Motion in secondary school, it's fairly straightforward, apart from the counter-intuitive idea that things continue to move unless something stops them, which is resolved by understanding that friction is that "something". Easy enough. Why then did it take a genius of that stature of Isaac Newton to work it out? Similarly, most of us cope with basic calculus, yet it took giants like Liebniz and Newton to pull it together.
So there's no way average kids are going to discover genuinely difficult concepts de novo without being spoon-fed quite carefully. Put another way, us average oiks can understand many things that we could never create.
Second idea, wasn't it Piaget who said something like: "you can only learn what you almost know already"? Eureka moments only happen when a logjam of almost-understanding is released. Most of us simply don't have them.
Third idea: Temple Grandin writes that (non-human) animals do not generalise from specific experiences anything like as much as they appear to. We impose our interpretation of what we would have learnt onto the animal's behaviour, but it's probably using a much simpler mental construct than we realise.
Fourth idea: we all learn in different ways. I find it difficult to learn words in a foreign language without seeing them written down. I usually skip the maths when reading a science book except to look and see the "shape" of an equation: simple, difficult, exponential, derivative, etc. But most of us probably learn best if we learn a topic in more than one mode, so one mode reinforces the other. Some abstract concepts I grasp happily, others feel "bedded down" only after seeing or doing a couple of exercises.
In that vein, I was seriously impressed when a lass teaching me a man-overboard sailing maneouvre first told us what we would do, then drew it out in diagram form, then she and a colleague walked through it on the slipway, then we went and did it. Excellent.
Too many teaching programmes (e.g. the old Nuffield approach to science) seem to come at learning by saying "this is the best way for everybody".
Perhaps the problem is that once one has learned something, it's difficult to appreciate what it looks like to someone who hasn't learned it.
One HUGE problem with the "real world" problems is that they're like NOTHING that an eighth-grader would want to do in the real world. I mean if I wanted to know Frankie and Johnnie's ages, I'd ask them, not work out some wierd algebra problem. If I wanted to know when trains meet somewhere, I'd look at a timetable. If my bathtub was filling up, I wouldn't calculate how long until it overflows, I'd turn off the water. It took years for me to overcome the damage done by ridiculous word problems.
I wonder if something similar isn't happening with these undergrads. It almost sounds like the concepts and methods are too childlike for them to be motivated to succeed.
I also have to ask, have the experimenters considered that undergrads are usually at an age when abstract reasoning is foremost in their learning. I know this kind of contradicts my rant against boring irrelevant "word problems", but maybe the undergrads using abstraction did better because in their course work and at their stage of life they use abstraction more and so are more adept with it.
The second paragraph of your post sure sounds counterintuitive. And yet, when I read the description you gave of the experiement, the outcome of this particular experiement didn't feel counterintuitive at all.
Part of what's weird about this experiment: From the description above, it looks like the students who learned from familiar examples were learning Addition Of Rational Numbers. Even if they were told to ignore everything else they learned about addition except for how to work the types of examples offered (and I don't know if they were told that or not), it would very hard for them not to see this as a review of the vast theory of arithmetic as they know it. Whereas students who learned the abstract theory failed to make the connection that this "new" theory was just a tiny subtheory of the addition of numbers they already knew. I propose some alternate interpretations of this experiment: Students drawing on a vast, well-known theory (i.e. arithmetic of numbers) may not apply the knowledge to particular unfamiliar situation as well as students who learn only a particular portion of the theory which is particularly well-suited to the particular unfamiliar situation at hand.
Maybe a better, shorter interpretation: Students have trouble abstracting principles which may be gleaned from very familiar problems and using these principles in vastly unfamiliar problems. Hmm. That doesn't seem very counterintuitive.
interesting stuff,
i would also like to see whether practice at moving from the abstract to the concrete is helpful. Say, two groups get taught with the abstract method, and then one of those groups gets practice applying it to concrete example, the other group just gets extra practice with the abstraction. Then both groups are tested on a new (concrete) example.
Would also like to know whether practice at abstracting (like say developing your own formalism) from concrete examples is more useful than being handed the abstract notation in the first place.
The subjects were college students -- the average college grad has an IQ of about 115, or 1 standard deviation above average. They are thus more likely to grok abstract concepts and want to "get the bigger picture" when fed only a mishmash of real-world examples.
However, when you're teaching students who are average or below-average in intelligence, you just see how far abstraction goes! I was a math tutor for 2 1/2 years, but it's pretty obvious.
We've known about similar results for quite some time, although Jennifer Kaminski's interpretation is a little bit more sensational than some other researchers'. This research is not about how real-world examples fail to help students transfer their knowledge. This is about what happens when you don't carefully think through the analogies you give to students. The measuring cup example asks students to figure out how much liquid will be left over after combining two measuring cups. I'll believe that combining two 2/3-full cups would give you a 1/3 cup left over. That's pretty easy to see. Combining a 1/3-full cup and a 2/3-full cup, however, does not leave a full cup left over in the same way, but that's the way the example works. The pizza and tennis ball problems are equally inconsistent.
We know good analogies help learning and bad analogies hurt learning. In 1982, for example, Frank Halasz and Thomas Moran found that giving bad analogies to students did not help them learn about computers. In fact, the analogies actually hurt them. See Halasz, F. and Moran, T. P. 1982. Analogy considered harmful. In Proceedings of the 1982 Conference on Human Factors in Computing Systems (Gaithersburg, Maryland, United States, March 15 - 17, 1982). ACM, New York, NY, 383-386. DOI= http://doi.acm.org/10.1145/800049.801816
Dedre Gentner was even able to take a couple of analogies, point out where they failed to make sense, and show that students who had been taught with a specific analogy made the most mistakes where that particular analogy fails. See Gentner, D., & Gentner, D. R. (1983). Flowing waters or teeming crowds: Mental models of electricity. In D. Gentner & A. L. Stevens (Eds.), Mental models (pp. 99-129). Hillsdale, NJ: Lawrence Erlbaum Associates. (Reprinted in M. J. Brosnan (Ed.), Cognitive functions: Classic readings in representation and reasoning. Eltham, London: Greenwich University Press).
Let's not confuse the well-known problems of poorly-chosen examples with real real-world examples which reward rather than punish students for their everyday knowledge.
I teach high school math. I commonly see this problem. However, I think the underlying problem is not the 'realworld' examples themselves, but that instructors do a poor job of teaching students how to make connections between situations. I don't think this particular research addressed the connections part of it.
Interesting. I never really thought about it at the time but the 'real world' examples in my maths text book were the hard ones. You had to read the question and then pull the right numbers out and turn it into an abstract formula, then get the answer and feed it back into the example. You can't do maths with trains or marbles - you do maths with numbers and symbols.
I guess people think in different ways though. I remember that I understood doing maths with negative numbers almost immediately and then spent most of the lesson trying to explain to classmates how to add and subtract negative numbers. A few years later my sister had great difficulty grasping maths with negative numbers because she couldn't correlate them in her head with physical objects to think through the arithmetic. She was wearing pyjamas at the time which had sheep on them and I offered to explain negative numbers using sheep. She didn't think I could do it but I did.
She was fine with adding negative numbers to positive numbers, and taking away positive numbers from negative numbers. But within fifteen minutes I showed her that if I took away her negative sheep she'd have more sheep than she had before and she never had problems with negative numbers again.
I would also love to see a couple of followups to this research.
I was thinking that perhaps because the abstract concepts are so difficult, it creates more cognitive demand. I mean, I really had to think hard looking at those symbols, but the pitcher was immediately obvious. Perhaps that "thinking hard" part helped encode it better.
I was also thinking that by not providing a concrete example, the college students were able to create their own. For me, when I learn something very abstract I need to create my own metaphor. "Oh, that's like when....." Perhaps creating your own concrete example is really the way to go. This would also help explain why both abstract and concrete were worse. Students weren't forced to create their own connections.
Great comments.. I wonder, when those ideas will get in learning systems..
Btw, there is an interesting article about teaching (math) in schools: http://www.maa.org/devlin/LockhartsLament.pdf
One problem with teaching is that we are forced to learn (often even to only memorize), and not motivated to analyse and explore - that's, what knowledge is about, isnt it?
I�m afraid I might be revealing myself as someone a tad innumerate, but were you serious about the trains meeting in 4 hours? I have to admit getting caught up with trying to figure out what was going on in this word problem. My basic thought process: Okay, if they are both traveling a different speeds, how could they meet in the middle? Maybe the problem really wants to know when they crash? What assumptions are involved here? They immediately start the journey at full speed? No stops, slowdowns, or trying to avoid collision?
Now I remember why I always hated so much of what passes for math in the classroom�to often it is treated as objective TRUTH, ignoring that although certainly not subjective, there is a strong normative component to interpreting the �facts� of the case, trying to figure out what real world factors can/should be ignored, and then trying to determine what answer is really being sought. Cause you know, two snakes plus two mice do not equal four animals, it equals two fed snakes�or one really fat snake and one still looking for a meal�or ???
The idea of emphasizing abstract math over �real world� problems is compelling. And we could do better with using various modes of teaching and emphasizing �justify your solution� in answers. That, or really jazz up the word problems. Make the trains problems something out of 24, or figure out the volume of cocaine �ball-bearings� that you can smuggle in shipping crates, the profit you can gain if y% of your shipment is sold at street value, or the range of jail time you will serve if x% of your shipment is discovered DEA agents. These types of problems, although not PC, get the kids interested�and provide a good teaching opportunity to show how a range of solutions are more or less appropriate in solving problems. Or else just clarify the problem by pointing out that the DEA agent is not on your payroll.
Train A was dispatched by CTU to stop Terrorist Train B. They impacted one hour later. No DEA agents were injured.
Please note that the "train" comment belongs only to Kenneth Chang article in the New York Times and is not mentioned in the Science magazine under review.
The "train" problem in its various incarnations is mostly a putdown of mathematics by mathematically challenged outsiders.
In my 40 years carrier as a mathematics teacher I have never seen this problem presented as a prelude to abstract concepts.
A reasonably intelligent person with no particular fondness for algebra would handle it (the problem as stated by Chang http://www.nytimes.com/2008/04/25/science/25math.html ) this way:
1. At 7PM the trains are still 360 miles apart.
2. From 7PM the distance between them decreases by 90 miles an hour.
3. The trains will pass each other 4 hours later at 11PM.
I guess that most professional mathematicians would handle it in the same way.
Abstraction (in this case the use of equations) is there to expand the power of our brains beyond routine and not to put a veneer of sophistication on what is routine.
Hubert Halkin, Mathematics UCSD
Hi,
I left a comment on Thursday or early Friday. It hasn't shown up, and I was wondering if it had gone into your spam filter because it had a couple of links in it.
Thanks.
Clint
Clint - found it. Sorry.
And to the rest of you, the comments I'm getting since moving to ScienceBlogs are really superb. There's some fantastically thoughtful stuff in there and I'm really grateful for it.
Hubert, I didn't read the Chang piece; the fact that I used the train example was coincidental and probably reflects the fact that it's a familiar "real-world" problem. And Michael, you crack me up, but I fear that a syllabus based on your idea would simply lead to kids learning to ask themselves "What would Jack Bauer do?"
I never got the train questions, not in Primary School Maths, not in Sixth Form Physics, so I'm glad somebody's brought this up finally.
To be frank, learning about the laws opf physics first may have helped, I think most children can understand the basic aspects of the universe at that age. I even think some basic Particle Physics in Primary School would be a great way to teach kids, afetr all, it's just another model, like Playing with lego, leave the maths out of it and just tell them what things are actually made of.
I suppose it's all Lying To Children...
I think your train example is incorrect. How do you conclude the trains will meet in 'the middle' after 4 hours?
Simple reasoning shows this can't be true. Speed(S)=Distance(d)/Time(t). If you really mean 'the middle', you are referring to a distance of 30 miles, in which case, the train travelling at 35 m/h will be past the middle in less than 1 hour. Let's assume you mean the distance at which the trains pass each other. In time t, both trains will have moved a total of 25t+35t or 60 miles. In other words, 60t=60, which implies the trains meet in exactly 1 hour.
As for the rest of your post, I believe that students need both abstract and concrete learning.