This story bugs me. It's a heartwarming tale of an inspiring teacher in an inner city school, who gets young kids motivated to learn science. Or does he?
These are elementary school kids, so they're young and maybe the most important thing is that some enthusiasm for the subject is instilled…but I also see a lot of simplistic thinking, a reliance on rote memorization of trivialities, and stuff that is just plain wrong. I have to disagree with the article: the kids are learning discipline, but they sure aren't learning science.
"Name one main kind of organism on Earth," White is saying to two students seated at a table before microphones and buzzers. A girl from 112th Street slaps her buzzer first.
"One main kind of organism on Earth is a plant," she says in a burst of syllables.
"Correct. One point," says White. "Name the other main kind of organism on Earth." This time, the competitor from the Watts Learning Center starts to answer first.
"The other — "
White cuts her off. "You don't have the light." The girl hadn't hit her buzzer first. The student from 112th Street answers.
"The other main kind of organism on Earth is an animal."
"Correct, one point."
Uh, what? What is this kid going to say when a lichen or bacterium is waved in front of her face?
There's Philip Aubrey, one of the co-captains of the fifth-grade science team who shocked White by learning the seven systems of the human body — respiratory, circulatory, muscular, etc. — overnight. "Usually, that's a two-week process," White says.
Hmmm…respiratory, circulatory, muscular, skeletal, nervous, endocrine, urinary, digestive, immune…is reproductive a separate "system"? Are lymphatics lumped in with circulatory? What kind of nonsense is this, memorizing arbitrary blocks of organs? I don't get how you can say some set of basic memorization exercises is a "two-week process," either.
I detect an obnoxious level of rigidity here, and a lack of integrative thinking about physiology. I know, little kids have to have the subject split up into small and digestible pieces…but it's the way it's being partitioned that bothers me.
White is saying: "For 20 points, the definition of science?"
The girl from the Watts Learning Center pounces. "The definition of science is a body of knowledge and an understanding of the physical and natural world," she says, without a nanosecond of hesitation.
"Correct," White says. "Twenty points: The definition of the scientific method?"
She nails it.
Suddenly, things aren't looking good for 112th Street.
"Last 20 points, what are the six steps of the scientific method?"
Once again, the girl in plaid gets to her buzzer first. "The six steps of the scientific method is purpose, research, hypothesis, experiment and — "
White cuts her off: "Incorrect."
Jazmani hits the buzzer.
The words fly out of her mouth: "The six steps of the scientific method are purpose, research, hypothesis, experiment, analysis and conclusion."
"Correct, 20 points," White says.
For lack of the proper verb — the girl in plaid said "is" instead of "are" — Watts Learning Center has lost the points.
AAAAAAAAaaaaaaaarrrrrgh! THE scientific method? What about the important steps of blind memorization and regurgitation?
Seriously, I think the most important aspect of science to inspire in young kids is creativity and imagination, and this fellow's approach seems calculated to drive out exactly those desirable attributes.
I'm not on the ground at 112th Street Elementary School, so maybe this is how they have to start to get these kids involved in their education—it certainly sounds like their teacher, Stan White, is getting them fired up—but if they were to show up in my classroom 6-12 years later, I'd have to tell them that Mr White was all wrong, and that they need to forget everything he ever told them. And then they'd hate me and they'd hate science.
(via Science and Politics)
While I understand and agree with your enthusiasm for creativity and imagination - I would propose that real learning in children is the result of strong emotional forces. They somehow must first acquire an identity model from their environment. Once they've done that you can not prevent them from learning whatever is needed to fulfill that identity. They will willingly spend enormous energy and time for that purpose.
It could be a gang-banger or it could be a skate-board champion or it could be a scientist.
I think White is mostly providing an environment that is emotionally stimulating enough and acceptable enough to their peers - for them to choose that budding scientist identity model. If they've internalized that model there will be plenty of opportunity for them to overcome White's pedagogic approach.
Possibly, an even more important lesson they will pick up on is the often arbitrary and politically motivated authority figures and bar-no-holds competition they will have to deal with throughout their scientific careers. This could be especially useful for the girls.
They will also learn that everything they are taught by their professors is not necessarily true.
Just another view.
FWIW, this was my grade school/junior high science experience. No, not the "Jeopardy" format, but otherwise quite similar, up to and including (depending on the teacher) what I would consider disproportionate penalties for incorrect grammar, misspellings, etc. I always liked science, despite these issues, but perhaps because of them, my love for it didn't really blossom until high school.
Dear Mr. PZ Myers,
You need knowledge before you can have synthesis or evaluation. Some kids are a little slower than others. My guess is getting the majority of a classroom to pay attention is quite a feat. When is the last time you tried helping an elementary school curriculum other than criticizing it from afar? They need SOLUTIONS, not more CRAP from people who have never came to their grounds.
There's that desire for simplistic, instant-fix "solutions" for problems, again.
Here's a "solution" for you: DON'T TEACH KIDS FALSE FACTS ABOUT SCIENCE. Try implementing it.
In sixth grade, in a Central Valley, California, classroom, our homeroom teacher taught us the names for all the bones of the body (with the exception of a few of the smaller bones of the internal skull). We had to memorize the Latin terminology--parietal, temporal, fibula, tibia--with the exception of "ribs" (maybe she liked barbeque?), and be able to correctly write in the bones on a skeleton-and-skull mimeo sheet. There was certainly an aspect of "rote" to this, but we were proud of our accomplishment. The information has always stuck with me, and has proved of considerable "use" in all kinds of ways.
Most importantly, I think, the "scientific" names for almost anything were forever demystified. The barrier between me and the knowledge "hidden" behind the Latin and Greek was breached in a painless way.
"CRAP from people who have never came--"
BUZZ! incorrect tense! No more science for you today!
My guess is getting the majority of a classroom to pay attention is quite a feat.
My experience is with middle/high school math, not science (disclaimer: I tutor students one-on-one; I'm not in the classroom), but I can honestly say that the content of most math curricula is precisely why<\emph> it's so hard to get a class to pay attention. There's a lot that's interesting about math, but somehow most of this doesn't make it into the classroom.
My suspicion is that science suffers from a similar problem.
Forgive the intrusion once again, but when it comes to stories like this one, I feel compelled to say something.
Unlike a practicing scientist, I'm more interested in why people believe what they do, rather than in whether what they believe is true or not. (Truth is important; but it's not what I'm most interested in.)
When I've spoken with IDists and creationists, I've picked up on a certain narrative: a narrative of the humble but intrepid "little guy" who's standing up to the big, bad bullies of science. This narrative is deeply engrained in our culture, and it accounts for something -- perhaps much -- of the appeal of creationism.
Now: I mention this because I suspect that this narrative is especially attractive to people for whom science is not only intimidating but a kind of bullying. And this comes from bad science education. The elementary school kids in this story are obedient now -- but when their adolescent rebellion kicks in, I'm worried that one of the things they will rebel against is (what they perceive to be) science -- since it's now, in their pre-adolescent years, that science is associated with submission to authority.
In short: science education like this doesn't just work against proper scientific thinking, but it may drive them away from science altogether, and right into the waiting arms of the forces of anti-science and its David-vs.-Goliath narrative.
OK, I've said my piece.
I agree with what Dr. Spinoza so eloquently said, and lament that the memorize-the-vocabulary-words approach to "learning" "biology" is prevalent right through the freshman year of college. I saw for myself in my college-teaching days just how effectively years of this crap unfits students' minds for any genuine learning and thinking about science.
Being neither a teacher nor a scientist I wasn't quite sure how to articulate what made me wary of White's methods, but I am inclined to agree with you, Dr. Spinoza.
Ok, I have to take issue here.
I am a former Teach for America corps member ('97 corp -- Phoenix), and I have advanced degrees in math and environmental engineering (and have done research in both areas). So, I have had a foot in both camps (or, at least close enough).
The first thing I would say is that PZ is correct when he says:
I'm not on the ground at 112th Street Elementary School
I was, at least in Phoenix. One of the principles TfA hammered into us in training for teaching in their 5-week bootcamp is this: everything needs structure in your classroom. Lessons need structure; discipline needs structure; grading needs structure. In under-resourced schools, children often lack such structure either in the home or from the school in general. Without structure in your classroom, you cannot even begin to teach any material. Once you establish structure, you can start to branch out into exciting learning, but not before.
Prior to starting TfA, I had the same attitude towards science as PZ lays out here -- essentially, teaching science is not about the memorization, it's about the soul of science. However, I changed my tune quickly. I came to realize that, though I was teaching a science class, I was doing much more than teaching science. I was teaching study skills. I was teaching self-discipline. At heart, along with science, I was trying to get my students excited about science, but more to the point, I was trying to get them to like school and learn how to be sucessful at it. That seems to be the core of what this guy is doing. That is why I, too, played games like this. It gave my students a reason to prepare their groundwork of facts. It gave some of them a forum to take pride (and show off) the fruits of their achievement. It hopfully gave them a reason to continue school.
I think maybe one of my students has gone on to science. I think that that I got lucky getting one in that school in my two years there. The rest I pray finished high school -- at least I hope I beat the current 50% graduation rate that was the standard in our district. Because that was my goal: interest enough in some academic field to motivate my students to finish. And I was willing to do it by hook or crook, even if it meant taking the pursuit of science off of it's high pedestal and tarnish it by operating on it partially at the lower end of Bloom's taxonomy with my kids.
So, with all due respect (since I love to read your blog daily, and you convinced me to go to the geek prom -- i and my wife sat with you), this post makes you sound like you really have your nose in the air and a stick up your ass. You let me know when you think you have to stoop to make one of this guy's kids relearn science, and I will hoop and holler with pride that this guy got one of his kids to college.
Because really, when teaching in the inner city, as with most other things in life, the motto of the special olympics applies: It doesn't matter how far you go, it matters how far you've come.
ps -- there are other ways to teach the scientific method. I did have my students memorize the steps, but I also worked it with them. I took a chemistry flask with the wide bottom and narrow mouth (i forget what they are called), wrapped it in masking tape, vaselined up a superball, and shoved it inside. I then took a pen with a hooked paperclip taped to the end and stuck it in the flask and held it upside down. The pen stayed in the flask, and I took them through the method to figure out what was in the flask. I made them make a hypothesis (which was, at first guess, "there's something the paperclip is hooking into inside the bottle.") I asked them how we could test it ("straighten out the paperclip.") Then we tested it ("can't be that -- it still held the pen."), and then repeated with a new hypothesis ("there's a magnet in there."). We repeated the steps of the method over and over again, eliminating possibilities until they were stumped and I promised to tell them at the end of the year. I did this at the beginning of the year, and they asked about it every week. Oh yeah, and then we played jeopardy with the steps of the scientific method.
This reminds me of a bit in one of Feynman's books (can't remember which) where he discusses physics students he met in South America who could reel off definitions and laws verbatim on command, but drew a blank when given real-world problems - even when the answer depended on one of the laws they'd just recited.
Rough. I think that Mr. White will teach the kids science in the same way that we teach medical students science: as a list to be called up, with the assumption that if you know the name, you know the concept. As others have pointed out, it can lead to definite anti-science thinking, like creationism. There are a bunch of creationist M.D.'s, and they often are good doctors, just not honest scientists.
But I also think there's hope. Most of us in the game were in some way misfits as children, and we learned to use science as a way out. I recall a Freeman Dyson quote about using science to counter compulsory Latin and compulsory football at school. These kids' circumstances make them misfits in many minds. My hope for the kids at 112th St. is that they will supplement their newly discovered confidence with some healthy rebellion to show the world how bright and beautiful they are. And I'd love to have them in my class.
Yep, I remember that from one of his books, too (probably __Surely You're Joking, Mr. Feynman__).
But I also remember that when he taught his fresman class at Caltech the course that became the basis of his celebrated lecture series, the course was a flop because no one could follow him.
So, you can't just go rote, but you have to lay some very solid and tractable groundwork.
The last part made me cringe. If you ask someone a question about science and then tell them they're wrong because their grammar is incorrect: (a) you're wrong (on the most basic level of wrongness on which a person can be wrong); and (b) you're a jerk.
Um, that's just one of the rules of the game they're playing.
Contestants on Jeopardy who don't respond in the form of a question don't get points even if the content of their response is otherwise correct. Do you snark at Alex Trebek?
OK, important point here: They are in Fifth Grade.
That may not mean a lot to most of you, but getting a fifth grader to remember some of this stuff is quite amazing.
I remember when I had to take remedial social studies in the summer after 6th grade and I absolutely *HATED* the teacher for forcing me to learn the location of every country on earth. But now, Oh, now, I love the fact that when I see sierra leone or cote d'ivore(sp?) or uriguey in the news I know exactly where it is.
I think that the modern teaching establishment is so afraid of memorization, for decent reasons, that they refuse to use it as a base for learning. Which I think is a shame.
"Is this any way to run a science class?"
PZ and others. I teach high school science in a school where maybe 2-3 kids are going to go to college and study science/engineering. My teaching style (in chemistry as well as a mainstreamed physical science with a huge diversity of kids) is exactly the opposite of this. I focus almost exclusively on 'doing science'- laboratory based activities that the students might not even fully understand as they're doing them. But they're excited, they have real-world applications (hey, how much Aspirin is in one of those tablets and how on Earth do you make aspirin? How do we make pigments for paints?).
And then something amazing happens. I see the end of the year approaching and say "oh crap, we forgot to cover all these chapters out of the text that we need to do". And then we start covering them. Fast. And they pick it up. Quick. Suddenly, ionic bonding and types of reactions are a whole lot more interesting when they've made pigments and painted with them. Acid/Base chemistry that we touched on earlier when titrating our 'Aspirin' samples is really pretty darn easy. Learning how to write electron configurations? So much easier than quantitatively determining the percentage of a Tums tablet comprised of CaCO3.
I think this guy has it EXACTLY BACKWARD. Have the kids do science (I've never seen a group of high school kids off task when tackling a relevant, real-world activity). Then teach the explicit content. Obviously he'd have to tone down the material to meet his classes- but there's no shortage of 'real science' that can be done in elementary school.
Most importantly: I had a bright, talented kid say to me in class today "I never really liked science, I never thought I'd take science in college but your class has made me question that belief. Thanks."
I'm not trying to toot my own horn, but we need more teachers to teach this way if we want to re-build our science education.
I know it's one of the rules, Caledonian, I thought that much was obvious. If you're going to play a game in the classroom you'd think the rules would be geared towards, I don't know, learning rather than its exact opposite. The point is that if the kid leaves thinking "that's the rules" rather than "this guy is a moron and a jerk" she's left knowing a little bit less than when she went in.
Memorization is great when used in combination with real-world applications. Unfortunately, in most classes I have taken (elementary, high school, and even a few in college), you never get that opportunity. You spend a semester learning "facts," only to forget them a couple of months later because you had no opportunity to implement them. Using only memorization is a waste of time, because in the end you learn nothing but the ability to regurgitate information on cue.
It would not be difficult to incorporate the scientific method into a fun classroom experiment. My 6th grade teacher required the entire class to design and execute an individual project for the district-wide "Science Fair." We had to come up with our own topics, and most students had a blast presenting their projects. You'll never forget the six steps after that experience.
The words fly out of her mouth: "The six steps of the scientific method are purpose, research, hypothesis, experiment, analysis and conclusion."
"Correct, 20 points," White says.
Ridiculously oversimplified. What about funding, publication, breaking graduate students' spirits, conferences, departmental infighting, and teaching as few undergraduate survey courses as possible?
Jeff, my friend who teaches in Haiti has experiences that your post remind me of. Like in Feynman's anecdote that Geoffrey mentioned, her students could reel off detailed explanations of Cartesian geometry, but ask them to plot a simple graph from data, and they couldn't even start it. The problem's overdetermined (poverty and politics play into it), but a big part of it is the basis in the old French educational system, with its emphasis on rote memorization.
She started doing science and math activities in her classes, from all kinds of sources--one big hit was her adaptation of David Letterman's "Will It Float?", teaching the concept of "density" using locally-available fruits. I'd say her results (initial wariness, then growing acceptance, and increased demonstration of learning outcomes) bear out that we need more teachers teaching science the way you and she do.
At least he didn't have to have a student explain to him that no, it was the SPERM that had the variable chromosome and the egg was the constant one. I had to correct my 8th grade science teacher on that point. She had us do nothing but Punnett squares for 2 weeks straight because she just couldn't believe we understood the concept. I also found out from after-class arguments that she thought Reagan was the US's greatest president. I wonder if there's a connection here...
Whereas my 7th grade science teacher didn't believe in evolution, was under the impression that she wasn't even allowed to name what she believed, and thought that humans were the only animals with fully four-chambered hearts. She had the extremely dubious excuse of having a degree in PE; I forget what my 8th grade teacher's excuse was...
But now, Oh, now, I love the fact that when I see sierra leone or cote d'ivore(sp?) or uriguey in the news I know exactly where it is.
But Coathangrr would fail that 5th grade Jeopardy test on spelling, wouldn't s/he? And by that same token, can we then say that if 5th graders instead got some real insights into science, then they wouldn't be as easily misled by creationists, homeopaths, and other snake-oil salesmen?
I think that the modern teaching establishment is so afraid of memorization, for decent reasons, that they refuse to use it as a base for learning. Which I think is a shame.
Memorization is a base for learning?? News to me! Growing up in India, I had a lot of rote learning thrust upon me. Once I came to grad school here in the US, I remember feeling elation over the fact that I was allowed to take notes/books into most examinations. No more rote learning, and I loved it!
Feynman had a lot to say that's relevant here. Earlier commenters have already mentioned one thing, the expert regurgitators of South America. I'd add two more:
Feynman defined science as a method in which experimental verification is the final arbiter of truth. Spot on! Experimental method is important, of course: all that crap about purpose, hypothesis, whatever.
Someone once argued that astrology is scientific, because it is based on calculations of astronomical bodies. I shot the above Feynman quote at them. Alas, being adults, they weren't able to process that any more.
The other Feynman story is about how he decided to dabble in biology for a few months. At some point, he was presenting some results to his lab group, and started off by drawing the anatomy on the board, and labeling the bones. His audience interrupted to say that they already knew all that. Feynman was utterly astonished that they'd waste time learning all that crap by rote, when (a) it could all be looked up whenever needed, and (b) the rote memorization didn't aid any insights.
I couldn't agree more!
Y'know, I've worked with younger kids, especially in math, and there are a couple of comments I just have to make.
1. Requiring inner city kids to answer in Standard English is not the minor quibble it seems to some of us. Teaching inner city kids to speak Standard English is IMPORTANT. I go to an urban University, and have encountered a *lot* of people who were never forced to learn Standard English as children, and are unable to function in academic or business environments because of it. That's the point of that rule. It has nothing to do with science... but a lot to do with pedagogy.
2. This isn't a science class. It's a learning and study skills class, that uses science as a vehicle. Much of elementary school has that format. If the kids learn something about science at the same time, that's nice, but that's not the point.
3. Memorization has its uses. Specifically, if a kid can memorize a list of *steps*, he's on the right track. (When I'm working with kids on adding fractions, I've had *hours* where all I said was "Find a common denominator"... "Add the numerators"... "Reduce"... "Find a common denominator"... over and over again.) And, it would be a really good thing if we taught our children *how* to memorize things--because sometimes, you just need to store some information in your memory.
When I was a child, I had to sneak around behind my teachers' backs to learn to memorize. (They were captive to educational fads that said that said memorization was harmful... and at the same time only taught by rote.) It's a skill that's served me in good stead. I also had to sneak around behind their backs to learn how to deduce things from first principles. (I didn't memorize the quadratic formula, for example, until I used it a few hundred times in a Physics class; I spent a good three or four years deriving it, first.)
That being said... this *does* miss the point of the reason we teach children science. The parts of the digestive system really don't matter to most people (I mean, the terms for them, not that they keep functioning, heh), but the ability to think scientifically--evaluate evidence, test hypotheses, etc.--is vital. Whether elementary school is the time to teach this skill, or whether that should wait until high school, is another question.
Would you feel more comfortable if this teacher were using geography, or poetry, or some other subject? That it happens to be "science" is merely a coincidence, based on the teacher's own interests. Myself, I'd be much happier if he used a broad base, from various disciplines... because as far as memorization of trivia goes, a wide variety of trivia will help you more than any finely focused set of trivia.
So, no, it's no way to conduct a science class... but that doesn't negate its value as what it is, a passion-for-study class.
I've got a set of Feynman's famous "Red Books".
Do I get 20 points?
Going a bit off topic, this reminds me of a Geography teacher who, once put in charge of Social Education, clearly panicked at the start of the sexual education part of the curriculum. We spent most of the lesson expressing data on birth rates and such in different formats (pie charts, histograms...)
I mean to say, we got the knowledge, but the implications of it - and thus, proper understanding of it - weren't taught to us. Knowing the scientific method is a far cry from knowing how to apply it.
right. PZ can't stand that his little white, middle class methods aren't in use everywhere.
PZ is a lazy white guy who ignores the religious wingnuts in the town next door as they singlehandedly reintroduce Polio to North America, because he is too busy denouncing the wingnuts elsewhere for trying to subvert evolution.
Now, someone is not using the correct white, middle class, college level, imaginative teaching methods that PZ is such an expert at (how many of PZ's elememtary school students have gone on to college)?
How many of the overwhelmingly white, middle class fourth grade children in Morris are illiterate children with single parents enduring weekly drive-bys?
How many of the local Black and Hispanic students in your little, parochial world have no educated role models? How about Native Americans in Morris? Do you even know if there are any?
You should be standing up and shouting that someone is trying to help these children. You should be sending books or donations to that school, but no; you're disturbed that they're not being taught correctly.
You should post an apology for being slow witted and parochial. We'll let the racism slide for now, as you probably didn't intend it.
While I agree - very strongly - that rote learning alone is useless, it is daft to pretend that it is not critical. Partly it's about training the memory: any discipline, and especially any scientific discipline, will require the student to learn a lot of terminology, and that is something that does not come naturally to many of us. It's simply wrong to suggest it's not necessary because you can look it up at any time, too; yes, you can use a reference book or whatever to find the odd word but if you don't have the vocabulary of your discipline mastered then reading anything will be like trying to read a novel in a foreign language using a dictionary (oh, yeah, a trained memory is essential for learning languages, too).
Most of those criticizing rote learning seem to be those who have experienced it, at least to some degree (Amit Joshi, for example, complains that he 'had a lot of rote learning thrust upon [him]'). This may be blinding those commenters to what they actually owe to that training. I went to a rather wacky 'alternative education' type of school in which rote learning played no role whatever; the result of this is that while I learned to think and to reason fairly well (well enough to get through uni without real effort) I find actual learning incredibly strenuous - I'm trying to teach myself about molecular biology, for example, but find it hard going because the vocabulary is so hard to master.
What I'm saying is that while rote learning on its own is useless, without rote learning as a component of early training you're hamstrung.
Oh, and I think that Lovepettis' points above were well made and should be noted, since he seems to be the only one to have commented with the experience of teaching at the level described.
Thankfully, a few people have reminded everyone that this particular school is quite unique. The article mentions its standardized test scores are among the lowest in California. 100% of the students are minorities and 50% are learning English as a second language. No more than 1/3 are proficient in math skills and no more than 1/5 are proficient in English skills, even as measured by what are likely very low standards to begin with.
Others like lovepettis and lytefoot have stated so wonderfully and clearly some of the thoughts I had while reading this article. I would be inclined to agree with PZ if this were a standard elementary school in the middle-class suburbs. It's just not this kind of school.
I'll let PZ defend himself in detail, but for what it's worth: Leung Shu Ren: you're an ass.
Yes, it's good that someone is making an effort to educate them, but if it's not done correctly, it just leaves the children with more to unlearn later on, and limited incentive to do so. This is a suboptimal approach to helping them develop the skills they'll need to develop an acceptable and rewarding quality of life and function in the world, let alone appreciate true science.
As for all this "white middle class" bullshit...even ignoring the bizarre pseudo-postmodernist assumptions behind wielding this as a criticism, what you vacuously label the "white middle class" commercial/industrial world is the one these children are going to have to learn to function in if they're to avoid becoming single parents with illiterate children who endure weekly drive-bys and have no educated rolemodels themselves. What viable or desirable alternative is it that you imagine these children have?
But that's beside the point. Until you introduced it, race was not a part of this debate. There was no racism in PZ's post...unless you're under the impression that any criticism of anything related to a non-white person constitutes "racism." And as they say...
Are you really that stupid?
The reporter missed it: The last question was wrong not because of the incorrect verb but because the student failed to include "analysis".
Leung She Ren:
the only racism I see on this page is in your response. From reading PZ's post I don't see any mention of race whatsoever.
Do we even know if the students were black, white, or Latino? Perhaps Asian?
I don't know... I'd hate to criticize his teaching based on the Jeopardy!-style competition he's trying to get them to win. Perhaps they are just memorizing facts ... but the article mentioned that the kids talk science on the phone with each other, handled a tarantula, and learn Greek etymologies. Per the article, most are still learning English - rote memorization and oral drills are probably a great way to increase their vocabularies. (And he probably says it's a two-week process to learn something because that's how long it usually takes, given the kids' circumstances. Sure, memorizing seven words shouldn't take that long - but a lot more is involved here, including learning what those words mean...) (And teaching them that using Standard English is important is probably vital - but that's a different hobbyhorse.)
Anyway, I'm just saying that I agree with lovepettis - and I disagree with those who try to compare these kids with high school students, any high school. If he can get these kids motivated to learn *anything* he's way ahead of the game.
I don't take much issue with his methods - we can argue memorization v. process all day. My big problem is that if he's going to go the route of memorizing facts, MAKE SURE THEY'RE CORRECT FACTS. The two kinds of organisms are plants and animals? Criminey. Even my 5 year old knows about bacteria, fungi, and viruses, at least.
Jeez, biases, biases, biases...
Leung Shu Ren, you might want to read this:
The Implicit Prejudice
Mahzarin Banaji can show how we connect "good" and "bad" with biased attitudes we hold, even if we say we don't. Especially when we say we don't
Take a good long look in the mirror,bro!
What bugged me the most was not the emphasis on rote memorization, but the preacher-like mentality. I'd mind the lack of critical thinking, the gross oversimplifications, and the lack of application a lot less if White didn't teach the kids mantras.
As for the emphasis on standard English, is there any evidence it helps in the long run? There's bound to be some study about the effectiveness of being anal about standard English in black and Latino schools.
High school anatomy & physiology was still mostly rote memorization - we too had a textbook divided up into seven systems (California, progressive district, magnet school, the highest science course in the curriculum). There was rote memorization in several of my wife's undergrad biology classes, and don't even ask about the chemistry, physics, or electrical engineering courses I took.
On my cynical days I'm tired of creativity and imagination in my students, because (in my field, not biology) they can't replace facility with math and reading comprehension, both of which seem to be in short supply at my supposedly-strong-liberal-arts university.
There are a couple of separate issues here. One is getting kids motivated and disciplined so that their odds of academic success are improved. The second is teaching kids about science. This teacher seems to have a strategy for #1 that is working extremely well, and I think that's great. #2...well, #2 isn't coming along so well. That stuff I excerpted from the article is not science--it isn't science in principle, and it isn't science in the details. It's simply wrong.
As for racism, if you think it's OK to teach the wrong answers to science questions if the students are black or hispanic, then I know where the racism lies.
I think we would all do well to remember the level of understanding of science we each had in elementary school. Although I think science is in general not taught the way it should be and could be taught much more creatively and interactively, I think what's going on in this school is more positive than not and it seems like we're just looking for little things to attack here. That doesn't make up for the kids believing things that appear to be actually wrong, but that should be the focus then, and not the methods.
As for the emphasis on standard English, is there any evidence it helps in the long run?
Yes, in that reading (which is not only standard, but very formal compared to speaking) helps with reasoning abilities.
There are multiple "circles" of English, the language we use with our intimates, our acquaintances, passing strangers, our superiors....
By not insisting that students learn standard English, you are crippling them. This isn't a matter of eliminating non-standard English, but *adding* to their language (and social) capabilities.
Let's not lose sight of the fact that he's teaching them garbage. (Sweet, merciful Buddha on a pogo stick, he taught them the stages of a scientific paper as representing the scientific method!)
I have to agree with Dr. Myers, they are not learning (how to practice) science. They appear to be learning scientific facts (possibly incorrectly), which is not the same thing.
However, I also agree with some of the other comments. This instructor is getting the students interested in learning.
The article does not make clear what the intent of the instructor is. The venue, the hook, is science. The reporter has reported it as a science class. The competitions are related to science. However, what better subject than science to teach good study habits, disciple, and pride of accomplishment?
Using science as a subject means that the students themselves can verify the information independantly of the instructor. English, art, or creative writing are all more reliant on the whim of the instuctor, and the subtleties of these fields can be lost on a beginner. Mathematics is similar in that showing an expertise in math requires an understanding of previous principles.
The facts gathered by the scientific method can be learned, verified, and their level of correctness (at least at this level of education) is not disputable.
I submit that maybe the reporter didn't understand the background for the class as clearly as the instructors, or the commentor's on this site, do.
As usual, we have to consider the possible effects of the medium on distortions of the message.
Well put, Flex, the class is teaching the kids how to learn well. Which is actually a good thing for the research side of science, come to think of it.
Some people are losing sight of an important aspect of PZ's complaint. Yes, a certain amount of rote learning is probably indispensible. But rote learning of THINGS THAT AREN'T TRUE is simply not defensible, period. Nor is teaching "good study habits" by conveying information THAT ISN'T TRUE. Those having trouble grasping this, please keep reading the above until it sinks in.
For what it's worth, the Jeopardy game sounds like just one tool in White's teaching toolbox, and the journalist only wrote about that lesson because it was the only one she saw. I doubt if she hung around for more than a couple of hours, and I doubt she had the background to appreciate anything but the more unusual parts of White's techniques.
If White's getting positive results, then it's probably because he's using a variety of methods to keep his pupils engaged. From what I remember of primary school, it was the teachers who were enthusiastic enough to keep up many different approaches who were the best teachers.
Some kids are best at remembering rote, others are hopeless at it, but give them a good story and they'll remember the important parts. Others remember what they do better than what they say. But most benefit from getting all of their brains engaged at different times.
I feel like I should jump to Feynman's defense, now that he's long dead, because there's a subtle point at work here. The Caltech lectures which became The Feynman Lectures on Physics were not a flop when they were first given -- they just didn't work out as expected for the intended audience. Freshman students had the experience of "going to the Chinese opera": it sounded fantastic during the lecture, but an hour later, what the bloody hell did he say? Freshmen started skipping lectures, but more and more grad students and professors started sitting in, so that the lecture hall stayed full.
I had the privilege of experiencing something like Feynman's classes in the early 1960s: Professor Barton Zwiebach's undergraduate course in string theory, the first of its kind in the world. To be precise, I took it the second year it was offered. I heard from the students who had been there the year before that it started really awesome but by the midpoint got tangled in the most imposing mathematics they'd ever seen (this coming from MIT physics students, understand), such that nobody could get anything out of it anymore. "What the hell," I figured, "even half a class that good would be worth taking."
Turned out we second-year kids got a better deal. In the interim, our professor had begun turning the lectures into printed material, the first draft of the textbook A First Course in String Theory. We got the draft of each chapter in the form of lecture notes, and each week we scrutinized them for mistakes -- typos, errors in equations, unclarities of expression. What's more, because we had the mathematical guts of the course on paper, we had a much easier time following the tough bits. We got through the rough middle of chapters 10-13 and then rose to unforseeable heights in the second half of term. I loved it, and I still feel proud that I did my little bit to help improve the book.
Who can tell how the Caltech freshmen would have benefitted if they'd had Feynman giving lectures twice a week and his books to hold and read on their own time?
Several months back, Physics Today printed an excerpt from the new introduction written for the "definitive edition" of the Lectures. Apparently, Feynman's pessimisitc comments in his Prologue about how the classes didn't work were written just after the exam grades had come in, and he was bummed about the students doing more poorly than expected. His remarks about what went wrong got set down at the lowest point of his enthusiasm and -- for all we know -- might just reflect a badly written final exam. Yes, those do happen.
People (including a few professors of mine) have said that the Feynman Lectures are great for seeing things a different way once you've already learned the material "traditionally", but that you can't learn the stuff the first time from Feynman. I think this is total crap. I learned vector calculus, Fourier analysis, etc., etc. first from the "red books". That's how I spent the summer between high school and MIT: cranking up Radiohead really loud and reading Volume II.
In The Demon-Haunted World, Carl Sagan wrote,
And while requiring calculus, the most consistently exciting, provocative, and inspiring science popularization of the last few decades seems to me to be Volume I of Richard Feynman's Introductory Lectures on Physics.
This from the man who wrote Cosmos!
I have marvelous ideas for improving the state of science education which this blog comment window is too narrow to contain. However, I would like to offer my opinion that if you're truly concerned with pumping up children's interest in the subject, quiz-bowl competitions aren't a bad way to do it (although they cannot by any means satisfy all the requirements of a good science education). With some effort, it would be possible to write a much better set of questions.
"Put the following scientists in chronological order: Newton, Copernicus, Galileo, Aristarchus."
Or: "The gravitational acceleration near the surface of the Earth is 9.8 meters per second squared. By Newton's laws, how much is it twice as far from the center of the Earth?"
"Which of the following techniques helped Watson and Crick to discover the structure of DNA? A) nuclear magnetic resonance; B) X-ray crystallography; or C) inspiration from the Rig Veda."
I just tossed off the first questions I could think of; they're probably suitable for a somewhat older age group than the students discussed in this article. Writing good questions is not easy, and maybe it's not a job that should be left to teachers who have the will but not the experience. What if we could get the people who know science, work with it and deeply appreciate it together with the teachers who have years of experience working with children of different ages? And what if we could make a corpus of questions which we could give to schoolteachers all across the country -- heck, all around the world, in this modern age? We admit that a trivia contest cannot be our only avenue for teaching science, but as a tool for making children and parents care, it just might have possibilities.
Check out The Pleasure of Finding Things Out, the Feynman video to which PZ linked a while back. In this interview, Feynman says something quite pertinent: the best way to teach science is often to have no method at all, and to do it every damn way you can think of. If you want to hook everybody, you have to use a crazy mix: some people like pure mathematical reasoning; others enjoy learning about the applications to medicine or engineering; and still others love the historical details, the names, dates and anecdotes of how science grew to its current state.
In (the equivalent of) freshman bio, we were watching some very dated film about plants or life on the planet or something, which kept talking about the "fungal plants". (We had been warned, and anyways the prof kept muttering under his breath "they're not plants" every few minutes. I'm not even sure he knew he was doing it.)
"Fungal plants are very unlike plants because they do lots and lots of things differently. [Copious details about how fungal plants have very little in common with plants.]"
Steve LaBonne wrote,
"Some people are losing sight of an important aspect of PZ's complaint. Yes, a certain amount of rote learning is probably indispensible. But rote learning of THINGS THAT AREN'T TRUE is simply not defensible, period. Nor is teaching "good study habits" by conveying information THAT ISN'T TRUE. Those having trouble grasping this, please keep reading the above until it sinks in."
Okay, I've gone back and read the post and the article several times. Maybe I'm missing something, but here's the things I can identify as not completely accurate.
1. There are more than two main kinds of organisms on earth. And in fact, the two mentioned are not even the most populous.
2. Arbitrary classifications of anatomical knowledge. This is not exactly true, but not exactly false either. As Dr. Myers said, it is rigid, possibly obnoxiously so.
3. Science taught as memorization of formalized, discrete steps rather than the exciting and fluid method as practised by scientists. This is a shame, certainly, and as Jeff wrote above, I think an actual science class with extensive laboratory work would be far more effective at teaching the method of science. However, as the article clearly indicates, this is not a typical science class.
4. From the article, "The meaning of the digestive system is ...?" This is silly. But it's silly because a digestive system doesn't have meaning, and there isn't a way to parse the question to make the question intelligable. This could be a transcription error, and the original question might have been, "the purpose of the digestive system is ... ?" I find it fairly easy to believe that an editor, or the reporter himself, glancing over the article could have made a mistake and substituted 'meaning' for 'purpose'.
Of the four items I find, one is unintelligable, two are rigidly simplified, and one is factually incorrect.
Now I might prefer that factually incorrect information was learned correctly. But there is too little information in the article to determine how factually incorrect the information is in general. That's why some of us are suggesting that while Dr. Myer's complaint is valid, there may be other considerations before condemming a practice. Reading Dr. Myer's comments on the article suggests to me that he is not condemming the instruction, but sees room for improvement. Which I think we can all agree on.
As for having to re-learn information provided by rote memorization at an early age, I think we all have personal examples of going through this. I certainly remember grade school textbooks talking about the division of life into only two kingdoms, the plant and animal. You know, I eventually learned that knowledge was a simplification. The wonderful thing about human minds is that they don't crystalize at some point in development, we can continue to correct even knowledge learned by rote.
I don't think any of us have any trouble grasping your point.
My senior high school biology teacher was acutely aware of the memorization problem, and when asked about it said that knowing how much material one needs to purely memorize in order to do good investigations and experiments is not obvious and something on the lines of him constantly tweaking matters even though he'd been doing the teaching of this sort of stuff for a long time. I agree, especially now many years later. The latter actually create another problem - too many school "experiments" aren't - they are merely investigations and activities as there is no clear manipulation. I remember in my grade 10 physical science class it occured to me we could tweak a few of the activities to at least investigate some hypotheses rather than merely collect data. So after finishing one I asked if I could borrow a balance where none was called for in order to test a hypothesis I had about something or other raised by the required stuff. Fortunately I was praised for independent thought, but I wonder whether this would have happened at other schools.
I agree with those commenters who say that children need to acquire knowledge about science before they can be expected to understand general principles such as the scientific method. However, answers to trivia questions is, in my opinion, the wrong kind of knowledge. They would be better off starting out by just observing nature. Look at leaves close up. Look at bugs. See how magnets attract metal.
Knowing the names of things is useful for communication, but it isn't actually scientific knowledge. It's not knowledge about the natural world, it's knowledge about human conventions (well, I guess that's part of the natural world, too, but only a small part).
There are basically two complaints going on here: that memorization and fake quiz games are bad pedagogy; and that the kids are being required to learn incorrect information. All I'll say on the first is that I don't find anything intrinsically wrong with some memorization, as long as it isn't ALL that students do, AND that it is integrated with hands-on work. As other commenters have pointed out, we don't have enough information from the article to know whether these kids live in some kind of bizarre alternative Jeopardy universe their entire school lives. One hopes these activities feed into others where what is memorized is put to use. The second issue is more worrying to me. Far from being a reason to trash this teacher, though, it points the way to a useful intervention from well-informed people. At least the teacher seems to want to be in the classroom, getting kids interested in being there themselves. He does need to be set straight on some of his "facts" though. Makes me wonder about the science training for elementary school teachers in that state.
Okay, true story here, and I think this stretches the heck out of what this teacher's doing.
At the Junior College I attended in Northern California, we were required to "take a science class." Without really knowing what I was doing, I signed on for this class where the teacher would sell us his textbook, and at the back of every chapter, he published every question that would be on the test for that chapter. All we had to do in that class was write the question down on a flash card, then look up the answer, and write it on the other side of the flash card. The questions were written in the same order that the answers occurred in the text.
We'd then just memorize the questions and answers.
I know it's been a while since I had that class, but I can't remember much about it. All I remember was there was two ways to get "points"; field trips and/or tests. We had to attend a mimumum number of field trips and take a minimum number of tests until we had accumulated a number of possible points, and your grade was just a percentage of points earned.
I got an "A" in college biology, and can't remember a thing about the class, except for one thing: We called the class "Dr Scandoni's Field Trips 1-A."
Had I known, I never would have signed up for the class.
For those of you commenting on the incorrectness of his facts, i'll bet dollars to doughnuts he's teaching from a book and to test standards that state the things he's teaching be district tested. I taught from books that also had similar over-simplifications, but the test was the test, and the place to correct that was in the book or the standards. Take a hard look at the quality of materials available to average or poor schools as well as criticizing the teacher.
One thing I did learn in teaching in an under-resourced school was that teachers were often part of the problem, but never the majority of the problem. Good executive leadership, administrative support, and parental involvement were a much greater factor in school sucess than an enthusiastic teacher making some content errors detratcted from it. I am impressed with the commenters who have stated such agnostic views of the nature of the problem (h/t to outeast and Jesper, among others). A better, more intimate knowledge of the problems of education will help in it's reform.
This pearl-clutching over an enthusiastic teacher getting some of the facts wrong without more direct knowledge of the teacher, his setting, etc., is more befitting of my right-wing septagenerian relatives who make arguments like these and use it to support their pet theory that Mexicans can never be educated (Alberto Gonzales withstanding).
(As a side-note, these same relatives made me snort my rum and coke in a fit of laughter when, in the same conversation, they seriously contended that the reason Detroit was in the shape it was in was because it was ruined by Canadians.)
So, do something constructive, but don't throw rocks at this guy -- he's doing good work in a hard place. If you think you can do better, I suggest joining Teach for America, because it exists to get talented people from all walks of life who were not education majors into schools with a dire need for good, self-sustaining teachers.
OT: Detroit ruined by Canadians? ROTFLMAO!!!
It couldn't have anything to do with regressive taxation, a tremendously large percentage of rental properties, seventy years of racial tensions and corrupt governements?
Nah! It's the Canucks!
After all Windsor is in such bad shape.
(Who is SE Michigan native, and currently works in the Detroit suburbs.)
First time post, and of course I have to disagree with what I infer to be a knee-jerk reaction on the part of PZ. Is this the best way to teach a 5th grade science class? I don't know the answer to that for many reasons. First, does the article actually tell us how the class is taught or simply give us a snapshot of several activities, specifically those surrounding the interschool competitions? Second, the question, is this any way to run a science class? is dependent on the environment, which others have already noted and others have already discounted. In Utopia things may be different, but the environment here is the 112th school. Finally, as a university research scientist and teacher (aka professor), I have found it is easy to note deficiencies and/or problems in grant proposals, papers I review, etc. However, I find it is not sufficient to say "this is no good," it is crucial to suggest an alternative that is better. Also note this class appears to be doing more for the students than simply training to be future scientists, things like encouraging discipline, pride, and motivation to be better. All important! I think that for these students, if this class improves their self-esteem and helps keep them in school and/or out of gangs, I could care less if Dr. Myers may be forced to re-educate the student who actually goes to college in Morris MN or attends my class. So given these issues, I would like to hear an alternative solution to the huge numbers of problems facing these students including a receiving a proper science education. I do not appreciate the armchair quaterback approach taken by Dr. Myers here.
"(They were captive to educational fads that said that said memorization was harmful... and at the same time only taught by rote.)"
I've been there. Memorization is harmful, but lecturing is worse than beating kids to hear some of the teachers I've heard tell it...this when 90% of their instruction time amounts to a standard lecture plus visual aids. I was amazed at the level of delusion that went into proclaiming loudly that lecturing was never done while doing it every day with some trivial serial numbers filed off.
PZ, this op-ed in today's NY Times tends to agree with your position, although one might argue he's talking college, and not elementary:http://www.nytimes.com/2006/05/25/opinion/25thu4.html?ex=1148702400&en=…
I must admit I'm both troubled and intrigued by the story you posted. On one hand, it's good that someone has found a way to make just the knowledge of science useful in interscholastic competition. On the other hand, book and flash-card learning without lab experience -- or experience in the great outdoors -- just doesn't strike me as a way to turn on kids to science.
if they were to show up in my classroom 6-12 years later, I'd have to tell them that Mr White was all wrong, and that they need to forget everything he ever told them.
Ironically, this is what I have to do to many of my freshman chemistry students who were taught some "chemistry" in introductory college or high school biology and anatomy/physiology courses.
But they don't hate me for it. They should realize by that point in their lives that a lot of what they learned earlier was a simplification of how things really are.
Leaving aside the Factual Inaccuracies (about which I do also suspect there may be some school district imparatives), I liked Blake Stacey's point that good teaching at any youth level requires using a collection of different methods to hook the different kinds of learners. This memorization competition wouldn't excite my 5th grader and if he could get good feedback by this kind of memorization without understanding, it wouldn't bother him that he didn't understand it as long as it kept the authority figures off his back. On the other hand he is absolutley fascinated by experiments or research into the question of whether something is true or not true.
Holy Cow! I just read the article. It's worse than PZ implies. Did I miss something here, or is this article about a guy who culls the best science memorizers out of school classes for special treatment (with "swift and severe" consequences for those who dont know the "meaning of digestion") and leaves the identified losers to their scholastic (more than 2/3 failing everything) fate. Of course all the special kids are going to be calling eachother on the phone. The problem is that once the losers become part of the majority out-group there isn't any more incentive OR EXTRA HELP for them and they have also recieved the important (likeley again innacurate) information that they are BAD AT SCIENCE.
I've seen this before with extra help programs for minorities or poor peforming schools. High profile programs or "media-worthy" programs that in the end just focus on a chosen group like this one.
The reason ALL kids need to understand basic science AND critical scientific method is because it is important for being a functional citizen in the modern world, not just so they can get into a good college.
[I]f they were to show up in my classroom 6-12 years later, I'd have to tell them that Mr White was all wrong, and that they need to forget everything he ever told them. And then they'd hate me and they'd hate science.
When I started at secondary school (roughly analoguous to American high school), my science teach - one of many excellent science teachers I've been privileged to have - began by telling us that "All the science you learned in primary school is wrong. If you go on to study science at college, you'll learn that everything I taught you is wrong. If you go to university, you'll find that all the science you learned at college was wrong."
I forget what the point was, except maybe that sceince education is a series of ever-improving approximations of actual science. Certainly he managed to avoid having people hate him, or science.
Of course, most science teachers don't use an entire pound of potassium in a single experiment. Or get the fire brigade called out while demonstating how well phosphorus burns. That's science for teenagers.
"Did I miss something here, or is this article about a guy who culls the best science memorizers out of school classes for special treatment (with "swift and severe" consequences for those who dont know the "meaning of digestion") and leaves the identified losers to their scholastic (more than 2/3 failing everything) fate."
I don't quite get that from the article. It reads to me like there are two activities discussed in the article.
First, a science teaching program open to any student. The article suggests that the program is extra, beyond the required curriculum, and if the students don't keep up they are dismissed for that day (but can return on a later session).
The article also discusses a competition, which if my own High School experiance is any guide, would only be the top few students in the class.
Now, that may be considered culling because it sounds like the students have to volunteer and are asked to leave if they can't keep up. However, as the program is apparently open to anyone, I wouldn't consider it culling.
However, you do have a valid point in as much as I would certainly expect only a small handful students would volunteer in the first place, and those would be already motivated. After all, it is extra work beyond the usual coursework.
Ultimately, I would certainly like to see every student get a reasonable grounding in the sciences. Whether this program is a good first step or just another tough-love transitory approach remains to be shown.
It's important to remember it's far more likely for any one of these children to never graduate high school than it is for one to move on to a job or career in science.
This teacher may not improve their chances of becoming scientists, but he certainly may improve their chances of getting a diploma.
The people I've met who are most ignorant about and contemptuous of science are the ones who think science is just memorizing other people's facts. University-educated adults who somehow remain blissfully ignorant that scientific research is taking place every day.
High school diplomas and careers in science aren't mutually exclusive; they usually go together. Maybe engaging inner-city kids in science rather than trivia might help both to happen.
(I do agree that it's likely the teacher uses other techniques in his classroom. In my high school experience, quizbowl games were just fun ways to review for tests, not everyday occurences.)
Just to clarify a fewpoints:
Clare wondered about the science training for elementary school teachers in that state. (California) Mr White is not a scientist nor is he a credentialed teacher in California. He is a businessman, and his science knowledge is self taught.
Lovepetts wrote "For those of you commenting on the incorrectness of his facts, i'll bet dollars to doughnuts he's teaching from a book and to test standards that state the things he's teaching be district tested." Well you lost that bet because the program does not use a textbook nor was it developmented solely to met any state standards.
The article erroneously implies that the students are motivated by science when truly are motivated the opportunity to handle animals. Here is an eerie quote. "The children are eager to please White, in part, because they like him, in part because they know the consequences for displeasing him will be swift and severe."
I had the opportunity of having Mr. White come to the school in which I taught in the San Fernando Valley several years ago. He worked with two of my third grade classes and one of my fourth grade classes over a three year period. As someone that has worked directly with Mr. White I had to reply to the above comments. Some very legitimate concerns were raised. Please keep in mind that Mr. White runs a program to supplement the teaching of the regular classroom teacher. His work lays a foundation in which the classroom teacher builds upon, allowing students to use the information he brings and apply it in deeper ways.
There were many complaints about the "facts" that he teaches. For example, it was mentioned above that he teachers there are ONLY two kinds of organisms. That is not true. He teaches elementary students two types of organisms, NOT that there are ONLY two kinds. Big difference.
Others complained about his method of teaching and that he is too strict. I suggest you talk to the kids. Ask them what they think of him. Ask the students that he has removed from the group because of their behavior. The article accurately described students hanging their heads and not arguing when dismissed. This is because Mr. White teaches students personal accountability for their actions. They work hard to get back into the group and rarely have to be removed again.
Many complained about Mr. White telling a student she was wrong for using "is" instead of "are" when answering a science question. Harsh? Maybe. But I'll bet that student learned the difference between "is" and "are" that day. So what that she lost 20 points and a contest. That incident may have taught her a lesson that could be the difference between a successful or unsuccessful job interview in the future. And what is wrong with telling a student that he or she is wrong? When Mr. White tells students they are wrong, or when he removes them from the group, he is firm and fair. But he does it in a manner which conveys he has high expectations and more importantly, that he loves them.