There's a lot of fascinating robotics work being done these days, although it is disheartening to see how much of it is designed to help kill people or just kill them outright. But not all of it and over at Boingboing there was an example of a project designed to do household chores. We already have Roombas to clean the floor and a lot of laundry is sort of automated (at least the wash, scrub, rinse, dry cycles are). But then there's the unpleasant task of taking them out of the washing machine, dumping them into the drier, taking them out of the drier and folding them. The firs three of these would seem to be pretty straightforward to automate, but folding them is a challenge. In fact, Mrs. R. says I am naturally folding-challenged, which is not only true but a convenient way for me to get out of doing an unpleasant chore. On the other hand it would be nice if neither of us had to do it. We need a robot.
That's where this amazing video clip comes in. It shows a robot folding towels after picking them up from a haphazardly dumped pile on a table. This is the product of a National Science Foundation (NSF) grant, and before you start deploring how our tax dollars are being used, just take a look at this and tell me it isn't really mindboggling that you could get a robot to do this task, which is not at all trivial. Here is the Abstract of the paper which is to appear in the proceedings of the International Conference on Robotics and Automation (ICRA) this year:
We present a novel vision-based grasp point detection algorithm that can reliably detect the corners of a piece of cloth, using only geometric cues that are robust to variation in texture. Furthermore, we demonstrate the effectiveness of our algorithm in the context of folding a towel using a generalpurpose two-armed mobile robotic platform without the use of specialized end-effectors or tools. The robot begins by picking up a randomly dropped towel from a table, goes through a sequence of vision-based re-grasps and manipulations— partially in the air, partially on the table—and finally stacks the folded towel in a target location. The reliability and robustness of our algorithm enables for the first time a robot with general purpose manipulators to reliably and fully-autonomously fold previously unseen towels, demonstrating success on all 50 out of 50 single-towel trials as well as on a pile of 5 towels. (J. Maitin-Shepard, M. Cusumano-Towner, J. Lei and P. Abbeel, Cloth Grasp Point Detection based on Multiple-View Geometric Cues with Application to Robotic Towel Folding; hat tip Boingboing)
And here it is:
PDF of the paper here; Videoclip of the above and one other here.
- Log in to post comments
i can fold towels blindfolded and with my toes. i want to see that robot fold fitted sheets into a neat rectangle and fold jeans so the crease is centered on the front and back of each leg.
@GrllScientist:
Exactly what I was gonna say. I won't be impressed until I can find a robot who can fold a fitted sheet into a perfect rectangle!
(And I would be tempted to purchase said robotic folding apparatus)
It's Rosie, the robot from the Jetsons!
One problem with robotics research is that the expectations are so huge, after many decades of fictional robots. This robot is doing something very difficult - even granted the convenient homogeneous background and everything - and all non-specialists can see is that it's still not anywhere near a human or a Marvin in capability.
By the way, robotics research for military applications almost all seems to originate from one single country. I agree it's distasteful and depressing, and I'd certainly not want to work on such a project. To each his own, though, and as long as the research is reasonably open there'll at least be some actual useful results as a side effect.
I agree with Janne - what we're seeing here a huge improvement in robotics - not just because the robot can move its arms around, but more specifically because it can recognize objects of distinctly different textures and get the folding right every time.
The improvement is not in the machinery, but in the algorithm that the machinery uses to interpret its surroundings. I think we do not appreciate just how complex our perception of our surroundings really is and how hard it is to design a robot *from scratch* that does it without billions of years of evolution on its side.
Speaking from the standpoint of computing/software/control algorithms...damn that IS impressive!
Seriously. Wow. I like what they've done.
[b]i can fold towels blindfolded and with my toes.[/b]
You really are bored.
I'm having flashes on that robot generalizing application of the algorithm to folding, over and over until they lie flat, people.
And on future robo-sapiens looking back on just this picture and saying "Wow how far we've come" and "Isn't he cute?"
And on the anti-robotic-warfare conference coming up, here in the Mid-Columbia-River-Gorge region, April 16-18 (big one, with Sheehan and sponsored by bunches of nationwide orgs); a boeing-not-boing subsidiary on the WA side of the river is one of the big manufacturers of robotic UAVs,in particular miniatures, leading the local rush to techno-pseudo solutions subservient to big corporate funding, a rush of which the wind farms that will drive some of us out are but a fraction.
"Aren't they cute?" "Kills the wildlife, sickens the people, but green as the day--and wow look at those hospital corners!"
I'm with GrrlScientist and Kevin: Sign me up in advance for a robot that can fold a fitted sheet into anything even resembling a state of flatness. I sure haven't been able to manage it, ever.
Interro, Grrl: Not a politically correct quote, but it is, after all, Samuel Johnson, who was not the most enlightened when it comes to women. That said, my attitude to the robot folding towels could be derived from Johnson with a search and replace:
From Boswell's Life
Once we get to the point where robots can do all useful work, we can simply reduce the global population to 300 million elites and perhaps 100 million sex/entertainment workers, and save the planet from mans CO2. Just need to find a clean way to get rid of the surplus, the robots can clean up the mess I guess.
Read a book a while back called "The World Without Us" People are already researching how long it will take for our buildings, roads and stuff to disappear on it's own, paid for by you know who.
It's Rosie, the robot from the Jetsons!
"taking them out of the washing machine, dumping them into the drier" - washing machines that dry clothes completely are common in England.
I thought it was an April Fool's joek until I read on Boingboing that it had been speeded up 50 times.
"and before you start deploring how our tax dollars are being used, just take a look at this and tell me it isn't really mindboggling that you could get a robot to do this task, which is not at all trivial"
>> Revere, this is a waste of tax dollars. They probably spent millions on this robot -- money which could easily have been used to help the poor pay for their medications. It's very clear that with the current economic crisis, only the rich will be able to afford this toy. I'm sure Chomsky would agree. He spoke many times about how tax dollars are wasted on meaningless research which advantages a very small group of people.
Marc: By that criterion, consider the waste of money on studying abstract mathematics, philosophy or literature. Or the waste of time and money I've spent blogging or you reading me.
Marc: This isn't meaningless research. Just because this particular robot probably has no direct real world application, it is a significant advancement in using robots to understand cloth and other non-rigid material manipulation. This sort of research has plenty of industrial applications, not to mention enriching our understanding of how to have machines manipulate real world objects.Research requires a foundation. Without a foundation, things down the road cannot be invented. The problem is we have no way of knowing which parts of the foundation are useful, so we have to explore as many parts as we can. Several hundred years ago, we couldn't have just set out to build a computer. It required much research in many different, seemingly unrelated fields.
@Marc
By that criterion, you wouldn't even be having this conversation. The internet itself is something that would only benefit a very few and the very well off for the first 20 years of its existence.
Although there is much effort developing robotic hands - particularly 5-finger hands that can work with other hands (see the DARPA ARM project), the real issue is the unchecked exponential growth of robotics research in all areas of commerce and defense.
I've written a blog on the unchecked aspect - the forthcoming Singularity aspect.
I'd appreciate comments:
http://www.everything-robotic.com/2010/04/rethinking-singularity.html
Thank you.
@Revere: I don't see how you answered my argument. First of all, the "abstract math" has more uses than you may know. Think Hilbert Spaces are abstract and useless to society? Wrong. They are essential to quantum mechanics. Second of all, let's see how much money is actually spent on me studying math. Millions like for the robot? Hardly. And with the new raises in tuition rates, even less. My point still stands: every year large sums of money are wasted on toys (robots that can do laundry, iPads, etc.). This doesn't mean we should shut down the fields of AI and Robotics. How about a useful robot? One that can vaccinate patients or one that can perform bypass surgery (that one already exists).
@smidget:
"This isn't meaningless research. Just because this particular robot probably has no direct real world application, it is a significant advancement in using robots to understand cloth and other non-rigid material manipulation."
>> No it isn't. Alex, who also comments here sometimes, has a teacher in an AI class who does research in both AI and Robotics. For example, he has done research in cloth modeling. I can show him the video but I'm sure he would agree that there is nothing revolutionary here. As a matter of fact, I will show him the video.
"This sort of research has plenty of industrial applications, not to mention enriching our understanding of how to have machines manipulate real world objects."
>> Again, I seriously doubt it but I will check with an expert. To me, this is just a toy.
"Research requires a foundation. Without a foundation, things down the road cannot be invented."
>> Extrapolation. The foundation for Robotics and AI is not a robot who does laundry.
"The problem is we have no way of knowing which parts of the foundation are useful, so we have to explore as many parts as we can."
>> This is simply ridiculous. A robot who performs bypass surgery is far more useful than one who can do laundry. This is very clear.
"Several hundred years ago, we couldn't have just set out to build a computer. It required much research in many different, seemingly unrelated fields."
>>No it didn't. The first computers came from mathematicians. Babbage, Turing, Lovelace, von Neumann, etc. Tell me, how did evolutionary biology help computer science?
@Lynxreign: You missed the trees for the forest (no, not the other way around). I did not say that Robotics and AI as a whole are a waste of cash but that this particular application is. In the same way, the Internet as a whole is not a waste, but a particular use of it, say online games, is a waste. Now, about the well off and privileged. The internet in the early days was used by scientists to communicate findings or solve complex mathematical problems. We directly benefited from their research, as opposed to this robot.
Marc: I think you missed my sarcasm in the abstract math comment. I made the comment because y ou do math. I am well aware of the use of Hilbert spaces in QM (it's part of my research), not to mention group theory in QM and crystallography, etc. You understand the value of math because that's what you do. You don't understand the value of robotics (BTW, there is some nice mathematics involved; see the book Geometric Folding Algorithms) because that's not what you do, nor do you seem to see the value of what looks to you like something trivial (but to robotics folks is far from trivial). Remember, too, the huge contribution of recreational mathematics to useful things. As for the cost of this, it is mainly software and unlikely to be millions. Less than the software for the 4 color problem or characterizing the simple groups, I'm sure. And I don't begrudge those mathematicians a penny of those incredible efforts, despite the fact that it has no apparent use to me (at the moment).
@Revere:
"I think you missed my sarcasm in the abstract math comment. I made the comment because y ou do math."
>>No, I picked it up.
"I am well aware of the use of Hilbert spaces in QM (it's part of my research), not to mention group theory in QM and crystallography, etc."
>>Cool!
"You understand the value of math because that's what you do. You don't understand the value of robotics (BTW, there is some nice mathematics involved; see the book Geometric Folding Algorithms)"
>>It's clear you didn't read my answer completely since you wrote:
"You don't understand the value of robotics".
Of course, Robotics and AI are useful, as I said:
"This doesn't mean we should shut down the fields of AI and Robotics."
My criticism was:
"How about a useful robot?"
In other words, it is this particular robot that I have a problem with, not the whole field of study.
"because that's not what you do, nor do you seem to see the value of what looks to you like something trivial (but to robotics folks is far from trivial)."
>>This is false. I don't do medicine but I understand the importance of vaccines. I never said that Robotics and AI are trivial. My argument was against this particular robot, not the whole field.
"Remember, too, the huge contribution of recreational mathematics to useful things."
>>Yes. And how does that help your argument? A robot doing laundry has as much potential for scientific research as a mathematical equation?
"As for the cost of this, it is mainly software and unlikely to be millions."
>>I will also check for the cost of this but I really doubt that it's as inexpensive as you say. Alex's AI teacher got a grant with six zeroes for his research in cloth modelling. And that's just modelling. To do all that the robot did above, it's probably much more expensive.
"Less than the software for the 4 color problem or characterizing the simple groups, I'm sure. And I don't begrudge those mathematicians a penny of those incredible efforts, despite the fact that it has no apparent use to me (at the moment)."
>>As for the 4-color problem, it definitely affected you if you work in QM. A simple Google search points out this article by Kainen of the Math Department at Georgetown:
"A complete theory of quantum mechanics must, therefore, contain a solution to the Four Color Problem"
http://atlas-conferences.com/c/a/f/y/06.htm
Marc: Given what you say, I am surprised you don't see the value of this. This is an exceedingly difficult problem and the algorithm (corner detection in a very amorphous folded material) would seem to have lots of applications. Plus it is likely very cheap, the kind of project that is done by talented undergrads. As for the 4 color problem, it sounds like the implication arrow is going the wrong direction for your point. The 4 color problem doesn't imply QM but (in this view, anyway) the other way around. So instead of putting a dime into the 4 color problem, the money should have been in finding a complete theory of QM. Of course the thing that's wrong with this argument is that it isn't a zero sum game nor is the value of the 4 color problem confined to any connection to QM. Incidentally, a complete theory of QM would do a lot more than provide a solution to the 4 color problem, a solution we didn't need from the practical point of view because cartographers didn't need a proof to use it. So by your argument it was a waste of money.
revere: You say "consider the waste of money on studying abstract mathematics, philosophy or literature." Yes, let's consider it. I seriously doubt the annual teaching budgets of 10 or even 100 university philosophy or literature departments come near the cost of this one robot. And as for making one's living in those two areas--mostly a joke, no matter how well regarded one's work. But look, guys, there's a bigger issue, as you know. Imagine you are a mother with a baby to raise and your husband or partner has gone and your family can't or won't help much; you have rent to pay and need food and some way to get around and sometimes medical providers. Your baby wakes once or twice each night and, like any little one, gets sick relatively often, so you can't keep a job (when you try, you get sick from exhaustion and overwork and end up in the hospital). How do you manage? Should large hunks of what monies the military-industrial-tech doesn't eat up go to help people like you, people needing medical treatment generally, people who are homeless, or to making laundry-folding robots (surely the algoriths exist without building the robot)? Or do we remove the military-indust-tech (which, yes, may include oodles of robots)? Which reminds me, those of you interested in the anti-robotic-war issue, there's the big NW conference on this in mid-April--look on columbiariverpeace.org for details).
re 18---terrific blog, Frank. I'd wonder what Searle means by the impossibility of robotic consciousness--probably just the philosophy point that nothing would lead us to call it "consciousness" (nothing, that is, in current language usage)--and the empathy/fraternity point is certainly crucial (for us--not nec'ly for robo-civ). Nice blog.
@Revere: Ok so I concede the 4-color problem argument. However, I still believe this thing to be a waste of money. As Paula pointed out, there are people in the US now who can't even afford the most fundamental health care. And I seriously doubt that this robot has been made by "talented undergrads" or that it is "very cheap". I have friends in computer engineering (with intentions of going into robotics). They do make robots for their final year project and they are quite smart. None of their robots is as sophisticated as this one. And again, just for accurate modelling of cloth Alex's teacher got a grant of over 1 mil. Ultimately, my biggest problem with what you wrote is this:
"In fact, Mrs. R. says I am naturally folding-challenged, which is not only true but a convenient way for me to get out of doing an unpleasant chore. On the other hand it would be nice if neither of us had to do it. We need a robot."
Many people on this planet live in horrible conditions and don't mind folding their clothes themselves and as best they can -- if at all. You complain because you don't have the latest toy to do it for you. I mean we're talking about folding something. How hard can it be??? Imagine how many vaccines could be made with the money used to make this damn thing.
Adding to my previous point. Look at the intro of the video (first few seconds) and Google the 4 names. Here's what you get:
1) JMS: PhD student at UC Berkeley
2) PA: Assistant Prof. Dep. of Electrical Eng. UC Berkeley
3 and 4) JL and MCT: Senior Math and Comp Majors (assisted the project but didn't do much).
Since the leaders of the project are a PhD candidate and a prof., it's clear that a grant was involved. This was not done by "talented undergrads", although 2 undergrads "assisted". It's also unlikely, given the above, that this was "very cheap".
http://www.physorg.com/news189698140.html
Marc: Lighten up, Marc. The "we need a robot" was clearly a rhetorical device to introduce the video clip, which somehow has been elevated into a major ideological issue. But for the record, since I have some idea of how much academic research costs, what is involved here isn't so much the cost of the project but research involving the algorithm, education of a doctoral student, some undergrads, partial support of a faculty member. I don't have any idea what kind of "cloth folding" project is being supported for $1 million (over 1 year? 2 years? 3 years? includes tuition? overhead? all of which do multiple duty in this world besides the research) but why is it being supported at all? Someone wants to fold cloth? As for the people who live in horrible conditions, you are right. I am a doctor in an inner city medical center so I see it all the time in a major US city, and I suspect you don't know the half of it. But consider the logic. How many kids could you immunize for the cost of your number theory class? You are trying to compare non-comparables by monetizing them. I suggest it works for this no better than it works for comparing art, music, pure math or epidemiology. I'd certainly like to see the resources devoted to military purposes used to heal people rather than kill them. That's the Big Picture. What you are complaining about, without proper foundation in my view, is that this project is a waste of money because what is learned or produced is of little value compared to the cost while neither knowing the value nor the cost. I don't either, I'll grant you that, but that's the Little Picture. I think the value beyond the excessively literal interpretation that it is confined to actually folding laundry is quite plausible, as it is for most robotics research, which is in its infancy. I'm impressed by the difficulty of it. So that's my bottom line. I think we've probably exhausted the topic.
@Revere: The cloth modelling costs one million because it's a team project over (from what I remember) 3 years and done by 5 researchers. The idea is to simulate on a computer, as accurately as possible, the folding of some cloth. Sounds retarded and I also believe it is. A perfect example of the art of wasting taxpayer cash.
You gave a very bad example with my number theory class. It costs around 500$/student for Canada, Quebec residents. Everyone in my class is a Quebec resident. There's 20 of us, including Alex. That's 20 x ~500 = ~10,000$. Of this, we pay ~20% and the government ~80%. So that's ~8,000$ of taxpayer fees that goes into my class... as opposed to who knows how much into the robot (but very likely much more than 8k). Oh and did I mention that number theory has many useful applications in fields like biology? It's likely that one of us will do some kind of research that will help society in a much greater way than folding cloth.
Also, I doubt that there's more value into this robot than the folding of cloth. From what I read in the article, it seems like this is just some "Hey, let's try something really cool" project.
And I do know more than the half of it. I am involved in activist groups at the university: The Global AIDS Coalition, Demilitarize McGill and Greening McGill. Also, recently, a family who is friends with ours has adopted a Haitian baby. So I've seen bad shit too, probably not as much as you though.
Is this whole discussion about the robot part of the Big Picture? Yes, it is. The robot, just like the cloth modelling, are examples of how taxpayer money is wasted in research that does not even remotely benefit the taxpayer. There's plenty of other examples. When you add the money of all this useless research, it becomes quite significant. Now, here's an example of useful robotics research:
http://www.ohgizmo.com/wp-content/uploads/2006/09/bionic%20arm.jpg
Marc: Of course your number theory course cost more than you say. You just added up the tuition, but there are costs to it you aren't counting ("overhead", administration, time spent doing that you aren't working with AIDS patients, etc., etc.). It's one semester. So let's say $30K for the year (3 semesters), annualized. To teach an elementary number theory class. Taxpayer supported, 80%. That buys a lot of immunizations in Benin or BC, for that matter. How does teaching you number theory benefit the taxpayer? Well, you learn a useful skill, perhaps one day you will do something really important to everybody, it's of value independent of whether it's useful, etc., etc. All the same things can be said for the robot project. We don't know the cost of the robot project. It's probably more than $30K because when students are involved grants like this pay tuition and summer salaries and all sorts of things that are used to support what goes on in a research university and allows you to take number theory for one fifth the upfront cost.
Maybe this is a waste of taxpayer money (some things are), but on its face it doesn't seem to me that this is. It seems to me to be quite sophisticated, like number theory, although its immediate use is doubtful. And no different than supporting art, music, athletics and all the rest of what goes on in a university. And it provides educational support for 4 people. A humanities professor gets as much per year as a grant like this I'm willing to bet (maybe more; this project was probably mainly in salaries, since the hardware was developed elsewhere). Should we just abolish the music department and give the money to . . . what?
@Revere: I will concede that my number theory class costs more than that. Although you were unfair in your calculation. Remember it's my number theory class we're talking about. You wrote:
"So let's say $30K for the year (3 semesters), annualized."
So what? I only go through one class of number theory, not 3. Why not compare 3 number theory classes with 3 laundry robots then?
For the sake of argument, suppose it actually costs as much as the robot (which is impossible, regardless of what costs you add). My main argument with your example of number theory is about potential. Say the gov. spends money on my number theory class. Short term gain for society: 0. However, say I make a significant contribution to scientific field X. Then, in the long term, paying for my number theory class was very rewarding to society. Say that instead of spending money on my number theory class, the gov. spends money on building laundry robots. Ok, so now, in the short term, a few people in university get to play around with a robot folding clothes. What about the long term? UC Berkeley and America gain a critical edge in automated cloth folding technology? Again, there is useful research in robotics that really leads somewhere and acts as the foundation for further useful research in the field -- but this is not an example of that research.
Now, you wrote:
"And no different than supporting art, music, athletics and all the rest of what goes on in a university."
>>This I am willing to concede. These programs appear to me quite useless. Athletics is negotiable since it keeps people healthy.
"And it provides educational support for 4 people."
>>Ok, well there's 20 of us in number theory...
"A humanities professor gets as much per year as a grant like this I'm willing to bet (maybe more; this project was probably mainly in salaries, since the hardware was developed elsewhere)"
>>Humanities and the social sciences help students think critically about the world they live in. Much more useful than having a robot who folds stuff for you. So, the value is greater.
"Should we just abolish the music department and give the money to . . . what? "
>>Sure. Abolish it and give the money to cancer research. People can take private classes if they feel like playing the piano. But they can't play the piano if they get brain cancer.
What's so hard about a fitted sheet? I follow Martha's instructions and can usually achieve a somewhat flat and rectangular shape. The trick is to fold based on where the sheet meets the top corner of the bed, not the elastic. The first step is to put your hands into the sheet inside-out so they are in the corners. Then bring your hands together, and take the elastic on one side and bring it over the other hand. Repeat with the other side. Then lay out flat on the bed, and fold up nicely.
Oh he's so lovely, so sweet. He can recognize the sizes of the towels and put them in the right places. Anyway it takes quite a long time for him to do it. However, he folds the towels quite quickly (of course more slowly than a human, but is acceptable with a robot). It will be great if he can also deal with big size towels.