Pharyngula

Posted by: Glen Davidson | August 21, 2010 11:31 AM

He doesn't understand how complex and information-rich the development process is, and certainly has no clue as to how to mimic or to compensate for the highly complex biological and environmental factors affecting brain development.

Who knows how much we'll understand in 20 years? It took a long time for us to even get to a fairly good understanding of a mere protein folding, while the brain is almost infinitely more complex than is protein folding.

Glen Davidson

Posted by: theswede | August 21, 2010 11:43 AM

What I find most astonishing about these predictions is that today, on computer systems of a power which was unimaginable a mere two, three decades ago, we're using a system which first saw the light of day in 1968.

And we haven't gotten any further at all since then. Skype? They had a functional equivalent. A Window based OS? Yep. Co-operative work on the same document in real time, something we still have problems with? Yep.

With all this exponential growth, we're finally reaching where we were 40+ years ago. And this is why it's so incredibly naive to assume that just because we get more, faster transistors we suddenly will be able to do such amazing things without learning how to actually do these amazing things.

For the interested, the demo of the system is here:

http://sloan.stanford.edu/mousesite/1968Demo.html

Posted by: JD | August 21, 2010 11:46 AM

...unless Henry Markram is working on a keyboard solo to accompany Kurzweil's exponential "Jump."

Posted by: Graves | August 21, 2010 11:50 AM

I agree completely that Kurzweil is off his rocker. I am a bit confused by what you say about pyramidal neurons, though. You say that the differences between them that make each unique are not specified in the genome. I imagine you are emphasizing that they emerge through the unfolding of development, but that really does not mean they aren't specified in the genome. Just because physical rules and the environment are major factors in the end organism does not mean traits aren't specified in the genome. If we take that stringent stance very little at all will be specified by the genome, because we know strong environmental perturbations will change essentially all traits. Maybe I am misreading you, though.

Posted by: Steven Dunlap | August 21, 2010 11:59 AM

Kurzweil (and lots of others) in going ga ga over technological developments gives technology some sort of magical hoo-doo voodoo sort of capabilities that it clearly does not have. The results of hi-tech devices requires a human brain to interpret them and the results do not necessarily give us as much information as we might like to think.

For a concrete example, researchers are only recently subjecting functional MRI research to falsification. Researchers at Dartmouth put a "test subject" in an MRI then showed the "test subject" a series of photographs which have been used in a functional MRI test to determine the brain's reactions to emotionally charged vs. emotionally neutral images. The "test subject's" brain lit up for each picture. Who was the "test subject," you might ask? A dead salmon.

Posted by: Andyo | August 21, 2010 12:05 PM

Coming from a layman's perspective, I wonder how freaking stubborn (dare I say, fundamentalist) do all these Kurzweil fans have to be, when pretty much everyone who knows biology and the brain are telling them they're wrong. And just as the fundamentalists, they bring up one or two "biology" names that according to them support their theories and think they're vindicated.

I'm pretty sure the Galileo gambit also popped up there, and in the later posts there was some dumbass doing old quotes from "skeptics" about computers which were proven wrong with time.

Posted by: setad7 | August 21, 2010 12:17 PM

This was great! I'd love to see you go after the Cryonics madness, that seem to be all the rage in trans-humanist circles.

Posted by: Zzarchov | August 21, 2010 12:20 PM

Well I guess it depends on how many people you are willing to murder and dice up for testing purposes.

To go with your "number of transistors" analogy (and I don't understand biology, so I am just going off of technology and this may not apply AT ALL to biologgy). If you know the number of transistors and how they are arranged in all its complexity (even without understanding why) you could build a replica machine. You could then take the information from another machine (with halo reach) and clone it over, having two copies.

I imagine to get that information from a brain would require quite a bit of murdering and trial and error until you get it right.

I hope he doesn't plan on forming unit 731b or working for Dr.Mengele Jr.

Posted by: ralphgentile3 | August 21, 2010 12:24 PM

I predict mail will be delivered to my house today at XX:XX o'clock, and the Universe will cease to exist sometime in the next gajillion years.

Posted by: Pareidolius | August 21, 2010 12:27 PM

In the interest of accuracy (that, and I don't want to get dogpiled for my Criswell paraphrasing), here is the actual quote which is even more germane . . .

Greetings, my friend. We are all interested in the future, for that is where you and I are going . . .

Pure Plan 9.

Posted by: Zeno | August 21, 2010 12:38 PM

John Horton Conway and his collaborators demonstrated decades ago that amazing complexity can arise from an extremely parsimonious set of simple rules. Martin Gardner brought this to our attention when he published his first Scientific American column on Conway's game of Life, the two-state cellular automaton that was soon shown to be universal (capable of instantiating a universal Turing machine).

Such power indicates that Conway's Life will eventually evolve to take over the universe. It's only a matter of time.

Posted by: raven | August 21, 2010 12:47 PM

I agree completely that Kurzweil is off his rocker>

Maybe not. Kurzweil's problem seem to be that he is afraid of dying or wants to live much longer.

Don't blame him, some sort of life extension sounds good to me.

People deal with it in various ways. Magical wishful thinking known as religion is common. Kurzweil is putting his faith so to speak in technology.

Something like that may even be possible someday but his timeline seems to be off. Everything takes longer and costs more than people expect as we've seen with our two current wars. Cheney claimed that the Iraq fiasco would pay for itself because the Iraqi's would be so grateful to the allies they would pump lots of oil.

Posted by: robert.rayborn | August 21, 2010 12:59 PM

PZ and others,

Kurzweil aside, I would be very interested in what you have to say about Jeff Hawkins models of the neocortex. I am a CS major, not a biologist, so I would appreciate your opinion. If nothing else, his models do learn quite effectively (better than traditional machine learning techniques), although this by no means proves that he is correct in his assumptions.

Posted by: Miles670 | August 21, 2010 1:03 PM

I do enjoy these responses. Some parts i had to read twice to understand, but still.

Posted by: James Quin | August 21, 2010 1:06 PM

Mr. Myers,

Do you think AI scientists will ever develop an AGI equivalent to a human intellect or beyond?

Sincerely,
James Quin

Posted by: https://me.yahoo.com/a/S5prevQVie1EfQXER5NMFxrrHTRG#fff61 | August 21, 2010 1:10 PM

Dear PZ,

While I do think that Kurzweil's optimism is on very shaky grounds, your criticism is on as shaky grounds.

Whatever it is that you mean here by "baseline value" - he has provided the exponential curve http://en.wikipedia.org/wiki/File:PPTSuperComputersPRINT.jpg

Whether or not you agree that the exponential is going to hold or with his estimate of computational power required to get a working brain simulation is a different issue. Yes, as others pointed out the estimate ignores the chemical soup and only takes into account (basic) neuron interaction, but they fail to show that the soup is indeed necessary for simulating intelligence. Given all the progress in various types of ANN's, it's a reasonable null hypothesis to assume that none of that stuff has a major contribution.

The analogy to Halo Reach is completely befuddling me. It's so similar to some creationist "analogies" that it's really weird coming from you. Ray's argument was about initial complexity of the brain neuron pattern, the rest is learning (again, this ignores random individual variations, which are unlikely to have a major contribution, as well as the soup). So yeah, the blank slate of an Intel chip has little complexity, and Halo Reach is very complex, but what in gods' name does this have to do with how our programs in the brain are learned? Take the blank slate, insert it in a body and let it go. Quite unfortunately Xbox games are not produced in such a fashion.

To emphasize one more time - yes, his estimates rely on a lot of assumptions - the main ones being that the exponential growth will continue long enough and that minor variations and weak interactions are irrelevant for the big picture. But representing it as incoherent as you seem to imply is incorrect.

Best,
Ed

Posted by: daedalus4u | August 21, 2010 1:17 PM

For those who keep talking about the "unimaginable" progress that has been made in computing; it was never unimaginable.

Turing showed that there were limits to what was computable.

Has anything that Turing showed was not computable been computed using "modern" computers? How long will Moore's Law have to run before what Turing imagined could not be computed will be computable?

Posted by: mitsu.hadeishi | August 21, 2010 1:26 PM

Okay, let's try to get something clear here:

"I think I understand it better than Kurzweil. If we have a seed of information that initiates a process, followed by many activities and interactions that add progressively more information to the process, you can't use information theory to measure the amount of information in the seed and then announce that you've put an upper bound on the amount of complexity in the process."

Again, the point, as I understood Kurzweil, was simply trying to get an upper bound on the Kolmogorov complexity of the algorithm needed to generate the *brain* --- which is totally different from the Kolmogorov complexity of what one might loosely call the "software that you'd run on the brain", or human behavior. It's obviously correct that what the brain is capable of doing (in terms of its behavior) is far, far more complex than the algorithm that might be necessary to simply generate the brain structure itself.

So yes, of course you are right that generating an Intel CPU is not the same as generating Halo or OS X or whatever else --- they're quite separate things. However, I don't see that Kurzweil is making the claim that once you had a system that was as complex as the brain you would then have AI. My belief is that he may be right that we could potentially build systems with the same computational power as a brain within our lifetimes --- but he's probably very wrong that we'll get anything like human-level AI in our lifetimes, though I can't say for certain he's wrong.

I mean, think about what we do with computers today, despite the fact that they're immensely faster and more powerful than the computers of 10, 15, 20 years ago: we still mostly type into them and read text. It's hardly the massive leap forward one would hope for, because the advances in software are far behind advances in hardware. So yes, you're right I think in your criticism but this doesn't invalidate the simple claim that the upper bound of the algorithmic entropy of the brain may be roughly the size of the genome. That argument can be correct and yet not say that much about how to generate AI.

Posted by: nejishiki | August 21, 2010 1:31 PM

#34
if you're talking about Turing's work "On computable numbers, with an application to the Entscheidungsproblem," i.e. his version of Godel's theorem that introduced the Turing Machine, then no, computers still can't compute some things. This is a permanent, logical hurdle, not a practical one, and it should apply to the brain as well: the brain will face the same kinds of limits as a Turing Machine. There is no indication that this hurdle will be overcome - or, however, that it is relevant to the study of the brain.

Posted by: Marco | August 21, 2010 1:34 PM

According to the Newsweek review, Byrne maintains that we must be "nice to water", because if we are not, then water turns nasty on us, or something like that.

Apparently, this is all the more urgent and necessary because “the inside of your head is 80 percent water”.

I understand the depth of her concern, because, probably, the inside of her head and that of a great majority of her readers is 100% water as well.

Posted by: Marco | August 21, 2010 1:37 PM

Oops, I posted my message (#40) in the wrong thread. My apologies.

Posted by: Nerd of Redhead, OM | August 21, 2010 1:46 PM

Kurzweil and I both contend that the information content of the "hardware" is coded in the genome.

Posted by: https://me.yahoo.com/a/S5prevQVie1EfQXER5NMFxrrHTRG#fff61 | August 21, 2010 1:53 PM

#44: adding randomness from the environment does not add information

Posted by: Form&Function | August 21, 2010 2:03 PM

Greylander, I'm not sure I understand your correction to PZ's analogy. In terms of the brain, what are "fundamental structures and processes"? Is a cell a fundamental structure, or is it wiring? Because even a cell isn't "coded for in the genome". If the "wiring" of the brain is software, what is neural activity?

Posted by: Stephen Wells | August 21, 2010 2:11 PM

The genome on its own doesn't do anything. It has to be transcribed. What gets transcribed depends on the current state of the cell.

Now, Greylander, if you can establish that the state of a just-fertilised egg depends solely on the genome contained in that egg- then you might have a point.

Otherwise, no. There is information in the system that isn't in the genome, ergo the information content of the genome doesn't limit the information in the system, and we're done.

Whoever just claimed @46 that "adding randomness from the environment doesn't add information" desparately needs some remedial education.

Posted by: mitsu.hadeishi | August 21, 2010 2:16 PM

I really don't understand the point of all the vitriol in this thread. Admittedly I didn't bother reading the other massive thread of comments but just skimming over some of the comments here it just seems totally out of place.

I believe and I argued in the other topic that PZ misread Kurzweil's argument and I think he's still conflating things a bit in this post (though less so). PZ, above, is throwing around some imprecise terms, such as "information content" which is really a totally different, orthogonal notion than algorithmic entropy of Kolmogorov complexity which is the thing Kurzweil is referring to. The picture of "information content getting added" is really not directly relevant to the Kolmogorov argument regarding the algorithmic entropy of the basic structure of the brain. As an algorithm unfolds it may well generate all sorts of complex structures and behavior but this doesn't really change the initial conditions.

But again, PZ is *correct* that merely generating a brain, even if Kurzweil and Sejnowski are correct that the genome is a rough upper bound of the algorithmic complexity needed to generate the brain, would not, of course, generate human-level AI. That would of course require actually running this "brain" in interaction with its environment for a long time, so it can learn and so forth. What Kurzweil is clearly referring to, however, is just the raw brain itself, untrained and without having learned anything via interaction with its environment.

There's another thing left out of Kurzweil's argument, which is that merely determining an upper bound of the algorithmic complexity of an algorithm needed to generate the brain doesn't say much about FINDING an algorithm that could generate brain-level behavior. Our brain in the result of billions of years of evolution. Clearly, an algorithmic "brain" would be totally different in structure, but even if one could in principle find a algorithm capable of generating a brain-like entity, this doesn't say much at all about the ability to actually find such an algorithm. It could take millenia or millions or billions of years to find it. Not at all easy, even if Kurzweil's argument is correct. The exponential increase in hardware power doesn't eclipse the difficulty in searching the even more vast exponential space of all possible algorithms.

I really think these sorts of debates ought not be held in an atmosphere of vitriol. That's for politics (which seems to be increasingly based on demagoguery). This should be a rational discussion, not some ridiculous flame war.

Posted by: Nerd of Redhead, OM | August 21, 2010 2:19 PM

I really think these sorts of debates ought not be held in an atmosphere of vitriol.

Posted by: theswede | August 21, 2010 2:24 PM

@36: Again, the point, as I understood Kurzweil, was simply trying to get an upper bound on the Kolmogorov complexity of the algorithm needed to generate the *brain*

And again, doing it this way is BOGUS.

To analogize; if RK claimed that the upper bound of a known to be immensely complex function is the compressed size of the rather small argument passed into it, would you consider that a persuasive argument? Would it be some form of argument worth putting forth, even?

The *brain* is generated by the environment the genome operates inside, starting with the cygote, continuing through the womb and the growing body, and taking its final steps inside the functional body as it feeds on sensory input.

Of the *brain*. Not the mind. Not the "software" which "runs" on it. The brain itself is built this way (although I have simplified the process to the point of incorrectness).

This is why any argument that the complexity of building a *brain* can be found by compressing the genome is sheer ignorance at best, stupidity at worst.

Posted by: https://me.yahoo.com/a/O.jullMj0I2VvJaxMMVeNKSfOPf73voLSxJAe9PdlOWwi8Y-#258ec | August 21, 2010 2:28 PM

our brains structure is very plastic and changes over time and activity as I understand so that the brain as "designed" by the genome is not the same at birth or at other times subsequent. The development depends of stimulation and learning which seem to be very much time age dependent.
The interconnections of the neurons can and do change over time. I think one of the mistakes that scientists and other people who are not trained in biology make is thinking of things and not processes. Life is not a thing. A thing is an abstraction that helps facilitate thought.
If and when someone makes a "electronic replica" of a brain it will be just as much dead meat as any brain in the butcher shop and just as useful .. unless we understand and replicate the plasticity of the process of brain.

see
http://en.wikipedia.org/wiki/The_Treachery_of_Images

the "thing" as brain AI is a representation of a something that is dependent on the material that is made of and all of that but is experienced and acts as a process

Posted by: https://me.yahoo.com/a/S5prevQVie1EfQXER5NMFxrrHTRG#fff61 | August 21, 2010 2:34 PM

Whoever just claimed @46 that "adding randomness from the environment doesn't add information" desparately needs some remedial education.

Do provide that education if you will. I'm giddy with excitement to learn how adding randomness (shall I say noise, to make it easier for comprehension?) is going to add information. Maybe you're thinking of describing the specific random state, which does indeed require more information, but guess what - it's far easier to just put a random generator instead (unless for some unknown reason you want to copy the specific random state - pretty irrelevant for discussion of creating a blank brain).

Posted by: Greylander | August 21, 2010 2:34 PM

@ #48 Form&Function:

Greylander, I'm not sure I understand your correction to PZ's analogy. In terms of the brain, what are "fundamental structures and processes"? Is a cell a fundamental structure, or is it wiring? Because even a cell isn't "coded for in the genome". If the "wiring" of the brain is software, what is neural activity?

Thanks for asking good honest question(s). Neural activity is *also* software -- much faster software, if you like to think of it that way.

If you mean that "any particular neuron, of any particular type at any particular [x,y,z] coordinate was not directly coded for in the genome", that is true. The genome is not a blueprint, it does not include a listing of each individual cell and the coordinates for that cell in the body. But this does not alter the information theoretic argument one iota. No such claim is made.

Have you ever looked at a fractal image...? intricate, beautiful, complex... apparently, dare I say, information rich. Yet that fractal image, with millions of pixels -- millions of byts of data -- is generated by a simple, tiny program only a few bytes long. Which means the whole fractal image actually only has a few bytes of information content. Yet the fact than any particular pixel at any particular location has any particular color is not in any direct way "coded for in the fractal generating program".

Is the neuron coded for in the genome? Not directly. Does the genome govern the process by which a certain rough patterns and distributions of neurons is grow in the brain? Yes. Is there randomness in the exact locations of the neurons not coded for in the genome? Yes, but there is no information in randomness -- because any other minor slight random re-arrangement would have been fine. So the general presence and overall pattern of neurons is coded for in the genome, as are the rules which govern how the wiring forms -- how each individual neuron adapts to it's own local environment of other neurons.

So "fundamental structures and processes" would include the overall pattern of neurons and the rules governing how they grow & re-wire synapses. It would not include minor random variations in the exact positioning of the neurons (no information content there anyway). It does not include the detailed wiring, which, as I said, is "software" or "data" (same thing, really) along with the activity patterns, also "software" or "data".

All the above is a rough sketch. There is more to it than that. And I'm sure someone is going to pick on some detail I have not addressed and say "what about blah blah blah"... despite my newly acquired reputation for epic posts, I can't write a freakin' book here.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawmVT1LBhwmO9ej9LNg7a5e9d-AVJ8ezfmE | August 21, 2010 2:42 PM

Counting the number of transistors in a CPU would give you a measure (upper bound) on the information content of the hardware only

Put down the shovel and walk away. You're apparently not computer-literate enough to be engaging in this discussion, either.

There's this stuff called micro-code - CPU instruction counts mean exactly - nothing. The brain/computer analogy is very interesting nowadays because of the malleability that has been built into silicon (silicon emulating hardware by running software) There are analogs to how brains appear to work but (as a computer guy, not a biologist) I'd treat them as thought-provoking only and not illuminating.

Posted by: theswede | August 21, 2010 2:42 PM

Of course as the zygote develops all sorts of environmental interactions occur.

No shit Sherlock. That's what PZ has been saying all along, and you (and others) have pretended isn't so. Glad you're finally on the train.

I think it's at least plausible to suggest that the amount of additional algorithmic entropy contributed by the environment of the womb, etc., is relatively small.

On what grounds?

Obviously it leaves out a lot of detail but even if it's off by an order of magnitude or so it is nevertheless a plausible argument.

On what grounds?

I'm simply saying that the question of whether one could actually build a simulator is separate

Of course. That is a legitimate line of inquiry, unlike talking about the Kolgorov complexity of the genome.

Or do you find it illuminating to talk about the Kolgorov complexity of argument to functions in general? You never answered that. To me, such a line of inquiry is nonsense. Please stun me with your insight.

Posted by: eleusis | August 21, 2010 2:47 PM

And no, neural activity is not software any more than it is the electricity going through the transistors (both of which "sound" like good analogies on first pass). As long as you think like that, you will be constrained by the falsehood of your paradigm.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawmVT1LBhwmO9ej9LNg7a5e9d-AVJ8ezfmE | August 21, 2010 2:50 PM

Yet that fractal image, with millions of pixels -- millions of byts of data -- is generated by a simple, tiny program only a few bytes long. Which means the whole fractal image actually only has a few bytes of information content.

...yet it also depends on the vastly complex math library running in the processor - a blurred combination of software (the math library) and hardware (the math processing units in the CPU) and software (the logic of the math instructions in the CPU) and, and, and...

This discussion is eerily similar to the discussion about interstellar exploration. The optimists say "we'll just take frozen eggs and sperms and an artificial womb!" but the pessimists know that you also need bacteria to populate the intestines of the newborn, and a hugely collected and poorly understood myriad of other complicated, gooey, biological things. You wind up needing to bring damn near the whole planet along... Put differently, the programmers who wrote libc.a and gnu/cc are part of the "system" that makes the computer plot fractals.

Cretards like to throw around the 2nd law of thermodynamics as a "problem" for evolution but, of course, anyone who understands knows that Earth isn't a closed system. Your computer is not a closed system, either, at the point where you code in a few lines of fractal source code and tell it to run; there are dependencies vastly deeper and more complex than just the source code. That is the essence of Kurtzweil's mistake.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawmVT1LBhwmO9ej9LNg7a5e9d-AVJ8ezfmE | August 21, 2010 2:55 PM

(also, WTF is Kurzweil going on about 'lossless compression' for? if it's lossless, it's irrelevant since it's just trading storage space for time and complexity.)

Posted by: SaintStephen | August 21, 2010 3:00 PM

If you want to recreate a generic human brain, it won't work if you just make every pyramidal neuron exactly identical; there have to be spatial differences and differences in connectivity.

You don't know, Professor Myers. Period. You've fallen prey to your own nay-saying.

So please stick to posting pictures of flowers that resemble female genitalia, and quit tearing down good-hearted technological optimists like Ray Kurzweil. Your recent attacks serve no purpose as far as I can see (except to further your own ends, whatever they might be). Dr. Kurzweil is not an enemy of New Atheism, or rational thinking, nor does he advocate religion or supernaturalism. He tirelessly advocates SCIENCE and TECHNOLOGY, and he doesn't need flower and vegetable porn to do it.

What gotten into your corn flakes, anyway. Your negativity and pessimism on this fascinating topic is most unseemly. Yes, it's going to be a "long hard slog." Thanks very much for that piece of sage wisdom.

Yawn. Where's the door. I need some California sunshine. This cold Minnesota weather is chilling me to the bones.

Posted by: nejishiki | August 21, 2010 3:09 PM

#59
Are you taking epigenetics and alternative splicing into account here? This is a level of complexity that is not included in the raw sequence of the genome, but has far reaching effects. Kurzweil, in his response, says the there is a 'small amount of information' in epigenetic mechanisms; one wonders where the math is - or even what it would look like.
Probably wouldn't fit on the back of his envelope.

If Kurzweil's entire contribution to this controversy is a rough calculation - perhaps off by an order of magnitude or more - and a promise that technology will eventually deliver the goods, then I agree: PZ shouldn't have said what he said, because it wasn't worth commenting on in the first place.

Posted by: Nerd of Redhead, OM | August 21, 2010 3:19 PM

This cold Minnesota weather is chilling me to the bones.

Posted by: Stephen Wells | August 21, 2010 3:23 PM

Something to consider: the fact that the genome is transcribed by polymerases to produce RNA, and some RNAs are translated by the ribosome to produce proteins, is not in "the information content of the genome" that Kurzweil is playing with.

Sure, once you _know_ about transcription and translation, and you have some polymerases and ribosomes, and a supply of tRNAs and some ATP and all that other bio goop... then you can see that some of what's being produced is more transcription and translation machinery. But there's nothing in the text string of ACGTAGCTCTTCCGATATATA... that would tell you that.

A fact for Greylander to ponder on, since he thinks the uterine environment is no biggy, information-wise: at least half of fertilised human eggs fail to implant and never get to develop. The right environment is something which after billions of years of evolution our bodies routinely get wrong. And don't even get me started on things like this: http://en.wikipedia.org/wiki/Hemolytic_disease_of_the_newborn

The fertilised egg with its brand new genome gets its start with a bunch of inherited protein machinery all buzzing away. So Kurzweil needs at the very least to consider the information content of a complete description of an entire cell and of an environment it can develop in. And of the laws of nature. And of an algorithm that can simulate all of the above.

I think we can all agree that the information content of the brain is probably bounded by the information content of the genome plus a perfectly accurate simulation of the physical universe. That's really helpful.

Posted by: Greylander | August 21, 2010 3:36 PM

@ #50 Stephen Wells:

The genome on its own doesn't do anything. It has to be transcribed. What gets transcribed depends on the current state of the cell.

Now, Greylander, if you can establish that the state of a just-fertilised egg depends solely on the genome contained in that egg- then you might have a point.

Otherwise, no. There is information in the system that isn't in the genome, ergo the information content of the genome doesn't limit the information in the system, and we're done.

I've been over this in detail in the first thread. I'm done repeating myself (mostly). I you weren't bothered to respond there, why would I expect you to here?

Whoever just claimed @46 that "adding randomness from the environment doesn't add information" desparately needs some remedial education.

In pure information theoretic terms, it is true that random bits contain maximal information. However if the bits are truly random rand than only seeming random (as in a maximally compressed file) then there is no useful information. You can always remove randomness from your encoding of a system and put it back later during reconstruction if the randomness performs some useful purposes -- but you don't need to include a big chunk of random data in your encoding, since being random, by definition the system still does essentially the same things regardless of what random information you include, and will just be different in the details.

Randomness from the environment is certainly not part of any description of the fundamental structures and processes -- except perhaps to not places where randomness is injected into the system and how much.

Posted by: F | August 21, 2010 3:38 PM

Forget this Kurzweil noise - how about some Kurt Weill instead?
http://www.youtube.com/watch?v=_QXJ3OXWaOY

Or perhaps one might prefer another version
http://www.youtube.com/watch?v=hLIrS5dtTZI

Posted by: bobh | August 21, 2010 3:40 PM

Reading K's rebuttal. and a couple of the comments, it seems that K confuses the brain's interaction with its environment and the developmental process.

Posted by: llanitedave | August 21, 2010 3:52 PM

What is responsible for the difference between the human brain and that of a chimpanzee? Is it environmental or genetic?

If you placed a fertilized chimpanzee embryo inside a human uterus would you end up with a human infant?

Of course not. It's really irrelevant to talk about all the epigenetic factors that influence brain development, because they are tightly constrained and consistent between one individual and the next. Otherwise, every newborn brain would be completely unpredictable and there'd be no definition of "human" at all.

In an indirect sense, the womb environment is also ultimately genetic, as it's the genes that specify the pregnancy hormones that will be produced, their timing, the proximity of other tissues that interact with the new brain cells, etc. Environmental factors that are totally nongenetic, such as nutrition levels, chemical or drug exposure, and potential trauma, are usually a very minor aspect of the development process. Otherwise, very few infants would be born "normal".

The point is that, as PZ states, to accurately model brain development, you need to take these other factors into consideration, but why that should turn a 20-year project into a 50-year project, or more, seems obscure. It's not like these other factors are being neglected. And for the most part, they are constants that vary little even between closely related species. It's not voodoo. Ultimately, it's still going to be the genetics that specifies the shaping of the product.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawmVT1LBhwmO9ej9LNg7a5e9d-AVJ8ezfmE | August 21, 2010 4:02 PM

Stephen Wells writes:
I think we can all agree that the information content of the brain is probably bounded by the information content of the genome plus a perfectly accurate simulation of the physical universe.

That's the simulationist view. After all, if Kurzweil is right and we soon have enough computronium to completely simulate a universe from the big bang onward, eventually humans will evolve and we can watch over their shoulders as they post messages like this one to blogs. Like the fractal analogy: all you need is enough storage space plus the laws of physics, and a fast enough CPU, now that we know the universe is not infinite, it's "doable" Of course there's already a 'computer' you can run a simulation the size of the universe on - it's called "the universe"

Excuse me - I've got to go reboot my simulation; the creobot infestation on Earth #4723 is really annoying me.

Posted by: msironen | August 21, 2010 4:12 PM

After all, if Kurzweil is right and we soon have enough computronium to completely simulate a universe from the big bang onward,

I don't know if this is being facetious or just stupid. The minimal simulator that can simulate the universe wholly IS the universe itself.

It's like claiming that predicting your own decisions is somehow different from actually making them.

Posted by: Greylander | August 21, 2010 4:21 PM

@#65 Stagyar zil Doggo:

Greylander and others:

In terms of the analogy between the brain and an Intel CPU, knowing the (human) genome at best gives you something like a Verilog/VHDL file for a System on Chip which includes say a CPU core, peripheral devices, as well as a variety of MEMS sensors (say accelerometers, clocks, thermometers, micro gyroscopes, and a few lasers) on the chip. You (in terms of this analogy) still know nothing of semiconductor physics, photolithography, material processing, or the basic physics of all but one or two of the MEMS devices. You've more or less figured out what resistors and capacitors are for, but are not entirely sure about transistors. Do you really think the Kolmogorov (or any other) Complexity of the Verilog/VHDL file is in any way informative of the difficulty of actually realizing the SoC in question? Or of usefully simulating it (along with the MEMS devices of course) on another computer?

Yes. But let me be specific. Kurzweil side does not propose we will magically figure out all that additional stuff (in your analogy: how transistors work, laws of physics or whatever). But the Kolmogorov complexity DOES give us a very good indication of the size of an actual executable program that we will need after the necessary stuff is learn. But I have to continue your analogy.

(1) Some kind of automated fabrication facility is avaiable to us, so we actually can build the systems and do experiements on them, look at the parts with microscopes, etc.

(2) Suppose the system coded in the Verilog file is modular/hierarchical, and we are interested in simulating not every detail but some higher level modules in one section. (analogy with brain: we are not necessarily interested in simulating what ever individual [copy of a] molecule does, but we will probably be interested in how aggregate concentrations or quantities of different chemicals interact with each other (how fast does A react with B to produce C and D might be important, along with how E modulates that rate... but having saiad that, much of the metabolic pathways themselves are probably "lower level" than we are interested in and we will be looking at aggregate results of medium to large networks of reactions, along with the more obvious "wiring" of synapses and such).

(3) Now in your analogy, what the Kurzweil side is proposing is that we will reverse engineer the functioning of the high-level modules we are interested in by poking and prodding and scanning and testing and all that.

(4) The Kolmogorov complexity of your verilog does provide an upper bound on the complexity of a simulation program (executable code) unles you want to assert that the "laws of physics" in your hypothetical world (or the details of how primitive components work, like transistors) include special information about the high-level functioning of the coded system.

Posted by: Greylander | August 21, 2010 4:26 PM

[just noticed another blockquote mess... repeat of #82]

@ #50 Stephen Wells:

The genome on its own doesn't do anything. It has to be transcribed. What gets transcribed depends on the current state of the cell.

Now, Greylander, if you can establish that the state of a just-fertilised egg depends solely on the genome contained in that egg- then you might have a point.

Otherwise, no. There is information in the system that isn't in the genome, ergo the information content of the genome doesn't limit the information in the system, and we're done.

I've been over this in detail in the first thread. I'm done repeating myself (mostly). I you weren't bothered to respond there, why would I expect you to here?

Whoever just claimed @46 that "adding randomness from the environment doesn't add information" desparately needs some remedial education.

In pure information theoretic terms, it is true that random bits contain maximal information. However if the bits are truly random rand than only seeming random (as in a maximally compressed file) then there is no useful information. You can always remove randomness from your encoding of a system and put it back later during reconstruction if the randomness performs some useful purposes -- but you don't need to include a big chunk of random data in your encoding, since being random, by definition the system still does essentially the same things regardless of what random information you include, and will just be different in the details.

Randomness from the environment is certainly not part of any description of the fundamental structures and processes -- except perhaps to note places where randomness is injected into the system and how much.

Posted by: redacted | August 21, 2010 4:36 PM

What a computer scientist means by saying "the functionality of this organ is encoded in the genome" is more or less what a biologist means by saying "this organ evolved to perform a certain function". Any evolved functionality of the brain must be encoded in some location that's subject to evolutionary pressures that favor improved functionality of the brain. That's not only the human genome; human language evolves independently of the genome and ought to count too. But the number of places where this functionality can be encoded is pretty small. It can't be found in the natural environment unless there's a selection process acting on natural environments and favoring those that produce better human brains. The environment is not like Halo: Reach, it's like, well, a computer exposed to the elements. Cosmic rays will eventually produce the text of Hamlet in the computer's memory, but that doesn't happen on evolutionary time scales. It requires much, much longer than the current age of the universe.

Posted by: Greylander | August 21, 2010 4:49 PM

#69 ljdursi Author Profile

I call bullshit on Greylander, Ed, Kurtzwell, and the rest of the accolytes.

The graph that Ed showed?

Havent' seen the graph. Can't comment on it.

We *can* simulate [turbulance] it in particular cases; but eve there, we fail, because turbulence is a stochastic process,

Wrong. Turbulence is a chaotic process that can occur in purely deterministic processes. Unless you are simulating quantum effect, this is true of your simulations. Because it is chaotic, you must run many trials in order to make stochastic predictions about what will happen.

Any single run of your simulation is a realistic (up to computational error) simulation of a single instance of a turbulent flow process.

No one says (certainly not anything close to the 20 year mark) that we will simulate the brain to make predictions, for example, about what some bloke will be thinking/saying/doing 5 minutes from now. No one is remotely suggesting that (at the 20 year mark) we will have any way to "scan" the necessary initial conditions from a live brain (not even with a 'destructive' scan). The current discussion is not about all the "uploading" stuff.

It may surprise many in these the last two Kurzweil threads that I've never even read any Kurzweil beyond a few blog posts and maybe half an article a while back. I don't know exactly what/when he predicts about brain-uploading. I find that particular notion a bit dubious, but whether it ever becomes technically possible it will be preempted by other technological adaptations -- long before that point we will have so deeply integrated out ordinary neurons with artificial 'neurons' of some sort -- quite possible artificially engineered but mostly biologic neurons... that by the time all our original neurons completely die off, we won't even notice. I have no mystical speculations just what this means for continuity of consciousness.

so to see how it plays out, you really need to do many many realizations of the same simulation to see what happens. And the number of realizations you need to do to get a stochastically good sample *grows with the size of the simulation*. Increasing computer power does not automagically save you.

To put it mildly, human consciousness is more complicated than fluid motions.

And that's with todays computers. Let me tell you something about the exascale computers of 2020 -- no one has *any* idea how to program these beasts. We've been using the same techniques today as in the 1980s, and the wheels are about to fall off that bus entirely. As the computers get bigger, *even if you understand the physical model well enough to simulate it at that scale*, programming the computers gets exponentially (see what I did there?) harder as the computers get bigger. Why? Because if the computer goes from size N to 2N, the number of interactions between components goes from N to N^2.

I know that Kurtzwell really, really wants to be Max Headroom, because he thinks then he'd be immortal. (I don't understand this; a copy of him would be, but he'll still grow old and die.) But it's still not going to happen in 20 years. Sorry, dude.

Posted by: https://me.yahoo.com/a/IdqMjN5n2IB.CRlIrQvxEwkliQoG.51jOw--#077cd | August 21, 2010 4:52 PM

PZ, I have come across your blog several times when it's been linked to various aggregators, but I'm not a faithful reader. Likewise, I barely know who Kurzweil is.

In general, I think you are spot on with a lot of your prior posts. You are thoughtful, and have a keen mind. In this case, though, you are simply wrong.

I'm with Greylander on the fact that you fundamentally misunderstand information theory. The fact of the matter is that you can, with nothing but the DNA code, create a brain. You can argue that you need the translational and transcriptional machinery to decode it, but that, too, is included in the blueprint. When you can fully define a system using a circumscribed plan, then the number of bits of information in that plan place a theoretical maximum on the number of bits in the product.

You say that the complexity of evolutionary history of the animal must be taken into account. That's not true: one impressive bit about evolution is that all of the millions of years of experience get incorporated into the DNA, and passed on.

I fail to even understand your software vs. hardware vs. operating systems analogies. If you can build the product with the blueprint, it can't be more complex than the blueprint was, regardless of what inputs and outputs it takes once you create it.

This has little to do with when, exactly, we will understand the brain, but it is a legitimate calculation on his part.

Posted by: jmlingeman | August 21, 2010 4:54 PM

Note: I am but a lowly CS grad student specializing in Machine Learning, so take what I say worth a grain of said.

The main problem I have with Kurzweil's assertion that the brain can be accurately modeled by a computer comes down to that even if given unlimitedly powerful hardware, we have no idea how to program a system like that. We have no idea how to program the parts of it. Strong AI has been a dead field since the mid-70s, with no signs of revival. Weak AI (single-task learning) has been growing as a field, but we're only trivially better at things like object recognition than we were a decade ago.

The field is moving quickly, but I don't see Machine Learning being able to emulate a full brain within my lifetime without several revolutionary leaps in quick succession.

Posted by: j-brisby | August 21, 2010 5:02 PM

Shame on you, Professor Myers. A man of your education should know the proper usages of 'its' and 'it's'.

Posted by: Nerd of Redhead, OM | August 21, 2010 5:04 PM

The fact of the matter is that you can, with nothing but the DNA code, create a brain.

Posted by: Greylander | August 21, 2010 5:09 PM

@ #97 BigMKnows:

[to:] #85

Right, a brain "before" interaction with the environment is like a computer fractal before processing by the CPU.

Wow. This is the kind of stupid reply that makes me want to reach through the internet and slap you silly. I guess I forgot to put a warning on that post, so I'll let you off the hook -- this time.

I never made such a comparison. If you are unable to distinguish the limits of an analogy used to illustrate a specific point and how it applies to the subject at hand, you are not qualified to participate.

Posted by: Greylander | August 21, 2010 5:14 PM

[wow... twice a quote screwup on same msg... now i'm being compulsive about fixing it... sigh.. sleep where art thou? ]

@ #97 BigMKnows:

Right, a brain "before" interaction with the environment is like a computer fractal before processing by the CPU.

Wow. This is the kind of stupid reply that makes me want to reach through the internet and slap you silly. I guess I forgot to put a warning on that post, so I'll let you off the hook -- this time.

I never made such a comparison. If you are unable to distinguish the limits of an analogy used to illustrate a specific point and how it applies to the subject at hand, you are not qualified to participate.

[ah finally]

Posted by: csreid | August 21, 2010 5:20 PM

I think Kurzweil is starting from a flawed platform. Allow me to play reverse-creationist for a moment.

Evolution does not produce simple things. The human eye is a good example: tons and tons of stuff, rigged together with layer after layer of post hoc additions to produce something that works well enough, most of the time. If we had looked at the eye when trying to design a camera - well, it wouldn't have worked so well.

Why would the brain be any different? If anything, it should just serve as a (terrible) proof-of-concept, showing that it is possible to create intelligence. I think it may be possible to simulate intelligence (maybe even relatively soon!) but certainly not the way Kurzweil is proposing.

Posted by: amphiox | August 21, 2010 5:23 PM

The genome is not a blueprint.

The genome is not a blueprint.

The genome is NOT a blueprint.

THE. GENOME. IS. NOT. A. BLUEPRINT.

Posted by: CJO | August 21, 2010 5:30 PM

The fact of the matter is that you can, with nothing but the DNA code, create a brain.

Um, no. The cellular machinery to express that code the DNA gets "for free" as well as the in utero environment.

You can argue that you need the translational and transcriptional machinery to decode it, but that, too, is included in the blueprint.

You are badly conceptually confused. The DNA (which isn't like a blueprint anyway) needs the expressed result of a prior course of development to be fully formed and already working. You're asking a blueprint to be the plans and the fully operational manufacturing infrastructure at the same time. You really need to understand just how wrong that is.

If you can build the product with the blueprint, it can't be more complex than the blueprint was, regardless of what inputs and outputs it takes once you create it.

Fine, but irrelevant, for the simple reason that DNA is emphatically not a blueprint, and is not even analagous to one. To the extent that analogies illuminate this question, the genome of an organism is much more like a recipe than a schematic. And a recipe assumes ingredients and a kitchen stocked with the equipment necessary to carry out the instructions.

You say that the complexity of evolutionary history of the animal must be taken into account. That's not true: one impressive bit about evolution is that all of the millions of years of experience get incorporated into the DNA, and passed on.

All those millions of years of experience get incorporated as generalizations and passed on. "The experience" is an aggregate of the experiences of billions and trillions of individuals, in populations, in a dynamic environment that includes as salient features the activities and tendencies to act of yet more quadrillions of individuals in billions of populations over millions of years. "All the millions of years of experience" at this level of granularity certainly could not possibly be encoded in the genome to be passed along, which you should know since you're so keen on information theory. No, what the genome encodes, in a sense, is minimal "descriptions" of some of the most salient gross regularities that the lineage has encountered through time, and that is all. Confound those expectations of gross regularities by altering the environment on just a few of the myriad parameters on offer, and the genome's description fails utterly at producing viable phenotypes for the new conditions and the lineage goes extinct.

Posted by: Sintesi | August 21, 2010 5:32 PM

I don't think information could possibly create a mind. So much of what we are is completely dependent on biochemical processes. If I speak to you smiling and joking, oozing warmth and regard or if I speak to you screaming, menacingly glaring and waving my finger in your face that is going to radically change the conversation even though the words spoken might well be entirely identical. How does information theory account for that? Such meaning strikes me more as a function of what the brain itself is made than patterns in the neuro-circuitry. Humans make decisions before they have even have a chance to think about them. Your body will tell you to be afraid, revulsed or accepting and more often than not your mind will follow those bodily decisions. That makes perfect sense in an organism that for millions of years functioned without a neo-cortex and had it slowly layered on over millions of more years. The mind is what it is and what it seems to us as possessors because of that incomprehensibly lengthy evolution. And that I suspect is inimitable due to the winding, eddying and blundering path that led to its present state. It's not like somewhere along the line we suddenly tapped into a platonic universe of intelligent thought. I want to say, "You know Ray, I don't think the mind is really 'out there'."

The other thing that gets me about Kurzweil is his reliance on exponential growth. Can he not see the possible parameters he's likely to butt up against? Properties like attention and focus, appropriate time allotment for meaningful interactions with fellow beings. You're going to need a godlike brain just to make your newly made transhumanist super brain function like a person. I'm reminded of that famous case study of the Russian gentleman who had a flawless memory which sounded great except he wound up paralyzed by the burden. Because when he remembered one thing he actually remembered everything and it all would come dumping out of him like pulling a thread on a sweater. There simply wasn't enough time to use the information in any meaningful way. In essence he had a simulacrum of the world in his head and unfortunately for him for there's only room for one world in this time scale. So what are the dimensional properties of the real world that won't accommodate the use of massive data streams? The imperfections and flaws in our present brains (e.g., lousy recall, myopic focus, etc...) might be the very properties that allow us to function. Imagine an engineer creating a computer that doesn't work properly in order for it to work properly. What's the point of having a super brain if you can't find the time or space to use it? These might well be actual barriers instead of mere hurdles. I dunno, maybe Ray can make a second computer brain so the first one will have something that can actually recognize and understand it.

Posted by: amphiox | August 21, 2010 5:43 PM

If you are unable to distinguish the limits of an analogy used to illustrate a specific point and how it applies to the subject at hand, you are not qualified to participate.

Oh, there are qualifications required to participate now?

And who decides what those qualifications are, Greylander? You?

Posted by: ljdursi | August 21, 2010 5:51 PM

amphiox sums it up very nicely in #115.

Let's take something the genome *does* encode - proteins. Let's take (say) one protein.

Here were, 3 years out from Kurzweil's "functional human brain simulation" as shown in Ed's graph, and we can't even simulate the folding of anything but the simplest *single* *proteins*.

First principle simulations of an entire single human cell are I have no idea how far away - but it ain't 3 years. Even though we can easily, trivially, store the human genome on a computer, we are nowhere near capable of simulating an single entire cell from first princples. There are 100 billion neurons or so in the human brain. This should put to rest the idea that because we know the genome, the brain quickly follows.

Now, you might argue that you don't ned to simulate the entire chemistry of each neuron to study, or even model, brain function. Fair enough. Lots of CS students learn to code up very quickly simple little "neural nets" inspired by neurons - in the same way lots of crappy movies are "inspired by real events" - which are capable of really interesting little tasks. That's great. But real neurons are vastly more complicated than that, and how much of that complexity you can wipe away in a reduced model and still get something which correctly models even simple mammalian brains is *not known*, and Kurzweil certainly doesn't know.

Posted by: co | August 21, 2010 6:02 PM

This is the so-called "magic" of exponential growth. It's a horizontal line for a long time and rapidly increases far into the future (imagine plotting the value of your one-penny investment over 20 years. The plot would only leave the x-axis toward the end).

Except that if you cut it off at 10 years, and rescale, it looks the same. If you cut it off at 5 years, and rescale, it looks the same. If you cut it off at 9 months (heh) and rescale, it looks the same. *THAT*'s the magic of Exp(x). It pops right out of its definition as the solution to dy/dx = k*x.

Posted by: ljdursi | August 21, 2010 6:05 PM

Continuing on with my series entitled "what ampiox said":

Water plays an absolutely, completely, essential role in protein folding. Where in the genome is described how to make water, exactly?

Posted by: poke | August 21, 2010 6:13 PM

I just noticed this hilarious quote from Kurzweil:

It is true that the information in the genome goes through a complex route to create a brain, but the information in the genome constrains the amount of information in the brain prior to the brain’s interaction with its environment.

Exactly when does the 'brain' not interact with its environment? It's already interacting with its environment before it's a brain. Its entire development, the process of becoming a brain, involves interacting with its environment! Kurzweil really has absolutely no idea what he's talking about. He appears to think there's a plan for the brain in the genome that doesn't even rely on the brain being a physical thing that can and can't connect in various ways, is subject to the laws of physics, etc. These are all ways a developing organism interacts with its environment and the concomitant process of interacting with its environment through use (experience) is not separable from them. It's a continuation of the same thing. This is why it's really misleading to speak of DNA being a program, or even data, let alone a blueprint.

Posted by: eleusis | August 21, 2010 6:20 PM

and by "functional group" I meant "functional domain"

Posted by: eleusis | August 21, 2010 6:26 PM

This point should be obvious. We already have the sequence of the genome, minus about 8% of heterochromatic material, which is the most gene-poor and irrelevant. If the genome sequence gave us the information that we needed to build a brain, we could already be building it. We are only beginning to get information on which markers predict susceptibility to some diseases (with some probability). But even that information is coming through the hard slog of empirical research.

So the question isn't how fast or cheap we can sequence over the next decades. It's how fast we can do real empirical work to link genetics to everything else we want to know, whether it's disease or the structure and function of the brain. You might claim that such work follows the law of accelerating returns, too, but if the doubling time is 20 years, we ain't gonna be there in 20 years.

Posted by: eleusis | August 21, 2010 6:38 PM

#125 poke: You hit the nail on the head over and over again.

Kurzweil's critical mistake, as PZ and many of us have been trying to point out, is that nothing in biology is a purely informational science (yet). We have a the genetic sequence, and as Venter recently said, it's mostly useless. There's still a lot of empirical work left to do.

Also, the link between transhumanists and IT professionals, and IT professionals and Aspies, has been made many times. :)

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkTzptVlEZQ2er6ymj1D_wskpB_1kIRN_o | August 21, 2010 6:50 PM

Benjamin Franklin says:

PZ can tuna fish, but Kurzweil can tune a piano.

Posted by: Tartessos | August 21, 2010 6:56 PM

By the way, people really should drop the "Where's my flying car?" nonsense, since we already have one.

http://www.terrafugia.com/

Find a new drum to beat on.

Posted by: theswede | August 21, 2010 6:56 PM

Tartessos @139, you're arguing that RK misspoke when he said "I said that we would be able to reverse-engineer the brain sufficiently to understand its basic principles of operation within two decades"?

Really?

You're arguing his position is not what he states it is, but something else we will learn from reading more? That the plain statements made are incorrect?

If you're right, RK is such a horrible conveyor of his ideas that reading his works would be a complete waste of time.

Posted by: chaseacross | August 21, 2010 7:06 PM

@Forbidden

They already made a jet pack! And flying cars! They both sucked, which is why no one uses them. Make a new benchmark for "the future." I for one dream of a world where I no longer have to seperate seeds from cotton by hand.

Posted by: MadScientist | August 21, 2010 7:12 PM

@theswede#4: Well, I wouldn't attempt to stuff *that* mouse into my pockets. But as far as progress with exploiting silicon goes (and it has little to do with studying the brain), I regularly use a chip that's the size of my little finger nail and it is the equivalent of an Apple2e (from early 1980s) but with numerous peripherals built in and it can run off a watch battery rather than the bulky power supply of the Apple2e. In my lab 20+ years ago we had an instrument operated by a computer the size of a desk; 5 years ago I bought a machine that's the equivalent of about 6,000 of those computers and it's only the size of a small suitcase - and even it's a dinosaur compared to a cheap desktop machine that you can buy today.

Posted by: Michael Kingsford Gray | August 21, 2010 7:12 PM

Ray worships at the altar of "Cargo Cult Science", as Feynman so pithily coined.

Posted by: jafafahots | August 21, 2010 7:41 PM

As usual, I must preface my comment by explaining that I dunno shit about shit. 8th grade education and all that.

But it does remind me of an experience from around 1998. Even then, relatively early years of Photoshop, I ran into confusion over chemical versus digital processes.

There was a client in a shop I worked briefly at that had a large format hi-res photo (way before digital cameras were useful) and they wanted the "blues bluer."

Photo was shot on film for a huge banner (billboard, essentially). No matter how hard he tried, the photographer could not get across that while he could "push the blue" by manipulating the chemical process to create a stronger blue, there was no way to avoid that having an effect on all the other colors. Chemical process. Chemicals do what chemicals do. The client wanted everything else unchanged, but just "bluer blues," and having passing knowledge of digital manipulation could not understand why one color couldn't simply be changed without a change to everything.

Dunno if this adds anything to the discussion (hell, lately I feel like I never add anything to any discussion) but it felt somehow similar to me.

DNA, cell structure, brain structure, neurons are not flash memory in other words. Chemicals.

Posted by: csreid | August 21, 2010 7:51 PM

I think Kurzweil is starting from a flawed platform. Allow me to play reverse-creationist for a moment.

Evolution does not produce simple things. The human eye is a good example: tons and tons of stuff, rigged together with layer after layer of post hoc additions to produce something that works well enough, most of the time. If we had looked at the eye when trying to design a camera - well, it wouldn't have worked so well.

Why would the brain be any different? If anything, it should just serve as a (terrible) proof-of-concept, showing that it is possible to create intelligence. I think it may be possible to simulate intelligence (maybe even relatively soon!) but certainly not the way Kurzweil is proposing.

Posted by: theswede | August 21, 2010 8:02 PM

I think what we're fundamentally debating here is whether it is possible, in principle, to grow a human fetus in a test tube (that is, without the inherent complexity of a mother's womb).

And without a cell. And without regulated nutrient flows. And without temperature controls. And without proper oxygenation. And without *anything* of note except a chunk of DNA.

The fundamental debate is whether it is possible, in practice, to grow a human fetus FROM DNA, and nothing else. Kurzweil is, indeed, implicitly assuming this is possible, and PZ overtly states it isn't.

Posted by: theswede | August 21, 2010 8:11 PM

Regarding the flying car; no, no such thing has been made. What has been made is an airplane which can be driven on roads to the airport and there folded out into an airplane, but that is a very different thing. It's still just an airplane. and you can't fly it from work downtown to your garage in suburbia like you could a flying car.

A flying car is a car when flying, and not an airplane when flying. That's the whole point of them, and why having one would not suck in the least.

Posted by: p | August 21, 2010 8:22 PM

And without a cell. And without regulated nutrient flows. And without temperature controls. And without proper oxygenation. And without *anything* of note except a chunk of DNA.

The end of the middle paragraph of my first post (#153) addressed all of this. But to go into a bit of detail: how many bytes of information does it take to convey a temperature suitable for incubation? I'd say 1. How many bytes of information does it take to specify a reasonable level of oxygenation? Perhaps 1, or 2. Regulated nutrient flows? Sure; specify a handful of carbohydrates, amino acids, and a rate for each to be provided to the growing fetus. How many bytes? 1000 maybe? 10000? Not much at all. So 3 of your complaints can be easily addressed with an amount of information that doesn't even raise our information quota to 51MB. Of course, compactly describing a viable cell is clearly harder than describing a temperature. But perhaps you can imagine an argument for the existence of such a description on your own?

Posted by: Nerd of Redhead, OM | August 21, 2010 8:31 PM

I really get pounced on by people who didn't read it.

Sure; specify a handful of carbohydrates, amino acids, and a rate for each to be provided to the growing fetus.

So 3 of your complaints can be easily addressed with an amount of information that doesn't even raise our information quota to 51MB

But perhaps you can imagine an argument for the existence of such a description on your own?

Posted by: sylverfyre | August 21, 2010 8:33 PM

Even more useless is the fact that his prediction of "we would be able to reverse-engineer the brain sufficiently to understand its basic principles of operation" is SO VAGUE that, in 20 years, he could make the claim that he was correct with his prediction assuming we make pretty much any advances in the field of neurobiology at all.

A key to making predictions that are always correct: Make them so vague and far enough in the future that you can't be wrong. Useless.

Posted by: Nerd of Redhead, OM | August 21, 2010 8:39 PM

It's very telling how any attempts to defend for example Kurzwiel's points, even by people who don't actually agree with with Kurzwiel on the big picture, are labeled as Kurzwiel's fanboys

Posted by: msironen | August 21, 2010 8:50 PM

"almost everything else just seems to display a willful ignorance that is best characterized by his most notorious anti-science opponents.

Any concrete examples? This is rather a brutal accusation, so I expect you have a long list of concrete grievances."

Okay, I'll bite the bullet.

Firstly, there was the whole "computer games are and cannot not be art". This would have been a subject of genuine debate 10 years ago; 5 years ago not so much and as of today, it simply displays the ignorance of the proponent. It's not very much unlike Morris citing his creationist "research" back from the 60s.

Secondly, the anniversary of the atomic bombings. Painting his opponents as amoral nuclear 'apologists' who supposedly see nothing wrong with the destruction of 1000000 human lives, he argues that the bombings were undertaken only because of some unimaginably callous experiment of seeing how the bombs would affect a sizeable civilian target (while offering no evidence except by anecdote of Hitchens and Grayling, while contradicting well-document historical proof such as the failed military coup to prevent surrender even after the bombings).

Thirdly, this whole Kurzwiel fracas where no amount of misrepresentation of his opponent seems to be below him (such as ascribing completely simplistic notions of "emergency" to his opponents).

Posted by: theswede | August 21, 2010 8:57 PM

And the answer, of course, is NONE. Physics does it for you.

Are we even in the same reality here? If your statement here is true, and RK has access to software which does protein folding with NO information, why doesn't he share it? And how do you know of its existence?

Or have you forgotten what we're talking about? A computer implementation of a brain starting from DNA requires a model of protein folding, and does not get physics for free. For that matter, if it's so simple, why isn't it modeled already?

I have no idea why you're talking about some alien lab. We're talking computer models here, nothing else.

Posted by: Nerd of Redhead, OM | August 21, 2010 9:01 PM

In short, the information theory argument is not trying to claim that with only 50MB of information one could simulate every particular of the way you were born, but rather that around 50MB of information is enough to describe the development and birth of what almost anyone, after the fact, would agree is a normal human baby.

Posted by: tylerofmanyminds | August 21, 2010 9:01 PM

p, do me a favor and answer the following questions:

1. What is the entropy of a four character alphabet a, c, g, t subject to the distribution p(a) = 1/2, p(c) = 1/4, p(g) = 1/8, p(t) = 1/8.

2. Plain Kolmogorov complexity is bounded from above by the binary ____ plus a constant.

Some knowledgeable about information theory should be able to answer these questions or find the answer easily.

Posted by: theswede | August 21, 2010 9:03 PM

msironen, I'll give you the nuclear bomb, that was just silly, but condemning the whole blog minus two categories of posts on that basis is pretty harsh. And you really have to be a bit more concrete on how demolishing precise statements is "misrepresentation" in this post and the one preceding it.

As for the gaming one, I missed it. I'll have to go look, as I would disagree with such a statement as well.

Posted by: msironen | August 21, 2010 9:14 PM

Well I'm not comdemning the whole blog considering that 80%+ (you gotta give me that's a fairly generous estimate) is either about biology or pointing out some religious outrage.

But for the rest, it seems like PZ is just as fallible as the rest of us (which is not to his detriment) but his implicit claims to the mantle of rationality are, for a disturbingly numerous parts, completely bogus.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkXEEO8_L7jQdb7CgEfig4ZsqzwCdYwjDw | August 21, 2010 9:20 PM

and so it is revealed that PZ's subconscious actually DOES believe in a soul, and his pontificating is a result of the cognitive dissonance.

Posted by: Jadehawk, cascadeuse féministe | August 21, 2010 9:26 PM

and so it is revealed that PZ's subconscious actually DOES believe in a soul, and his pontificating is a result of the cognitive dissonance.

fucking moron

Posted by: theswede | August 21, 2010 9:30 PM

p, I'm a computer scientist, and information content is rather useful to me. However, this use of it does not convey anything *in context*, namely prediction of how difficult mimicking a human brain is, and therefore isn't useful. At all. It's just wankery. That a compressed genome is 50MB says precisely nothing about whether we will be able to model a brain in 20 years or not.

Posted by: David Marjanović | August 21, 2010 9:37 PM

So, sure, protein folding is "complicated", in some handwaving intuitive sense.

Man.

Predicting the 3D shape of a folded protein from its sequence is one of the great big problems in... all of biology. It requires insane amounts of computing power to get anywhere near right. I'm surprised you didn't know that there's an analogue to SETI@home that does such calculations.

The sequence simply provides an incredibly large number of possible foldings, one of which is slightly more optimal than many others. It's an NP-complete problem, AFAIK.

Nothing but electrostatic attraction and repulsion are involved. In that sense, protein folding is extremely simple, har har.

Posted by: eleusis | August 21, 2010 9:40 PM

He has supplied NO basis on which to accept his claims.

Posted by: David Marjanović | August 21, 2010 9:48 PM

You're assuming that P=NP. To the best of my knowledge, this is still up for debate.

A few days ago someone published what he calls a proof that P ≠ NP. His colleagues are currently looking for errors in it.

Posted by: The Very Reverend Battleaxe of Knowledge | August 21, 2010 9:49 PM

So let's assume that Kurzweil is correct—soon it will be possible to simulate a brain, my brain, on a computer. Further, it will be possible to "upload" my brain state—my memories and (heavily edited) stream of consciousness that I call myself—onto it.

So there's a copy of "me" running somewhere else. What good does that do this me, here in this body? Am I supposed to kill myself so there'll only be one copy running? I can think of a way to save myself a shit-ton of trouble in that case.

Posted by: p | August 21, 2010 9:52 PM

The sequence simply provides an incredibly large number of possible foldings, one of which is slightly more optimal than many others. It's an NP-complete problem, AFAIK. Nothing but electrostatic attraction and repulsion are involved. In that sense, protein folding is extremely simple, har har.

Again, the thing that provoked this whole discussion was Kurzweil saying:

I mentioned the genome in a completely different context. I presented a number of arguments as to why the design of the brain is not as complex as some theorists have advocated. This is to respond to the notion that it would require trillions of lines of code to create a comparable system. The argument from the amount of information in the genome is one of several such arguments. It is not a proposed strategy for accomplishing reverse-engineering. It is an argument from information theory

Whether you find the abstractions of information theory conceptually useful/relevant is up to you. But it is what it is.

Posted by: The Very Reverend Battleaxe of Knowledge | August 21, 2010 10:04 PM

Wow! Somebody quoted that imbecile Searle's Chinese Room Argument? I remember reading that for the first time in Douglas Hofstadter's column in Scientific American. I had to keep checking the cover to make sure it wasn't another April Fool issue hoax. That is beyond doubt the stupidest thing I've ever seen written down in black and white—not because of how computers work, or how brains work, but because of how languages work.

Posted by: tylerofmanyminds | August 21, 2010 10:11 PM

"If DNA were completely random like you say, then yes, it would take about 700MB to store the human genome."

I'm not sure where I said this. I did assume that a sequence of base pairs could be modeled as a Bernoulli random string, but it wasn't uniform, if that's what you mean by "completely random".

On the other hand, I'm pretty certain that Kurzweil did make this assumption. His calculation that it would take six billion bits to three billion base pairs would require an average code-word length of 2, which is given by the entropy function when all the probabilities are 1/4.

Posted by: Greylander | August 21, 2010 10:29 PM

@ #184 David Marjanović:

Predicting the 3D shape of a folded protein from its sequence is one of the great big problems in... all of biology. It requires insane amounts of computing power to get anywhere near right. I'm surprised you didn't know that there's an analogue to SETI@home that does such calculations.

The sequence simply provides an incredibly large number of possible foldings, one of which is slightly more optimal than many others. It's an NP-complete problem, AFAIK.

It is only an NP-complete problem when performed as a search of the space of possible folding. And even if you had infinite computing power to search the entire space and found the lowest energy folding in the space, that would not necessarily be the correct shape.

However, it actually *is* more simple than that. Real proteins don't search their space of possible shapes. The just fold. By that I mean the parts of the protein move and wiggle around according to the laws of physics. The may be high-energy barriers to the lowest energy state(s), so they do not necessarily fold to the lowest energy configuration. The easiest way to find the proteins folded shape is to simulate the physics -- start it in an unfolded shape and let it do its thing in the simulation. Right now, we can do "small" of Order(100) monomers. What we can do will scale linearly or nearly so with computational power, so Moore's law will trump this problem in time. Right now heuristic searches are faster for larger molecules, but the combinatorics virtually guarantee that the physical simulation approach will outstrip it in time.

Posted by: Nerd of Redhead, OM | August 21, 2010 10:45 PM

Boy, the K-fuckwit fanboi idjits still haven't figured out that the development timing and turning on/off of regulatory developmental fuctions is absolutely critical to making anything, be it an arm or a brain. And nerves are not just connected to their nearest neighbors, but travel farther. Environment is extremely important, along with all the chemicals present, not all of which are proteins, but they are vital to the final product. And they have no idea how to do that or how it unfolds until the science tells them. No wonder idjits like Greylander come across as fucking morons. The problem requires basic science, not computing power. He just can't get that basic fact. But then, if he did, he would have to admit he is wrong, and it don't think he can do that. His ego is too big...

Posted by: tylerofmanyminds | August 21, 2010 10:47 PM

"Right now heuristic searches are faster for larger molecules, but the combinatorics virtually guarantee that the physical simulation approach will outstrip it in time."

It's unclear what you mean here. Heuristics all have exponential worst cases if P != NP, but they work well in practice since they don't have to return exactly correct solutions. On the other hand, accurately simulating a physical system could require solving a number of initial value problems exponential in the size of the input. I don't think there is any complexity math to indicate that this is true.

Posted by: tylerofmanyminds | August 21, 2010 11:01 PM

"methinks someone is having trouble understanding exponential growth. maybe he should read some Kurzweil."

No, no one should do that. He probably wouldn't ever learn that exponential growth accurately models just about nothing, because actual, physical systems can't sustain exponential growth indefinitely.

Look up sigmoid or logistic growth and get back to us, it's a much more realistic model.

Posted by: Malcolm | August 21, 2010 11:13 PM

p@165

I think this comment is really at the heart of the misunderstanding here. How much information does it take to describe protein folding? And the answer, of course, is NONE. Physics does it for you.

In a way you are correct: This is at the heart of the misunderstanding.
If all you need is physics, what do chaperone proteins do?
If you were right, then there would be no such thing as Mad Cow disease.

So you are correct in that the heart of the problem is people with absolutely no idea what they are talking about getting upset when they get called on their crap.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkXEEO8_L7jQdb7CgEfig4ZsqzwCdYwjDw | August 21, 2010 11:25 PM

> Look up sigmoid or logistic growth and get back to us, it's a much more realistic model.

ok - but i can easily envision molecular computers which continue true exponential growth for the next century.

i can easily imagine machines which will be able to fold proteins.

you can't? really?

so my point stands.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkXEEO8_L7jQdb7CgEfig4ZsqzwCdYwjDw | August 21, 2010 11:38 PM

> Given those premises, we should have been able to reverse engineer a fruit fly brain years ago

your argument is valid, but your math is off. by a lot.

if we simply look at your basepair comparison - there's a factor of 25 difference - which by Moore's law is about 9 years. so we should be able to simulate ONE fruitfly brain, on a supercomputer, in 11 years.

do you really doubt that prediction?

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkXEEO8_L7jQdb7CgEfig4ZsqzwCdYwjDw | August 21, 2010 11:42 PM

> Watching Kurzweil's defenders coming in for round two is like watching a gambler double down on a hard 20 and a full shoe

watching PZ's defenders is like watching a sick cult - and one that's REALLY bad at math.

Posted by: Nerd of Redhead, OM | August 21, 2010 11:48 PM

and one that's REALLY bad at math.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkXEEO8_L7jQdb7CgEfig4ZsqzwCdYwjDw | August 21, 2010 11:52 PM

> Except fool, it isn't a math problem, but a science problem. Ergo, your problem is category error. And you keep repeating the same error over and over and over...

hey, PZ - these are the guys defending you. surely that gives you pause...

Posted by: Nerd of Redhead, OM | August 22, 2010 12:02 AM

hey, PZ - these are the guys defending you. surely that gives you pause.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkXEEO8_L7jQdb7CgEfig4ZsqzwCdYwjDw | August 22, 2010 12:05 AM

> You are the well meaning fool of Heinlein's advice.

ah yes, and then the name calling. well, this has been truly enlightening guys.

oh wait, no it hasn't.

g'night.

Posted by: BrianX | August 22, 2010 12:14 AM

#219

Let's see: I compared you and your ilk to a somewhat dim compulsive gambler and the best response you have is tone-trolling? Why should I dignify anything you say with a serious response?

Posted by: Steven Dunlap | August 22, 2010 1:06 AM

@ BigMKnows # 51

Do you have a reference for that study?

The dead salmon study:

reported in

Sanders, Laura. "Trawling the brain." Science News 176, no. 13 (December 2009): 16-20. OmniFile Full Text Select, WilsonWeb (accessed August 22, 2010).

What's interesting is that even these results turned out to be false:

A reanalysis of the salmon's post-mortem brain, using a statistical check to prevent random results from accidentally seeming significant, showed no red-hot regions at all, Bennett, now at the University of California, Santa Barbara, and colleagues report in a paper submit-ted to Human Brain Mapping. In other words, the whole brain was as cold as a dead fish.

The main point I intended in my initial post was not that functional MRIs are crap, or that a dead salmon's brain reacts to visual images, but that the use of technology and the interpretation of the results remains far more complex than Kurzweil would have us believe. Unfortunately for Kurzweil with his pop culture understanding of science and technology, it's not like Star Trek where Mr. Sulu tells the Captain, "There are abnormal readings coming from the object, sir." Readings of what? How are they modeled? What baseline exists to indicate the "readings" are abnormal? (And on and on).

The article goes on to describe other studies which show biases in functional MRI research:

In a paper originally titled "Voodoo correlations in social neuroscience" in the May [2009] issue of Perspectives on Psychological Science, Edward Vul of MIT, Pashler and colleagues called out 28 fMRI papers (of 53 analyzed) for committing the statistical sin of "non-independence." In nonindependent analyses, the hypothesis in question is not an innocent bystander, but in fact distorts the experiment's outcome. In other words, the answer is influenced by how the question is asked.

The entire article is very well worth reading in full. Science news typically only summarizes other already-published articles. But some of its independent analysis makes for very interesting reading as well.

Posted by: eleusis | August 22, 2010 2:35 AM

PZ: "You really are a fucking moron, Greylander."

And he doesn't seem to give up. I just went back and (tried to) read the 700+ comments from the first thread. Greylander has been arguing on this blog full time for four days straight. :)

Posted by: Stephen Wells | August 22, 2010 3:00 AM

And another one- HeLa cells. A tiny change in regulation and a cell with a human genome will give you, not a brainy vertebrate, but an amoeba with baggage.

Those pesky facts getting in the way of uninformed speculation again. Tut tut.

Posted by: KingUber | August 22, 2010 4:22 AM

Dammit Ray Kurzweil, I want my robot wife, and I want her now!

Posted by: Elladan | August 22, 2010 5:16 AM

I find it rather amusing that Kurzweil thinks that Moore's Law is actually some sort of law of physics.

Most of his arguments are really based on this. If something seems computationally intractable, then hey, give it a few years. Exponential growth and all that.

And for that matter, the argument is kind of valid: if exponential growth continues, then we'll be able to perform amazing computational feats soon enough. Today's huge protein folding simulation, for example, would be like saving a jpeg is today on tomorrow's computer, and soon after that we'd run trillions at once.

The problem is: Moore's Law isn't actually a law. It's just a short term fit of where computer technology has gone in the past few decades. For that matter, it's kind of a crappy fit: computers aren't just getting faster at some steady pace. They're changing architecture to work around problems.

Your fancy new Intel processor can't really run a simple linear program much faster than a computer from five years ago -- rather, it's able to run a dozen copies of the program at once. If what you wanted was one answer, that doesn't help. And turning a linear program into a parallel program is one of the great problems in computer science. We have some solutions for it that sometimes work, but it's hardly an automatic thing.

And of course, you also have the problem that if something is NP-hard, exponential growth still doesn't make it easy.

So maybe in ten years, you'll be able to run a huge protein folding simulation on a desktop computer, but it'll still take a month to finish. Is that good? Yes! Does that mean it's a simple problem now? No.

Or maybe computers will start to stagnate. Who knows? Certainly we've been rapidly approaching a whole slew of major problems recently when it comes to making computers faster. We already gave up on clock rate -- but more cores aren't a substitute exactly. And now we're running into thermal/electrical limits as well.

The whole idea of taking some short-term success in a technology and assuming it will continue indefinitely is kind of bizarre. If you do that, of course you get some spurious results like "...and then in 50 years, magic!"

Posted by: https://me.yahoo.com/a/Kl1q4vMTqftWtpJ2_1aiHTsCct3ZWnpvDQ--#0f7ff | August 22, 2010 9:21 AM

Jeff Hawkins and Numenta http://www.numenta.com/ among others already are well down the path of understanding the basic principals of the neocortex. Read his book On Intelligence. Ultimately these things will not be turing based architectures. The prototypes are already displaying (primitive)intelligence and its only 2010, this argument is over. its really only a matter of time.
I'm having trouble following any of the PZ arguments. I guess he feels there are no underlying principals? Can he prove that? My theory is that PZ likes to reinforce his self identity as this 'rigorous unforgiving debunker' like Dan Dennet (who rules). The PZ argument style too emotional and semantically vague to be convincing.

Posted by: co | August 22, 2010 9:57 AM

#234:

I'm having trouble following any of the PZ arguments. I guess he feels there are no underlying principals?

Er... wot? Why do you guess that? Has nothing that PZ has said got through?

Posted by: co | August 22, 2010 10:08 AM

I also looked up Hawkins. He's undoubtedly successful, and a bright guy. He has some good publications listed on the Numenta website. However, the code is closed-source, and people don't seem to be very enthusiastic about it (despite Wired's interesting write-up about the fundamentals).

That article (http://www.wired.com/wired/archive/15.03/hawkins.html) ends with the sensationalistic quote from Hawkins: “The core principle, the hierarchical temporal memory component, I cannot imagine being wrong.”

Now he just has to prove it.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawkXEEO8_L7jQdb7CgEfig4ZsqzwCdYwjDw | August 22, 2010 10:29 AM

> Bad at math, huh? Well I did the math and got 37500 lines of code, by using Kurzweils formula. A trivial amount of code, one could probably write in a week or two. So get to it!

you know, when you ignore what other people are saying, and argue against the voices in your head, you aren't very convincing.

well, except to the other fanbois.

Posted by: pccdrski.myopenid.com | August 22, 2010 11:11 AM

Recently got AI from Netflix and only got a third of the way through it. The brain is a recipe , NOT a blueprint.
To all of these futurists, the Promised Land is always just over the next hill. And that hill is surprisingly close!
Think of a social interaction an outgoing four year old can maintain. Thanksgiving at Aunt Mary's house. Meeting new relatives. Matching up cousins with siblings. Uncle with Aunt. Uncle Joe is mean. and took your seat. Granma is saying things that don't match the conversation and she is not to be trusted. Etc. Etc. I'm not a clinical psychologist, but I'm sure an analysis of that specific situation would reveal hundreds of evaluations and decisions made by that four year old. AI ain't human!

Posted by: eleusis | August 22, 2010 11:35 AM

lol. That would be considerably harder on unformatted text, but I get your point. :)

Posted by: Stephen Wells | August 22, 2010 12:59 PM

I've had an epiphany.

The genome does not code for the cell.

Bear with me. The genome codes for the components of the cell. If the transcription and translation of the genome is taking place within a cell, then those components go to maintain the cell. But if you take the human genome and the translation equipment and all the tRNAs and ATP you want and set them going outside a cellular context, will you get a human cell self-assembling from components?

You will not.

The genome doesn't code for the cell because in four billion years and more, it's never had to. Every successful replication of the genome of every ancestor of each one of us took place inside a cell, and that chain goes all the way back to some facts about the physics of phospholipid vesicles in salty water, a long long time ago.

I feel pretty deep right now. :)

So Kurzweil and Greylander and Uncle Tom Cobbley and all can spend as long as they like staring at their however many bytes of genetic sequence information, but there are facts about biology that they will never find in there.

Posted by: Stephen Wells | August 22, 2010 1:19 PM

@Sven: I am now forced to hate you for having had my great idea before me :)

Posted by: amphiox | August 22, 2010 1:42 PM

Chaperone proteins do exactly what the laws of physics tell them to do, given their sequence, which is encoded in the DNA.

Chaperone proteins do exactly what the laws of physics tell them to do, given their sequence, which is encoded in the DNA, and the local environment (such as temperature, pH, ionic concentration, and the concentrations of all the other proteins) which is NOT encoded in the DNA.

Posted by: amphiox | August 22, 2010 1:49 PM

Stephen Wells #246;

Exactly. The genome doesn't even contain enough information to code for a single cell, let alone a brain.

@Sven: I am now forced to hate you for having had my great idea before me :)

See my post at #115. But you expressed it more clearly, I think.

Posted by: Kristjan Wager | August 22, 2010 2:42 PM

I'm afraid I have to add to the list of failed analogies and misunderstandings on the part of PZ Myers. Actually, if you put a penny in the bank and doubled it every year, you would have $10,485.

So, in other words, he wouldn't have a million dollars in his retirement fund after 20 years. It would actually be pretty far from a million.

Yes, six more years would give him a million, if the growth was exponential, but that doesn't change the fact that he wouldn't have the money after twenty years.

So the analogy stands, and your reading comprehension fails.

Posted by: julset10 | August 22, 2010 2:46 PM

as Kurzweil's Law of Accelerating Turns predicted this entire argument and the whole premise of reverse engineering the brain has been made obsolete by two newer paradigms that require NO knowledge of the brain to capture it's secrets and augment it-

the path to AI will not be high-level software engineering or robotics- not because these aren't feasible but because they will take much longer than other routes that will make this approach obsolete-

the most likely and promising approach is IA- Intelligence Augmentation through BCI- BCI gives you understanding of the brain- not the other way around- BCI only needs the resolution to see and stimulate neurons- with BCI you can copy the brain without understanding it by simply recording the network connections and function over time - you interact with each person's unique sensory cortices by active trace probing

and the next most likely is Wolfram's NKS searches for intelligent code in the Computational Universe- this approach would yield incomprehensible black-box like kernels of code that unpack into even more incomprehensible and more complex programs that behave intelligently and with true consciousness yet will be more hard to figure out than the brain because they will be based on their own unique evolutionary history that will have nothing to do with ours- even the ideas of embodiment and senses may be irrelevant because 'found' AI will be so alien that these ideas of moving through and being aware of the environment may not even apply- they could come from regions of the computational universe that have no analogues of space or matter

Posted by: julset10 | August 22, 2010 2:48 PM

that was supposed to be Law of Accelrating Returns-

Posted by: julset10 | August 22, 2010 2:56 PM

>>You guys, including Kurzweil, just keep waving your hands about mathematical rates of increase, but you've got no metrics, no data, and no way to actually measure whatever the hell you're all talking about.

LOL!

http://www.singularity.com/images/charts/thumb_BrainScanningTime.jpg

a molecular recording of a brain IS by definition an AI and is an 'uploaded' version of the original brain- therefore as soon as molecular level resolution of the brain is possible the argument is over- WITHOUT any understanding at all- you don't have to understand to copy and reprogram- you only have to SEE all the elements of the system and record them

the very idea of trying to re-invent the wheel is not very clever- I can have sex with my wife and have her produce an intelligent being- i just need to copy that- not understand it

Posted by: Nerd of Redhead, OM | August 22, 2010 3:12 PM

you are advocating that the "world" is some magical thing that is set apart from the Computational Universe- it is not-

Posted by: eleusis | August 22, 2010 3:21 PM

Here's another figure from Kurzweil. Someone completed the part that he conveniently omitted. Turns out the Singularity already happened:

http://www.kk.org/thetechnium/Blog_Kurzweil.gif

Posted by: julset10 | August 22, 2010 3:26 PM

just some naive dualism- nothing to see here folks

the scan image gives you a handle on a real prediction when molecular resolution can be achieved- when it is you don't need to understand the brain to record it- if you can record it you can edit the recording and then begin PROPER EMPIRICAL science of watching the brain work and reproducing the results with pattern recognition - your primitive and flawed idea of science is merely the old 19th century scientism filled with wrong assumptions about everything- naive materialism- you assume that matter itself has some primacy or special vital aspect that is missing from the causal patterns of it's organization and rules of motion- completely without a reason other than the hidden Monotheism that lurks at the foundation of all such classical/materialistic/dualistic thinking-

is it any wonder why it is only no-nothing low-level engineers from podunk junior colleges in Indiana [or Montana] that are always criticizing Kurzweil and the idea of a Singularity- while the LEADERS of theses fields like Seth Lloyd the head of engineering at MIT or Lord Martin Rees the Astronomer Royal of GB or Steven Wolfram the creator of Mathematica or Marvin Minsky the father of AI himself who has been called "the smartest man in the 20th century" find some version of the Singularity idea to be self-evident? geniuses who got their first PhDs as teenagers and who created new paradigms of science and technology against the rabble of uninformed and ignorant meme-parroting fools who barely got their Bachelor's degrees

Posted by: eleusis | August 22, 2010 3:33 PM

LOL, julset10 basically just called PZ a monotheist. This shows how devoid of reality he is.

Also, argument from authority and nyah-their-smarter-than-you. I thought you all were committed to rationality?

Posted by: First Approximation, L'esprit de l'escalier | August 22, 2010 3:47 PM

is it any wonder why it is only no-nothing low-level engineers from podunk junior colleges in Indiana [or Montana] that are always criticizing Kurzweil and the idea of a Singularity

Wrong:

Many prominent technologists and academics dispute the plausibility of a technological singularity, including Jeff Hawkins, John Holland, Daniel Dennett, Jaron Lanier, and Gordon Moore, whose eponymous Moore's Law is often cited in support of the concept.[16][17]

Posted by: Greylander | August 22, 2010 4:00 PM

@ #199 PZ Myers:

You really are a fucking moron, Greylander.

This coming from the guy who thinks "ontology" is a good metaphor for a computer program. You completely lost credibility on matters computational with that one, PZ.

Astonishing! So protein folding really is a trivial problem, and all we have to do is sit back and wait for processing power to catch up, and then we'll just solve it all by brute force!

Many highly non-trivial problems nonetheless crumble under the weight of number crunching power, PZ. The worst-case algorithms for physical simulation of protein dynamics in all its detail are O(N^2)¹ (super hard-core simulations that tracked all the individual molecules of the solvent as well as the protein itself could conceivably be O(N^3) if simulated volume scales with length of protein sequence, but given we have good models that already that don't need to do that simplify the effects of the surrounding solvent and do just fine, such detail is unlikely needed for most practical purposes). So each factor of a 100 in computing power will get us *at least* a factor of 10 in the size, or in the number of simulations possible in a given time, or any combination of the two. O(N^2) simulations are only necessary if you model the forces between every pair of particles (atoms in the most detailed simulations) and since the forces drop off rapidly with distance, there are many ways to simplify the simulation to O(N log(N)) or even O(N) at the cost of some accuracy but huge speed up.

In any case, even at O(N^2) (or any polynomial complexity) we still get exponential rate of improvement (do I have to do the math or can you work it out for yourself?) as long as raw computing power increases exponentially -- and there is no foreseeable reason why a generalized Moore's Law is going to peter out any time soon.

We can already do all-atom physical simulations of "small" proteins. There is nothing that prevents scaling up to larger proteins except the need for computing power. There is nothing that prevents simulating for trials or longer simulated-times except the need for computing power.

There are also plenty of techniques and shortcuts already in use today, which will still be useful -- even more useful with more raw computing power. For example, if we have solve the shape for one protein, then simulation of proteins with similar sequences or long identical or nearly-identical sections can benefit by using the known protein shape to suggest a starting point.

A great deal of research in computational protein folding has to do with finding heuristics and shortcuts to speed up the searches or simulations while sacrificing as little accuracy as possible. All these techniques will scale, and research on these techniques will continue even as more computing power comes on line. Sufficient computing power would actually render many of the shortcut simplification unnecessary (in simulations, not so much true in the combinatoric search algorithms), but that is further in the future.

Also, the gathering of empirical data is going to continue. With better and faster, more automatic lab techniques. That empirical data will also be used to inform and simplify the computational simulations. No one suggests that we all just sit on our hands (except for chip manufacturers) and wait and then magically the computers will start telling us all the protein shapes.

¹ For the uninitiated, O(n) notation means that the time it takes to run an algorthm (program) is proportional to n, all else being equal (such as speed of cpu), where n is a measure of the "size of the input". In this case "N" would be the number of particles simulated. It would be the number of atoms in the case of an "all-atom" simulation of the protein. In more simplified simulations it might be the length of the protein sequence. O(N^2) then means if you double the number of particles, you quadruple the time it takes to run an otherwise similar simulation. Exponential growth of computing speed far outstrips N^2.

Posted by: Kristjan Wager | August 22, 2010 4:07 PM

elusis, PZ wrote a piece about that silly graph some years ago (you can find a repost here)

Posted by: Greylander | August 22, 2010 4:16 PM

eleusis:

Brian: heh, actually you posted a lot more times than he did. I just copied the 769 comments from the old thread into a text file and sorted them by unique posters and number of posts.

The top commenters were:

61 Posted by: BrianX
53 Posted by: KG
48 Posted by: John
44 Posted by: Greylander

Interestingly, there were exactly 200 unique commenters on that thread. Quite a response thanks to Gizmodo et al.

@ #243 theswede:

eleusis, try doing a word count instead of a post count. Many small comments is the norm on a medium like this; the walls of text Greylander barfed out isn't.

So fucking what?

Oh I get it. PZ writes a long blog post (this time it happens to be full of stupidity). Then the rest of us just get to say "Go PZ you're so right!" "No he's not" "You're stupid!" "No! You're stupid!"

I decided to take a shot at giving serious and thorough answers. Just to see what would happen. Ya know what happens? People just make hand-wavy generalized counter "arguments". Apparently no one (well very few) has the guns here to engage in serious point/counterpoint.

Posted by: Nerd of Redhead, OM | August 22, 2010 4:22 PM

I already told you that this simply showed an indication of when molecular resolution would be available

(this time it happens to be full of stupidity

Posted by: Kristjan Wager | August 22, 2010 4:35 PM

They don't talk the biochemistry, which is the topic, they talk computers, which aren't. Get with the program.

Yes. This.

It's be quite annoying that some people haven't been able to figure out that it's not the computer part of the equation we are talking about (though that is pretty bad), but rather the knowledge that we need to base the simulations on. It doesn't matter if Kurzweil is right about the exponential growth in computer power (he is not, and he also ignores the software side), it would still not be possible to create simulations without understanding what should be simulated. And our knowledge is nowhere close enough for that yet.

It's not really that hard to understand, but apparently some people think that if we just throw enough computer power at some problem, it will be solved. It won't. Computer power will help, but it won't do it alone.

Posted by: BrianX | August 22, 2010 5:12 PM

I decided to take a shot at giving serious and thorough answers. Just to see what would happen.

YM "bloviating pretentious filibusters". HTH. HAND.

Posted by: Stephen Wells | August 22, 2010 5:25 PM

Greylander, as a researcher in the field, let me tell you that when you say "We can already do all-atom physical simulations of "small" proteins", you mean "We can do simulations of a few nanoseconds of dynamics of a small protein in an unphysical situation using empirical force fields with a huge number of rather dodgy parameters and known-wrong functional forms and without taking into account any quantum effects such as any chemistry. Or we can try to do chemistry using quantum mechanics in which case we use DFT which is based on a functional which we don't know the actual form of so we use guesses which are known wrong."

Posted by: KG | August 22, 2010 5:44 PM

On the last thread, there were several references to the "Blue Brain" project headed by Henry Markram. Here's Markram from an interview in SEED Magazine:

Blue Brain scientists are confident that, at some point in the next few years, they will be able to start simulating an entire brain. “If we build this brain right, it will do everything,” Markram says. I ask him if that includes selfconsciousness: Is it really possible to put a ghost into a machine? “When I say everything, I mean everything,” he says, and a mischievous smile spreads across his face.

Henry Markram... has a talent for speaking in eloquent soundbites, so that the most grandiose conjectures (“In ten years, this computer will be talking to us.”) are tossed off with a casual air.

Now compare with an extract from the Blue Brain FAQ:

To what extent will the computer give the same response as actual living brain tissue?

Our goal is not to build an intelligent neural network, but to replicate in digital form the NCC as accurately as possible. We will perform similar experiments on the virtual NCC as in the actual NCC and keep doing this until the virtual NCC behaves precisely the same in as many ways as we can measure, as the actual NCC. Once this replica is built, we will be able to do experiments that normally take us years and are prohibitively expensive or too difficult to perform. This will greatly accelerate the pace of research.

How do you relate your research to the field of artificial intelligence?

We are not trying to create a specific form of intelligence, but rather trying to understand the emergence of mammalian intelligence. We will of course be examining the computational power of the NCC. In particular, we will explore the ability of the NCC to act as a Liquid Computer (a form of analog computer that handles continuous data streams). This could be used for dynamic vision and scene segmentation, real-time auditory processing, as well as sensory-motor integration for robotics. Another special ability of the neocortex is the ability to anticipate the future based on current data (the birth of cognition) and so we will examine the ability of the NCC to make intelligent predictions on complex data. We will also examine other forms of computing that can be used - perhaps hybrid digital-analog computing, but this is quite far in the future.

How much do we know about how individual neurons and brain molecules behave - is it possible to uncover everything?

The past 50 years have yielded an immense amount of information about the brain, the neurons it contains, the molecules that make up the neurons, and the genes that produce the molecules. There is still a tremendous amount to find out, but we now need a platform where all this information can be integrated and that is the main purpose of building the NCC. "Everything" is a lot, but we have definitely entered a phase of brain research where the brain's secrets are being revealed at an extremely rapid pace. I call this phase the Synthesis Phase where the fragments of knowledge are being collected and assembled to reconstruct the manner in which the brain works - the Blue Brain Project is just one part of this process.

Do you believe a computer can ever be an exact simulation of the human brain?

This is neither likely nor necessary. It will be very difficult because, in the brain, every molecule is a powerful computer and we would need to simulate the structure and function of trillions upon trillions of molecules as well as all the rules that govern how they interact. You would literally need computers that are trillions of times bigger and faster than anything existing today. Mammals can make very good copies of each other, we do not need to make computer copies of mammals. That is not our goal. We want to try to understand how the biological system functions and malfunctions so that this knowledge can benefit mankind.

Markram is clearly a fine neuroscientist - and an absolutely outstanding self-publicising bullshitter.

Posted by: Greylander | August 22, 2010 6:18 PM

@ #287 Elladan:

Greylander:

In any case, even at O(N^2) (or any polynomial complexity) we still get exponential rate of improvement (do I have to do the math or can you work it out for yourself?) as long as raw computing power increases exponentially -- and there is no foreseeable reason why a generalized Moore's Law is going to peter out any time soon.

Really Greylander? What makes you think Moore's Law is a law? It's not, you know. It's a crappy fit of the progress in compute power over time in the past. You have no particular reason to think it will continue. ... We don't know how powerful computers will be in ten years. We can try to guess -- but any such guess has to be based on semiconductor physics, research, and manufacturing techniques. Not on Moore's Law.

Elladan, this is an excellent objection. You are correct that neither Moore's Law, nor various generalizations thereof are "laws" in the sense of "laws of nature". No one with any sense disputes this. There are, however, very good reasons to think it will continue for the next several decades.

Although we are approaching the minimum size of circuits on silicon, we still have a bit of room left "at the bottom" which will take us out for a few years. After that, even if no improvements are made at the chip level, cost efficiencies of mass manufacture will more and cheaper cpu's, which means we'll be able to have bigger "grids" or "clouds" of available cpu's networked together, which is basically all a supercomputer is nowadays anyway. So that buys us several more years. Then there are TSV's, putting 3-D chips on the horizon. Chips of a few layers are already made for special purposes. True 3D cpu's have some engineering hurdles, but none that are intrinsically impossible to overcome. Then there are many completely new technologies on the horizon or in prototype from graphene substrates and carbon nanotube substrates, to optical circuitry, to self-assembling molecular circuitry constructed from artificial DNA or RNA. These are not pie sky fantasies. The baby-steps of all of these technologies and more are already being developed in labs.

Yes, it is not a "law", there is no guarantee it will continue, but there is no good reason to expect it to end in the foreseeable future. Many good reasons to expect it to continue.

Posted by: Greylander | August 22, 2010 6:42 PM

@ #286 Stephen Wells,

Greylander, as a researcher in the field, let me tell you that when you say "We can already do all-atom physical simulations of "small" proteins", you mean "We can do simulations of a few nanoseconds of dynamics of a small protein in an unphysical situation using empirical force fields with a huge number of rather dodgy parameters and known-wrong functional forms and without taking into account any quantum effects such as any chemistry. Or we can try to do chemistry using quantum mechanics in which case we use DFT which is based on a functional which we don't know the actual form of so we use guesses which are known wrong."

And yet, for all that, those little simulations do pretty a pretty darn good job. The same techniques will scale quite nicely with greater computing power. And with further accumulation of empirical data, the empirical force fields and other guesses can be filled in. Or additional computing power may allow simulation at a more fundamental level of physics so fewer empirically determined effects need to be included. Or a mixture (trade-off) of both of those along with larger models and longer sim-times.

You appear to know more about the state-of-the-art than I do, as well you should if it is your field. Since you bring it up, what is your role in that research?

Just think about what you are going to be able to do with computers a million times faster than today. A billion? A trillion?

Let's talk [generalized] Moore's Law here for a moment. If you dispute that, then we have a whole 'nuther discussion on our hands. But if you don't have a substantive disagreement with the expectation that [mere] "raw computing power" will continue to increase exponentially for at least the next few decades. Just think about the size and detail of simulations you will be able to run with computers a trillion times faster than the one's you have today.

Posted by: David Marjanović | August 22, 2010 7:47 PM

Long ago (well, about 20 hours), I wrote:

A few days ago someone published what he calls a proof that P ≠ NP. His colleagues are currently looking for errors in it.

...and have already "identified critical flaws".

Anyway:

[Protein folding] is only an NP-complete problem when performed as a search of the space of possible folding. And even if you had infinite computing power to search the entire space and found the lowest energy folding in the space, that would not necessarily be the correct shape.

So far, so good...

However, it actually *is* more simple than that. Real proteins don't search their space of possible shapes. The just fold. By that I mean the parts of the protein move and wiggle around according to the laws of physics. The may be high-energy barriers to the lowest energy state(s), so they do not necessarily fold to the lowest energy configuration. The easiest way to find the proteins folded shape is to simulate the physics -- start it in an unfolded shape and let it do its thing in the simulation. Right now, we can do "small" of Order(100) monomers.

Fine...

What we can do will scale linearly or nearly so with computational power

I think this assumption is why PZ called you "a fucking moron".

I do phylogenetics. To find all most parsimonious trees that explain a data matrix (a matrix of "species" and characters, consisting mostly of 0s and 1s) is an NP-complete problem if it's done by exhaustive search. Of course that's not done, because after 10 or more recently 12 "species" it would take forever. Up to 25 or so, branch-and-bound is used (I don't know how that works), and beyond that, heuristic searches are used: a somewhere-near-optimal tree is built by an extremely simple algorithm, and then it's rearranged millions of times to see if lower minima can be found. This way, dealing with a matrix with 102 "species" and 289 characters takes a day on a modern desktop, and increasing that to 110 "species" takes... two days.

It is still not linear. It's still exponential. The exponent is lower than for an exhaustive search, but it's still > 1.

And then, comments 250 and 286 come in.

so Moore's law will trump this problem in time.

You have no reason, other than random guessing, for thinking so.

Right now heuristic searches are faster for larger molecules, but the combinatorics virtually guarantee that the physical simulation approach will outstrip it in time.

That I want to see.

While it's probably not necessary to simulate every minute quantum interaction in order to simulate a human brain, Kurzweil has presented no compelling argument that the brain necessarily has such "basic principles of operation" waiting to be discovered. As it happens, some researchers have experimented with "evolved circuits", using genetic algorithms to produce a circuit with a specified behavior. In at least one case, they initially did not even understand how the evolved circuit could possibly work. The circuit was a mockery of the "basic principles" commonly used in circuit design. I'm no biologist, but I think the safe bet is that if the brain is analogous to any kind of circuit, it is analogous to an evolved circuit, not a designed one.

That circuit, it turned out, used what is called a parasite current. It relied on "it's not a bug, it's a feature".

And yet, for all that, those little simulations do pretty a pretty darn good job. The same techniques will scale quite nicely with greater computing power.

Why do you think so? Why don't you think that the currently ignored effects increase exponentially with the size of a protein?

Posted by: Greylander | August 22, 2010 8:19 PM

#285 KG Author,

KG, finally an excellent rebuttal. Or almost.

KG:

A basic point for Greylander: no, neither the brain's fine structure, nor neural activity, is anything like software. The brain, being an evolved rather than a designed system, isn't separated into hardware and software.

You start out (above) with what appears to be another fail based on a strawman. (Well it is that, but you get better below). You've seen enough of my "blather" to know that I make no claim that the brain is easily separated into hardware and software. Since nothing I've said is founded on such a claim, this gets nowhere.

But here we get to the heart of it:

... But evolution by natural selection does not rely on exploiting a limited set of "basic principles". It uses whatever works a bit better than the current multi-level tangle, given the ill-defined, ever-shifting range and distribution of extremely complex environments in which the genome finds itself. To see what I mean, read this paper by Adrian Thompson. It was indirectly referred to above.

Perhaps it will be an enlightening paper. I'll read it before I post again.

However, you may recall from from my earliest posts, previous thread, that in estimating the memory capacity needed for a simulation, I allowed as to that it could turn out that significant computation in the brain is done by molecular scale components (i.e. substantial leaned information stored in the detailed configuration/arrangement of molecules, as oppose to aggregates like the density of receptors on the surface of a synapse). This could explode the estimate by several orders of magnitude and set back the time frame several decades.

I've been busy rebutting absurd "counter-arguments" and gross misunderstandings (some from you, sorry, and you still didn't get much of what I said since you think the above is in substantial disagreement with me). And we've all been distracted by the "50 million bits"/information theoretic thing (after all that was what PZ's original post was about). So many people keep confusing that "50 million bits" with a Kurzweilian (or my) estimate of the total complexity of information in the brain it really muddies the water, since there are a handful of major points under discussion relating to simulation of the brain and how difficult it will be.

So let me be absolutely clear: Assuming there is no significant molecular-scale coding of information¹ (as oppose to moleculat aggregates like densities/quantities of receptors, neurotransmitters, etc.), I put my estimate of the complexity of a living active brain at (ballpark):

10^18 maybe 10^19 bytes.

That is the estimate which takes into account all those "unique snowflakes" of which PZ is fond. The original article that got this all started had it at 10^16 bytes, which would be highly optimistic, I think, and I said so.

Go back and see my first or 2nd post, previous thread, lest you think I'm backtracking or changing my tune with out "admitting I was wrong". Most of you just don't get what the "50 million bytes" is about and keep raising specious objections... I'm tired of trying explain that distinction.

So let's just deal with one number. Say, 10^18 bytes. That is how big/complex the simulation will likely need to be. That's data plus executable code. OK? The "50 million bytes" issue is then just an issue of how much of that is "executable code" per se, and how much is "data".

Now back to your point, KG. The multi-level tangle you mention could indeed involve molecular scale encoding of information (again, as opposed to encoding in aggregate quantities and structures like # of recptors on a synapse). I find it highly unlikely that much molecular encoding, if any, occurs for a number of reasons, but mainly because it is just too messy at that scale. If the molecular-scale information is not stored massively redundantly, it is way to subject to random effects. On the other hand, if it is stored highly redundantly, then the information is not stored at the molecular scale, it is stored in molecular aggregates. Aggregates make a great deal of sense, especially for an evolved messy tangle, as they are quite stable -- and stability is quite important for storing information as anyone can attest who has suffered a power outage with unsaved work on their computer.

Even without significant molecular-scale encoding of information, I still allow for... let's see... about ten to one hundred million bytes per neuron. You can dispute the estimate or my reasoning behind it. You can insist that it still must be much much larger than that. But stop suggesting that I don't have an appreciation of the sheer complexities involved.

Like you and anyone else here, I can imagine even greater complexity... bumping up the estimate to 10^20-something. I can even imagine that the important stuff happens at the quantum level and then all best are off. But what evidence to we have for molecular-scale or quantum scale encoding? None, so far as I know. And before you contradict me, make sure you get what I mean about the difference between molecular-scale encoding verses aggregate encoding. Perhaps there will be some evidence in the paper you reference that will make me reconsider the above line of reasoning, but I'm not holding my breath.

Posted by: Greylander | August 22, 2010 8:51 PM

@ #285, KG:

To see what I mean, read this paper by Adrian Thompson. It was indirectly referred to above.

I actually attended a Genetic Algorithms conference in Vegas conference in which this, or a very similar result was presented, circa 2000. (anyway, I think that is when/where I saw it, if memory serves) Not at all what I thought you were pointing to... I was expecting something about the intricacies of the multi-level tangle that is our brain.

I assume that your point there is the very surprising way in which "disconnected" portions of the circuit were still relevant, ergo how we must then expect messy and unexpected interrelationships in squishy wet evolved systems as well. I do not disagree at all.

But do see my points about aggregate scale verses molecular-scale encoding.

Now, if someone discovers, for example, that inside each neuron there are extremely long strands of say, RNA which appear to have different random sequences from cell to cell, but for which no function could yet be discerned. Then, i would grant that such might be an example of a potential mechanism for molecular scale encoding. But even then, to approach the 100MB I've already allowed for an individual neuron, you would need RNA strands that begin to rival the genome itself in order to bump up my estimated information-per-neuron.

Another possibility would be if each neuron was discovered to have wildly different configurations of epigenetic factors, thus allowing us to posit a different mechanism for molecular scale encoding.

Have either of these possibilities ever been discovered? Anything along these lines? Have any neurobiologists suggested that they are possible let alone likely?

It would be really cool. I stand ready to change part of my tune.

Waving your hands and shouting "complex complex complex" just doesn't cut it.

Posted by: p | August 22, 2010 9:08 PM

Oh, and, one more standard objection: "what about photographic memory?" Short answer: doesn't exist. There exist people who devote their whole lives to memorizing books, and, perhaps the best of these can memorize a few million words (10MB?). If photographic memory existed in any literal sense, the best memorizers in the world would be able to do way better than that.

Posted by: Greylander | August 22, 2010 9:59 PM

@ #299 p:

While peer pressure on this thread is getting Greylander to bend over backwards with generous estimates of how close to 10^20 operations per second

Not caving, p. I opened long ago with the 10^18-10^19 bytes of RAM... it was more like 10^21-10^22 FLOPS. That was somewhat high-balling it. But Kurzweils 10^16 bytes and 10^17 FLOPS seems a bit too optimistic, even restricted to the cerebral cortex alone.

Posted by: Greylander | August 22, 2010 10:40 PM

@ #295 Elladan:

Right, we might imagine that the cost per compute unit of silicon will continue to go down as industry cost-reduces its fabs. Except hold on a second: even right now, when there's still some clear headway even just from die shrinkage techniques, we're already running into a power problem in large data centers.

Elladan, that is a good point on the power consumption. But that only impacts that mass-production angle, which I rely on as a worst-case scenario. And there are considerable energy efficiences to be sucked out of silicon yet.

Perhaps you'll present reversible computing as a solution to this issue (despite the complete lack of any practical demonstration of it) or some other approach -- but come on.

Well... now that you mention it...

But seriously, there is nothing intrinsically energy intensive about computation. If nothing else we all agree here that our own brains are yet doing vastly more computation than today's supercomputers, while using the energy of a light bulb. You have to be an ardent pessimist to suppose we won't be able to find ways to compute with less energy.

Look at your list of possible new technologies. Some of these might work. Or they might not. Or they'll work, but take twenty years of research to be of use.

How do you look at those and conclude, "well, these theoretical technologies will surely be available in time to maintain my desired exponential growth curve." ? It's all just exciting possibilities, not some sort of roadmap to exponential growth.

I wonder how you look at those technologies and have much doubt? The essence of a computer requires extremely simple switching logic. You can do it with gears and levers. You can do it with vacuum tubes. With transistors. Hell you can even do it with appropriately water pipes (not kidding, I read a paper ages ago detailing a "water computer" proposed as a defense against EMP attack -- completely impractical for other reasons but interesting. Basically they showed what amounted to different shaped multi-pipe joints that acted like logic gates (and, or, not)).

None of these technologies relies on discovering new principles of physics. All of them are in their infancy, meaning they are beyond the "just an idea" stage. We know in principle they will work. Is it *possible* that none of these will be developed and have a significant impact for another 20 years, thus putting a kink in the Moore's Law timeline? Sure. I'm not claiming there might not be "bumps" along the way. It is also possible that after your 20 year dry-spell, once the new technology does come online, the exponential growth will explode even faster.

As I've said before, I have no argument with anyone who just wants to multiply my time line by a factor of 2 or 3, at that level there lots of wiggle room for unpredictables and intuitive hunches. If you want to push back the timeline by a factor of 10... push it out to centuries, millenia, or "never"... I still cannot "prove" that you are wrong, but in that case I think you're just kinda blind. I see it. You don't. Maybe I'm deluding myself. Maybe you lack vision. Shrug.

Posted by: tylerofmanyminds | August 22, 2010 11:42 PM

"Wrong. Complex forces may require more calculation per partical pair, but that just goes to the constant factor not to the n²"

I'm not talking about complex forces, I'm talking about forces that can't be quantitatively handled in an n-body simulation. N-body simulations can only calculate forces between particles that can be accounted for by integrating differential equations. If you only want to account for electrostatic and van der Walls forces, that's fine (some protein folding simulations formulated as n-body problems do just that, but it doesn't encompass all the relevant details).

"Yes... but being able to handle more particles means we can also include solvent molecule and chaperon proteins as part of an "all-atom" simulation."

They would be subject to different forces than the amino acids themselves, so they would have to be dealt with in a different pass (a separate n-body problem).

"In other words if you can get good empirical estimate of the force fields of a monomer, then you might choose to treat the monomer as a single particle in your sim rather than dealing with the atoms that compose it."

This is a ridiculous strawman. I never said that you have to simulate all the atoms of the system. What I said was that there are well known global aspects of the protein folding process that we don't understand. Simulating protein folding accurately requires understanding them.

"Um... the greedy algorithm *is* a heuristic. I'm not sure but you might be confusing heuristic algorithms with algorithms guaranteed to provide perfect solutions (though they may take forever and a day)."

You are mistaken. Greedy algorithms are correct for problems that are matroidal. MST's are an example of such a problem.

Posted by: Greylander | August 23, 2010 1:17 AM

@ #306 tylerofmanyminds:

We're likely the only two left here but anyway...

Last things first.

You are mistaken. Greedy algorithms are correct for problems that are matroidal. MST's are an example of such a problem.

Facepalm. I stand corrected on that point. Been over 20 years but I had to prove that myself once. But in this case, I'm not sure what you are getting at with the MST example. In general heuristics sacrifice optimality for time complexity... so your example of heuristics with worse complexity made little sense in the context. I suppose there might be heuristics (or not necessarily even heuristics but perfect algorithms) with worse O(), but much much better coefficient or at east much better time on average ordinarily even though they have horrid worst-case. Since we are talking a problem with combinatoric complexity, I'm going to go with the heuristics that trade optimality of solution for dropping O() (whether average or worst-case) to polynomial or better. And such will benefit nicely from faster computers.

"Wrong. Complex forces may require more calculation per partical pair, but that just goes to the constant factor not to the n2"

I'm not talking about complex forces, I'm talking about forces that can't be quantitatively handled in an n-body simulation. N-body simulations can only calculate forces between particles that can be accounted for by integrating differential equations. If you only want to account for electrostatic and van der Walls forces, that's fine (some protein folding simulations formulated as n-body problems do just that, but it doesn't encompass all the relevant details).

Are you talking about field dynamics? I don't think you can mean the electromagnetic field (you don't seem to suggest it), as propagation there can be considered instantaneous at these time&distance scales. Perhaps you mean some kind of field which approximates the effect of the surrounding solvent without modeling individual molecules or atoms. I think very worst case there might be that the volume simulated might need to be as much as O(N) so the simulation of your dynamical field might make your overall simulation O(N^3). But granting you O(N) volume is a big concession. I can already think of reasons that should not be necessary.

If you have something else in mind, I need an example.

"Yes... but being able to handle more particles means we can also include solvent molecule and chaperon proteins as part of an "all-atom" simulation."

They would be subject to different forces than the amino acids themselves, so they would have to be dealt with in a different pass (a separate n-body problem).

I don't follow your thinking here. Each pair of particles might require a different force calculation depending on types of particle. I see no particular reason this requires a whole separate simulation -- the whole point is to simulate them all togethre so they interact. This is still going to be O(N^2) (or whatever is required by your "complex forces").

"In other words if you can get good empirical estimate of the force fields of a monomer, then you might choose to treat the monomer as a single particle in your sim rather than dealing with the atoms that compose it."

This is a ridiculous strawman. I never said that you have to simulate all the atoms of the system.

You misunderstand me. If anything I am proposing the all-atom simulations. After all, they are already being done for "small" proteins.

What I said was that there are well known global aspects of the protein folding process that we don't understand. Simulating protein folding accurately requires understanding them.

Can you elaborate on "global aspects"?

Posted by: KG | August 23, 2010 5:23 AM

Two minor corrections:

1) "computational complexity of the working brain" should be "computational power...".

2) As I've said before but should probably repeat, I'm not saying GOFAI is useless, in either science or engineering applications. I'm saying the assumption that a human-like intelligence could be developed by "skimming" and implementing a software-like symbol-manipulating level was an error.

Posted by: Greylander | August 23, 2010 9:33 AM

According to you, almost all the Kolmogorov complexity of the "design" of the neonate brain is in the genome. Now, I hope you'd agree that (if Kolmogorov complexity has any useful application here, which I don't believe it does), the whole biosphere must contain at least several orders of magnitude more of it than a single neonate brain.

Where was all this Kolmogorov complexity 4.5 billion years ago? After all, you've dismissed the idea that information from the environment can contribute "in any worthwhile sense" before the brain starts actual learning.

Obviously in context of a developing organism "information from the environment" refers to the current environment impacting the developing organism. Information from oh, say just 4 years ago, is not going to have any effect on the organism today unless you are talking about light from alpha Centauri, to invoke your light cone. 4.5 billion years of evolution has done a pretty good job of incorporating information into the genome. I'm not sure where you are getting "information in the present environment does not appreciably alter the fundamental structures/processes of the organism [as opposed to contributing to its innate plasticity]" is equivalent to saying "the environment over the past 4.5 billions years has not contributed information to the organism". It has. The genome encodes lots of trial-and-error information about how to replicate itself in past environments.

10^8 bits is nothing to sneeze at, despite the fact that we all watch compressed movies with files that require ten times that much. We obviously mean very different things by the phrase "separating the hardware and software of the brain" (or whatever), but setting that aside, my 10^18 (as opposed to 10^8) is where all the information from the environment, as experienced by the developing organism, goes.

Another way to state the point about the genome would be to say that, in rough terms a 10^8 bit program plus a gazillion bits of data from the environment as input goes into creating the 10^18 bits of information content of the brain. And 10^18 bits is only the cognitive info. I think we both agree that something like 10^30 or more is require to specify every atom and electron and lets not even go to quantum state.

It seems to me that your claim amounts to saying that figuring out what the ~10^8 bit program is (or most of it) is just going to be too damn hard to achieve in the next 20 years. And perhaps you also mean that if we only get 99% of it, then that is still somehow a total failure, because that last 1% is still going to be extremely crucial to the problem.

Part of the issue here may be that what you and many on your side have in mind for the 2030 prediction is a level of knowledge of brain that equates to being able to achieve the fabled "upload" (or is it "download"?) in 20 years. I'm not defending the "upload" thing or anything even close. I've barely read anything by Kurzweil, but I'm pretty sure the brain-simulation-by-2030 is not referring to the upload capability -- not talking about copying and simulating a living brain at that point -- not really even close. I *think* Kurzweil has in mind that life-extension technologies may give him the extra centuries needed to reach the "upload" stage. "upload" or copying of consciousness itself is right on the boundary between "scientifically possible in principle" and "pure magic".

I also would not expect the 2030 era brain simulator to pass a Turing test -- it will be glitchy and imperfect. We won't have a good way to give the "initial conditions" so it won't have any normal sort of memories to start with. We won't have the capacity to simulate development from the egg to the brain, so we won't be able to make the initial wiring look like a baby's wiring except maybe in a vague statistical sense (on the other hand, much of a babies wiring might be somewhat randomized as an initial state, so that might be ok). I expect that the simulation will start with somewhat random wiring, but it will operate under the same (or a close approximation) rules governing self-modification (growing/pruning synapses) and so it will be able to learn. I expect that for quite a few years after reaching the raw ability to do a decent simulation all we will end up with are more-or-less "mentally challenged" (low IQ), insane, or brain-damaged (or all three) sorts of simulated "minds". These simulated minds might be able to learn enough to interact and communicate on some level, but lacking any ordinary human experience, they may seem quite alien. Over the course of another few years we'll shake out the bugs and have even more simulation capacity, and have had time to let the sim-brain develop and learn in real-time, so we eventually have a simulated mind that will understand more or less its own nature as a simulated brain and able to communicate that level of self-understanding to us. My "prediction" if you want to call it that is that by around 2060-2070, we could have a simulated brain either in or wirelessly connected to an artificial human-like robot body (or succession of bodies from baby to adult size, or possibly a genetically engineered biological human-like "robot" body). It would be able to develop in a roughly "human" manner, but almost certainly aware of it's own nature and therefore still rather "odd" personality-wise.

We'll probably have GOFAI as you call it before we have the latter degree of human-brain-simulation.

I think it will be centuries yet before it is possible to do anything seriously resembling the capability to "upload" the complete information-state of a normal organic brain, but I also think that by that time such an ability would be superfluous -- we (or more likely our near term descendents) will have been so enhanced by genetic engineering and cybernetic integration, we won't care much about those original chunks of grey-matter.

I'm sure clarifying my own "predictions" here won't matter horribly much, as most of you would replace my decades with centuries, millenia or "never".

BTW, KG, for all the "you're an idiot" jabs (mine as well) of our first exchanges, I feel like we're actually now having a rational exchange on the subject. Appreciated.

Posted by: Thomathy | August 23, 2010 12:14 PM

I'm finding it very difficult in this discussion to move past the words 'from baby to adult' or the implicit belief that brains don't learn or think while they are developing. Language learning begins in the womb in humans. Recognition of the mother begins in the womb in children (as a concurrent result of language learning). The developing brain (and I mean that quite literally) is in the process of functioning during development and continues this process to the end of life.

It seems to me that there is a fundamental misunderstanding of the nature of the brain and how it is grown (develops) by some of the people in this discussion.

It is trivially true to say that DNA provides some sort of code from which a cell might be born of an extant one, but DNA contains no information about anything that that cell might one day do. The functioning of a cell is determined by its interaction in the developing organism with other cells through chemical signalling and like processes. The DNA is certainly not a blueprint -at least not in the way it might be thought to be- and it's just not going to be possible to extrapolate from DNA what that initial cell is going to eventually develop into, except to say that it will develop into a cell of whatever species that DNA represents.

It's terribly important too, to recognize that the cellular component is absolutely necessary. That initial cell with that DNA in that environment within the womb is 'being told' what to do, to split at the appropriate time, and those two cells to then again split and form whatever structural patterns give rise to further stages of development not only from the DNA within that cell itself (that cell already being functioning), but from the interaction in that environment, literally with the cells of the mother's body and her hormones and all those chemical signalling pathways. These processes cannot be separated.

All of this has to do with the salient point that I'd like to make: the brain is never a blank slate. At no point in its development, at least as a recognizable brain, is the brain not doing something (of course it's doing something before it's a recognizable brain, but I wouldn't want to call that anything like learning or 'working', because it's still formative). The brain must necessarily be doing something during its development because there exists in humans the extant pattern there in the brain that provides the framework by which language is learned (a process we really, really do not understand -like so much else). This is all to say that the brain functions before we're born and I think that's important.

Nothing that the brain does however is literally not in the DNA and can't readily be extrapolated from it. And the process that developed that neuronal pattern that facilitates language learning is not merely a product of what's contained in DNA. Nor is it necessarily sufficient for functioning that those neurons are only structurally patterned the way they are, they also have to interact in particular ways with each other and with other neurons and deeper than that each cell must interact with every other cell, even with cells that are remote from it. The brain, further, is part of a dynamic system that is the entirety of an organism. A brain does not function independently of the sensory organs or the spinal cord, indeed it's quite accurate to say that the brain and the rest of the organism are inextricably linked extensions of one another every part of which is interdependent and codependent.

So, when it comes to reverse engineering the brain, well, I just can't see that it can happen in any particularly useful way as it's being discussed. So what if we take any given human brain a recreate it structurally? For that structure to work you must also recreate the way the components of that structure interact with each other on all the myriad levels that they do, and not forget that the nervous system and all the sensory organs are a part of that brain, indeed that the body is a part of a dynamic system that is inextricably linked to the brain and the brain to it. It all functions as one piece working together. It seems to me that recreating an organism virtually (indeed, in this case we're talking about a human) would seem to be incredibly difficult to accomplish and considering we're not even close to starting I can't see how it's possible given the time frame espoused.

And I really can't see what relevance at all DNA, or the amount of information it has, has to do with that at all. Are we talking about growing a functioning brain (really and entire organism) from some DNA, all of it virtual with all of the hormones at the right time and right amount virtually programmed and all the cellular communications virtually programmed? I can't see how that could be possible given we don't even understand, as has been pointed out, how exactly cells communicate with each other or, importantly, what importance it has. The brain is not like a piece of machinery that can be taken a part and analysed to see how it works. I'm going to go so far as to say that the idea of reverse engineering the brain is a fallacious concept -at best.

Posted by: Thomathy | August 23, 2010 12:27 PM

It is trivially true to say that DNA provides some sort of code from which a cell might be born of an extant one, but DNA contains no information about anything that that cell might one day do.

I didn't mean quite that. Of course, the DNA contains information relevant to how a cell will function, but the DNA doesn't know where that cell will be or what chemical signals will cause that cell to 'read' or 'not read' those parts of the DNA that determine its functioning. The DNA doesn't contain, in other words, any plan for that cell only minimalist instructions for it to follow to perform it's functions as a cell of a particular type.

Posted by: llewelly | August 23, 2010 12:50 PM

DNA could be viewed as a program, whose behavior is strongly dependent on its input. Oh, yeah, and something like 80% of runs end in fatal crashes less than 1 month after fertilization ...

Posted by: tim333 | August 23, 2010 3:21 PM

For those interested in the actual progress on brain simulation check out the Bluebrain Year One documentary preview - pretty interesting, even for the Kurzweil haters I think.

http://thebeautifulbrain.com/2010/02/bluebrain-film-preview/

Posted by: jvospost3 | August 23, 2010 6:26 PM

I was known as a Futurist in the late 1970s and early 1980s, when I was writing those Omni Magazine cover articles. But I split myself into someone publishing real Science in refereed conferences and journals, and someone openly putting the fun ideas in Science Fiction.

With all due respect to the bubbly and glib Ray Kurzweil, who even my little brother likes, one needs to know the difference. I can enjoy writing Singularity Science Fiction without being brainwashed into the Cult of Singularity by a talented inventor who seems to literally misunderstand the exponential function, and ignore the Logistic Curve.

I left A.I. after my 1975 M.S. in the field, because of all the Computer Science geeks who were willfully ignorant of actual brains, animal and human.

Posted by: BrianX | August 23, 2010 9:27 PM

Tim333:

Except that it turns out that chess is not the AI-advancing problem people thought it was -- more than anything else it's a game of memory and anticipating your opponent's choices. Chess lends itself very well to computerized players, to the point where computerized chess is very close to being a solved problem.

In other words, Kurzweil had every reason to assume that a chess computer would beat a human in the foreseeable future. However, it's not a good analogy for brain modeling.

Posted by: Greylander | August 24, 2010 12:00 PM

KG:

We obviously mean very different things by the phrase "separating the hardware and software of the brain" (or whatever),

So what do you mean by it?

Without going back and re-reading everything, I'm not sure I'm the one who first used that phrase. In any event, the notion is at best a metaphor. I have made the point that the brain codes information in the obvious physical structure of the "wiring" of neurons by synapses. This, and similar facts, is a great source of miscommunication in this entire discussion, because people want to equate "wiring" with "hardware". Since all information must be encoded in some kind of physical structure (even in computers electrons and electromagnetic fields must move and change from one stable state to another, and that is just as "physical" as growing a new synapse) the distinction between "hardware" and "software" is irrelevant.

In principle the following could be constructed, with time money and a perverse sense of computational masochism: A CPU ("hardware") can run a binary executable program that instructs it to read and execute the instructions in a textual programming language (say Pearl) which instructs it to read and execute the instructions in another programming language (say Lisp, just for perversity) that instructed it to read and interpret a file specifying the design (logic) of an massively complex electronic circuit, generate an actual circuit layout instructions for an automated manufacture facility to construct the circuit (chip). The constructed circuit could be something that accomplishes *everything* outlined above but does it all "in hardware". Each program above may be "software", but they are physically encode by configurations of electrons and electromagnetic fields in the circuitry of the original cpu & its memory... so each stage of the process above could be regarded as "hardware" running the "software" of the next stage.

From a purely computational theoretic point of view the distinction between hardware and software is arbitrary.

but setting that aside, my 10^18 (as opposed to 10^8) is where all the information from the environment, as experienced by the developing organism, goes.

But you appeared to be claiming that very little of this was taken in before birth – that (presumably) almost all comes in the form of sensory input between birth and adulthood. That’s an error.

Recall that I did mention as soon as the nervous system forms, little pre-natal brain do begin taking in some information, such as learning some of the "sound" of their native language and so forth. My main point was that the basic structures/processes that allow the baby to do this learning are gong to be the same regardless of environment (short of toxins and other traumas), the learning itself will be encoded in things like the details ("unique snowflake") of the wiring.

... variety of nutrients ...[affect]... how many of what kinds of neurons and connections form, what the balance of neurotransmitters is, etc. Now you will say “But the rules for this are in the genome.” Not so: the genome is not a program. As I’m sure you know, multiple parts are read simultaneously...

Then where are the "rules" that govern this process? "rules" do not have to be rigid inflexible things. There will be an immense amount of flexibility and adaptability in a process governed by (loosely speaking) 100 million bytes worth of differential equations operating on untold gigawhatsabits of state-variables with an additional gigawhatsabits worth of environmental variables (not controlled by the equations themselves). Again this does not mean that the genome codes in any direct sense for the differential equations either. Short of doing simulations at the atom scale or probably quantum scale, we won't ever be able to derive those differential equations from the genome+physical laws+initial state of fertilized egg. Such simulations may be on the far side of any S-curve cap on the presently exponential growth of computing power.

However, these "rules" are an 'emergent' phenomenon governed by the genome. Think about it. The genome contains the heritable information of the organism. If the immediate environment can alter the basic rules for how an organism develops and adapts [in response to its environment] then what is it that natural selection operate on in the first place? However you slice it, and how-so-ever indirectly and 'emergently', the genome encodes the structure of the phenome including the pasticity/adaptability of the phenome in response to environmental factors. Otherwise there is nothing for natural selection to operate on. You can hypothesize other sources of heritable information, but where is your evidence?

the genome is not a program. As I’m sure you know, multiple parts are read simultaneously...

No one says (I hope) the genome is "like a program" in the sense of strict, rigidly sequential single-thread processes of ordinary computer programs. It is more like a massively multi-threaded program, with lots of randomness and non-determinacy thrown in. The "control logic" of the genome influences how fast/frequently different parts of the genome get "executed", but exactly when/how they get executed is more or less random. Much of the randomness gets washed out in aggregates -- what matters is that the cell manufactures twice as many copies of protein X as protein Y and none of protein Z, which will influence the relative quantities of signaling chemicals A, B, and C... and so on and so forth, in very complicated ways. And the relative amounts of "incoming" signaling chemicals and other environmental factors will cause epigenetic factors to change so that different levels of X,Y,Z and every other protein will be produces later.

The genome, in short, can be “read” in a huge number of different ways, and which of these is realised is not and cannot be specified in the genome itself.

The same is true of even a simple single-threaded program. Example:

X = input();
loop forever:
Y = input();
Z = X + Y;
print Z;

If the user (the environment) types 10 4 2 5 ..., the output will be 14 12 15.... But if the user starts with a different first number, say 1 4 2 5 then the output becomes 5 3 6.... We cannot say what exactly the program will do until we know the input. I'm not saying we can interpret the genome anywhere near as easily as even the most complicated binary computer programs, let alone the above example. But just because the output (details of the developed organism) can be different because of different input (environmental factors) does not mean that the genome is not analogous to a program (with all the caveats of multi-threaded, quasi random, non-determinant execution sequence and speed).

Second, the brain does not start learning at birth. It begins to learn as soon as neural impulses can pass. Why do you think babies move in the womb, and chicks in the egg? It’s not just about developing muscles, but the nervous system. The embryo also has senses of touch, taste, pressure, proprioception and hearing – so even if you exclude non-sensory information, you have to talk about the (human) brain not at birth, but at about 20 weeks, when it is a far less complex structure, being formed without learning.

All this is conceded. If I appeared to gloss over this it was in the effort to make the point about the environment not significantly (if at all) influencing the basic structure, processes, or "rules". Perhaps I muddies the water. In any event, I think the heritability/natural selection argument above makes my point clearer.

It seems to me that your claim amounts to saying that figuring out what the ~10^8 bit program is (or most of it) is just going to be too damn hard to achieve in the next 20 years.

No: it’s not a program...

Lets stop getting hung up on "program" which is a metaphor when talking about the organism. (There will be a "program" that runs the simulation, and estimating the information content of that program is part of what we've been discussing, but that is another matter presently)

...and you can’t understand it without understanding the environment in which it develops.

This is a completely different claim than that the environment adds information to the "program" (as opposed to the data operated on by the program). And no one ever claimed that we could reverse engineer the "program" (again, loosely speaking, the differential equations, or a bit more precisely the time-evolution equations) without studying how cells and cellular machinery interacts with their environment.

Here’s the Prophet Kurzweil (May He Be Uploaded) himself, p.324 of The Holy ScripturesSingularity is Near:

The straightforward brain-porting scenario involves scanning a human brain (most likely from within*), capturing all of the salient details, and reinstantiating the brain’s state in a different – most likely much more powerful – computational substrate. This will be a feasible procedure and will happen most likely around the late 2030s.

That quote looks pretty damning. I certainly appear to stand corrected, re: what some on the Kurzweil side have said. He does say "late 2030's" so he is allowing an additional ten years to reach this point. He may have been being flippant or off-the-cuff in that quote, since I have seen other quotes or paraphrases that seem to contradict this and put his timeline for "brain-porting" much further back.

However, the focus of these past two Kurzweil threads have been the relationship between information in the genome to size of executable code portion of a brain simulator, and secondarily on whether "accelerating technological advances" will get us to such a simulator by circa 2030. The original referenced article makes no mention of "brain scanning" or "uploading/copying" or whatever. And Kurzweil, in his response says "...I said that we would be able to reverse-engineer the brain sufficiently to understand its basic principles of operation..." which is quite different from the "brain-porting" mentioned above. I have kept my own discussion almost entirely in the domain of the original article, and PZ's very flawed critiques.

Kurzweil may be exceedingly over optimistic. He may have slid into quack-land when it comes preserving his health. I'm not here to defend him in particular or generally. I'm defending particular ideas.

I came up with singularty-ish ideas long before I ever heard the term or heard of Kurzweil.

The rest of what you say looks to me like pretty empty and pointless speculation – unless as the basis for an SF novel.

"Empty and pointless" is a subjective opinion. All predictions inherently carry varying degrees of uncertainty. My "predictions" are considerably more certain than, say predicting that all people on the earth will turn into pink unicorns on Jan. 1, 2012, just before a giant asteroid strikes and destroys the planet. And 2nd half of preceding example. Predicting that we will have computers approximately a thousand times faster than today's in 2020 is vastly more likely than either of the preceding. And so forth.

We have no idea what 2060-70 will be like.

Quite likely, nuclear or biological warfare, or environmental disaster, will have destroyed civilisation by then. Even if not, we may well be struggling desperately to prevent such an outcome.

The above is a prediction, based on looking at trends in industry, economy, politics, and on intuitions about human nature. You might be right. Is this a pointless and empty speculation useful only as the basis for an SF novel?

Major global catastrophes, man-made or natural would of course delay or bring an end to my timeline. I'm already accused (with merit) of being too wordy... but I try to avoid increasing my verbiage to add obvious caveats or exceptions.

If not that, we have no way of telling what direction technology will take – and this is not something that will be determined solely by what is possible.

"No way of telling" is pretty broad. We can look at present trends, economic incentives, political/social pressures, available resources, and what is in principle possible. This allows predictions that are somewhat less than a "law of nature" but somewhat more than an "educated guess or speculation".

It’s not at all obvious to me that technological change is speeding up, as it affects most people.

It is entirely obvious to me.

The only big difference between now and 50 years ago in these terms is the prevalence of computers, mobiles, and the web - ICT.

(1) a great deal more has changed. Think about this a bit more and be honest with yourself. I'm not going rattle off a hundred examples.
(2) Even the above changes you dismiss with the word "only" as if to imply "mere"... These things alone are incredibly radical changes in terms of there impact. The incredible ease with which 6,000,000 of us can now communicate is the greatest hope we have for "world peace" and cooperation in the face of potential global catastrophe.

Between 50 and 100 years ago – massive expansion in road and air travel, television, nuclear weapons, cruise and ballistic missiles, tanks, huge dams for hydroelectric power, artificial satellites, antibiotics.

The above kind of things have continued to massivly expand -- even faster, in the last 50 years.

IT has grown so fast because that’s where the combination of relatively easy gains, and R and D money, has been.

Yes economic incentives and practicalities will always matter. The universal usefulness of ever more computing power guarantees there wil always be economic incentive. As to understanding the brain ... that falls under research into human health in general, something for which there will always be strong demand and economic incentive as long as there are humans.

And note that many of the predicted advances in ITC – particularly in robotics, AI and related fields – have not happened, because they turned out to be much, much harder than the techno-optimists predicted. Almost no-one saw that the most generally useful thing you could do with a computer – was to connect it to other computers.

Some singularity fans come along and talk about how nay-sayers were wrong in the past, and the anti-singularitarians say that proves nothing.

Likewise your example of failed optimistic predictions proves absolutely nothing.

Some predictions are better than others. Some people are better than others at making predictions. Assuming we are not talking about predictions of non-chaotic systems with well known initial conditions governed by well known laws of nature, then the predictions must necessarily be qualified with some sense of "probability" and some amount of educated intuition is going to be involved. This does not mean that the predictions are pointless, uninformed, baseless "woo". There is plenty of room for smart people to wildly disagree without any of them being ignorant or irrational.

Posted by: David Marjanović | August 24, 2010 12:11 PM

KG probbly only one to read it at this point

Nope. I just found you in the list of the 10 most recent comments, so I'll try to catch up now... :-)

Posted by: Greylander | August 24, 2010 12:51 PM

#328 David Marjanović:

KG probbly only one to read it at this point

Nope. I just found you in the list of the 10 most recent comments, so I'll try to catch up now... :-)

Hadn't thought of that. I don't usually notice the "latest comment" section. Usually I follow a thread until I close the tab it's on in firefox -- hit reload once in a while when I want to see if the conversation has progressed.

I wish ScienceBlogs provide a more forum-like comment section, and an actual forum (every blog post could automatically generate a new thread in the forum -- and that thread should be one and the same as the 'comments')

Posted by: KG | August 24, 2010 2:54 PM

Greylander,

Very briefly, I think we've about reached the end of useful interaction. If you want to talk about "rules" for how the genome is read at all (which is misleading), they are distributed across the genome itself, the contents and structure of the zygote, the mother, and the external environment. The point about natural selection is irrelevant, because the genome has evolved to make use of a vast amount of external information it can "expect" to find, which you will not have in a simulation. PDEs are not good at dealing with things like the membranes and organelles of the cell, which act as catalytic sites and reconfigurable filters. It's simply bizarre to say that the software/hardware distinction is irrelevant: the former, but not the latter, consists of strings of digital information - and there isn't any of that in the brain as far as we know (even the genome is very poorly modelled as such). When I said "we've no idea" about the 2060s-70s, I was not being literal. I meant the range of possibilities is extremely wide, and probabilities cannot be usefully estimated. Kurzweil was certainly not being flip in talking about uploading in the late 2030s - it's in his magnum opus, and he says and writes similar things frequently. I've still to see anything you've said that undermines PZ's critique in any significant way.

Posted by: David Marjanović | August 24, 2010 4:32 PM

I agree that molecular encoding is a highly unparsimonious hypothesis that isn't needed to explain anything we know.

Perhaps you mean some kind of field which approximates the effect of the surrounding solvent without modeling individual molecules or atoms.

I can't imagine how that could be possible. You'll have to actually simulate the solvent. And every additional atom will increase calculation time exponentially, even under heuristic methods.

Oh, and, one more standard objection: "what about photographic memory?" Short answer: doesn't exist. There exist people who devote their whole lives to memorizing books, and, perhaps the best of these can memorize a few million words (10MB?). If photographic memory existed in any literal sense, the best memorizers in the world would be able to do way better than that.

"Photographic memory" means to remember everything one sees, not merely everything one looks at. It further means to remember not just the text of a book, but what each page actually looks like, with all colors, all letter sizes and shapes, all dog-ears, all the stuff that computers either don't bother with at all or store once for the entire book.

Furthermore, learning a book by heart isn't the same as photographic memory. Everyone who's capable of learning a short poem by heart is capable of learning an entire book by heart, provided they have nothing better to do. Photographic memory is something a few people have and everyone else just doesn't.

(OK, photographic memory isn't an either-or condition. I know a guy who can tell you where on which page of which book he read something, and says he sees the page in front of his mental eye – but he can't read it with his mental eye. The resolution is too low. But that doesn't change the fact that photographic memory isn't something you can simply train.)

I wish ScienceBlogs provide a more forum-like comment section

Why?

The genome of a fish does not specify the properties of water :)

Worse. For instance, in vertebrates, things from the plane of the first cell division to the location of the blastopore are determined by the place on the egg where the sperm happens to enter.

Posted by: KG | August 24, 2010 4:36 PM

BTW, it's odd to say that my examples of undue techno-optimism are irrelevant (OK, strictly, as you say, they "prove nothing"), because they are in exactly the area of dispute - trying to produce machine intelligence through isolating a supposed set of "basic principles".

Posted by: Greylander | August 24, 2010 10:48 PM

@ #332 KG:

Greylander,

Very briefly, I think we've about reached the end of useful interaction.

I agree. The horse's corpse has been beaten beyond all recognition.¹

If you want to talk about "rules" for how the genome is read at all (which is misleading), they are distributed across the genome itself, the contents and structure of the zygote, the mother, and the external environment.

I think this is like saying like the rules for what my little "print the sum of each new number and the first number entered" program does are distribute through the program itself but also in the person sitting at the computer entering the numbers -- actually the rules are spread out over all the people who might possibly run the program and enter numbers -- actually the rules are spread out over the whole world, because lightning could strike and kill the person there by preventing them from entering numbers, or they might get distracted by a phone call and subsequently enter a different sequence of numbers than they otherwise would have. After all, there is no way to know what sequence of numbers the program will print out until you know what numbers the user enters. I know you will disagree because you will not see my analogy as valid because the genome, cell, organism, brain, "are not software". I think you don't get it. You think I don't get it.

The point about natural selection is irrelevant, because the genome has evolved to make use of a vast amount of external information it can "expect" to find,...

I completely agree after "because...", but I find the point entirely relevant. It's a mystery to me how we can come to such radically different conclusions.

...which you will not have in a simulation.

You still mean simulation of the brain, I hope. Such a simulation *would* be provided environmental input most likely through artificial senses (i.e. convert digital video into simulated firing of rod & cone cells). There is still the matter of initial conditions, but if our model (i.e. "rules") of the brains self-modifying behavior are good, then even somewhat random intial conditions may be all we need -- then let the sim run and interact with its environment and simulated synapse will grow or prune, it may learn and start to think. With the best of whatever brain scanning we can manage and some trial and error getting the initial conditions right, I see no reason why we cannot in effect have a a starting point roughly like that of an infant amnesiac. The first simulations will no doubt be very developmentally disabled.

PDEs are not good at dealing with things like the membranes and organelles of the cell, which act as catalytic sites and reconfigurable filters.

It's simply bizarre to say that the software/hardware distinction is irrelevant: the former, but not the latter, consists of strings of digital information - and there isn't any of that in the brain as far as we know (even the genome is very poorly modelled as such).

There is nothing inherently limiting about numerical (digital) approximation to continuous values. I don't get why you (and others) keep bringing up variations of this objection. As to the genome, evidence strongly suggest that the discrete sequence from a 4-symbol alphabet is what matters in terms of how it influences cellular processes, included how it influences subsequent changes in epigenetic factors.

When I said "we've no idea" about the 2060s-70s, I was not being literal. I meant the range of possibilities is extremely wide, and probabilities cannot be usefully estimated.

I've given error bars on my own predictions that are quite large. I'll be (if I'm still alive) quite satisfied if my estimates of "when" are within a factor of three, assuming there are no major global catastrophes and socio-economic upheavals. The bets are off in the event of global mayhem. But my timetable resumes once society restabilizes in a form that permits reasonably unfettered research and development. Let's hope the neo-Catholic Inquisition does not take over... or worse, egads.. the Sharia Inquisition.

Kurzweil was certainly not being flip in talking about uploading in the late 2030s - it's in his magnum opus, and he says and writes similar things frequently. I've still to see anything you've said that undermines PZ's critique in any significant way.

I think my point by point answers here summarize our disagreement. Answering my answers will definitely just mean we're going in circles -- unless something I just said makes you go "Oh... now I get it." Somehow I don't think you are going to say that. :)

Posted by: Stephen Wells | August 25, 2010 3:20 AM

Greylander, your disagreement with KG and myself about rules seems to imply that you think the genome provides rules of the "do this, then this, then this" type and that following those rules gets you an organism.

It doesn't.

If you think of the genome as rules, they are all conditionals; each transcribed region says "if you transcribe me you get this RNA", each coding region says "if you transcribe and translate me you get this protein".

Now, if the genome is being transcribed in the context of a viable cell, all those activities will go to maintain the cell, and in the context of embryonic development they'll go towards building the embryo. If on the other hand a cell splits open and the nucleus and ribosomes and all get spilled out, they do not reassemble a new cell, they just die. The genome doesn't have the instructions for building a new cell from scratch, because for all the billions of years genomes have existed they've only ever operated inside cells.

It's a collection of subroutines with no main().

So this supposed information-theoretic limit on the complexity of the brain fails because it looks only at the information in the genome, not in the whole cell, and the cell contains information (such as that it is a cell) which is not encoded in the genome.

No eukaryote has ever reproduced by passing on only its genome to its offspring; it's always been a cell with a genome.

Posted by: KG | August 25, 2010 6:48 AM

Nope, googlemess@341, it's you who have made an ass of yourself, by quite obviously failing to read any of the thread, where Kurzweil's idiocies have been dissected.

Posted by: Greylander | August 25, 2010 11:38 PM

@ #340 Stephen Wells:

Greylander, your disagreement with KG and myself about rules seems to imply that you think the genome provides rules of the "do this, then this, then this" type and that following those rules gets you an organism.

What I have written suggests no such thing. I may wright long posts, but all the verbiage is mainly in anticipation of just this kind of simplistic misunderstanding and to explain why, in spite of agreeing with (most of) what you say about how cells/genome work, there is still good reason to think that the information theoretic argument holds true. I have to conclude that you have not read my posts, or somehow you selectively ignore much of what you read to make a sweeping assertion about what I believe about what "the genome provides", even though I have quite clearly stated the opposite.

If you think of the genome as rules, they are all conditionals; each transcribed region says "if you transcribe me you get this RNA", each coding region says "if you transcribe and translate me you get this protein".

Those on your side of the argument are fond of saying things like "the genome is not executed like a program", but something just occurred to me. If you look at what *physically* happens inside a computer, at the actual shuffling of bits and so forth, then the execution of a program does not look much like the execution of a program either. This is actually a matter of a subjective interpretation as to what "execution of a program" even means, but my point is it is all just some kind of physical process. One can get too caught up in the "program" metaphor on either side of this debate. You can also take it either too literally or not literally enough.

Now, if the genome is being transcribed in the context of a viable cell, all those activities will go to maintain the cell, and in the context of embryonic development they'll go towards building the embryo. If on the other hand a cell splits open and the nucleus and ribosomes and all get spilled out, they do not reassemble a new cell, they just die. The genome doesn't have the instructions for building a new cell from scratch, because for all the billions of years genomes have existed they've only ever operated inside cells.

It's a collection of subroutines with no main().

So this supposed information-theoretic limit on the complexity of the brain fails because it looks only at the information in the genome, not in the whole cell, and the cell contains information (such as that it is a cell) which is not encoded in the genome.

All *prior* sources of information are encapsulate in (1) and (2), while (3) provide all new information. The laws of physics (4) are ubiquitous, and there is very little information in the laws of physics, per se. I have dealt with all of these. You may not agree with the arguments I have made, but don't tell me I have simply ignored "the surrounding cell" or "the environment".

There has been a 5th proposed source: ontology, the development of the organism. It is a process whereby information from the environment, the genome, and the original egg cell, are incorporated moment by moment into the structure of the organism. But in and of itself "development" is not a source of information, it is the process by which the actual sources of information are incorporated. So the proposed 5th source "ontology" is bogus.

No eukaryote has ever reproduced by passing on only its genome to its offspring; it's always been a cell with a genome.

Posted by: Stephen Wells | August 26, 2010 2:35 PM

A quick reductio:

Everything that happens in the universe obeys the laws of nature.

The laws of nature contain little information (Greylander, 2010).

Therefore everything in the universe contains little information.

Therefore simulating the brain, a supernova, or the entire evolution of life on earth, is trivial.

Posted by: Greylander | August 26, 2010 8:49 PM

#345 Stephen Wells,

Greylander, I realise we'll never convince each other of anything...

(i) you claim there's no appreciable information in the cell apart from the genome.

In a cell, there is lots of information in the *exact* details of the position of every atom. But that is overwhelmingly the result of thermal jitter and essentially random -- there is no usable information in those details. At larger scale -- molecular and structural, there is still plenty of information. I contend however that most of that information is identical to what is in the genome. By "identical" I mean in the same sense that if you compress the same movie with two different compression algorithm and get to completely different random looking sequences of bits, both compressed files and the original movie file still contain the same "information" even though they each have completely different "data". The portion of information in the cell which is usable and not already captured in the genome is "state" information. And yes, that state information, including things like epigenetic factors is part of what determines the subsequent behavior of the cell, including further modifications to epigenetic factors and which coding portions of genome get transcribed to become proteins, and so forth.

For relative brevity, I'm not going to elaborate on my argument as to why we can neglect the state information as pertaining to the "fundamental structures and processes of the organism."

in order to know what you get from a genome you need at a minimum a complete description of the state of a functional cell, e.g. a fertilised egg, and an environment it can develop in.

I find your claim that this is not an appreciable amount of extra information

Look, I've allowed about 10^18 bytes just to encode what the brain learns from the environment -- how is that not "appreciable"?

FOr heaven's sake, a cell can produce around 100,000 different proteins, and whether or not a gene is transcribed or not could depend on the expression level of any of those proteins (you do know about stuff like repressors and feedback loops in gene expression, yes?).

Even if you reduce expression to a yes/no question that would still give 2^100000 cell states. You don't know which one you want.

(ii) do not patronise me about computerese, as if you knew more about it than I do. Trust me, those terms you quote are not at all the point I was making.

Quite often, I am sure, at this point, each of our intended meanings something quite different from the perceived meaning. We've probably exhausted the present mediums capacity for useful communication.

(iii) your claim that the laws of physics contain little information seems bizarre to me.

In any case, to infer information computationally from a cell+genome,

Posted by: Stephen Wells | August 27, 2010 6:03 AM

I'll just reiterate that no eukaryote has ever reproduced by passing on only its genome. It's always involved a cell, in a particular state, and a genome. A genome on its own doesn't contain the information to produce a cell, let alone an organism, let alone a brain, ergo there's critical information in the cell state that's not in the genome, and doing information theory about the complexity of the brain when your input is the complexity of the genome puts you on a hiding to nothing because you're putting the wrong number in at the beginning.

I think we're done here. Reality can sort out who's right :)

Posted by: https://www.google.com/accounts/o8/id?id=AItOawnFoajdqDIAo9z1Lwb6KZx0TZowUrx8bXM | August 28, 2010 3:36 PM

klewismax #343:

what the true test of Crank or Not Crank for Kurzweil will be, is wether he sticks to his fringe futurist guns, or actually works with the scientific community to reconcile his fantasies with reality.

None of his singularity talk makes any hard testable claims, it is simply a prediction that might or might not come true. In 50 or 100 years you can look back and check how far off his general prediction was, but right now you simply don't have the data.

In so far I consider it rather ridiculous to try to debunk him by attacking this or that tiny little detail of his talk, as none of that so far has any relevance to the overall picture. Can't simulate the brain by starting from the genome? Do it by neurons. Doesn't work. Do more observations. Try again with something else. etc. He doesn't claim to know the one true route to understanding the brain, he is simply saying that improved computing power will give people the tools to observe and simulate the brain in more detail.

If you want to attack him, attack the core of his theory that information processing is speeding up and that that in turn increasing development speed. If that doesn't hold up, his predictions falls apart, on the other side if it holds on, then it is not a question of "if", but simply a question of "when" we will have artificial brains, as almost any complexity can be overcome by exponential growth.

PZ Meyers debunking of Kurzweil to me just sounds to much like somebody trying to say human flight is impossible by looking at birds. He might be right in theory, but a 747 flies quite well without flapping its wings. Just because there is lots of complexity from going to genome to brain, doesn't mean that there aren't a hell of a lot of shortcuts, possible optimizations or even completly different routes creating AI.

Posted by: https://www.google.com/accounts/o8/id?id=AItOawnFoajdqDIAo9z1Lwb6KZx0TZowUrx8bXM | August 29, 2010 5:46 AM

Yes, but again, thats not actually what his claim is (should have been a bit more precise). He doesn't claim that AI will just magically happen when we have enough computing power, but that that computing power will accelerate research and brain simulation, which in turn will make it possible to create AI.

Or to put it another way: Kurzweil is not claiming that he understands the brain, quite the opposite, he knows that he doesn't understand it, but he thinks that within ~20 years exponential computing power growth will help us to research and recreate it. Unless there is some not-yet-discovered insurmountable brick-wall that we can't get past, I don't see anything wrong with that claim.

Posted by: Mr Spinoza | September 6, 2010 6:57 AM

I thought we had already back engineered the brain and called it a transmitter, silly me.

Some defectors from the ranks of scientific dogma are accepting it and helping to
make sense of it. It has been given a label now, so we can discuss it. They call it
Cellular Memory. Simply, it’s the idea that the brain doesn’t contain memories. The
brain is likened to the transistor in a radio. Music doesn’t come from the radio, it
comes through the radio. Thought doesn’t come from the brain, it comes through the
brain. In the expanded model, near-term memory and current life thinkings are stored
in the cell of the physical plant. What some think and we all should hope is that this
electro-chemical transistor we call a brain can not only send and receive data
throughout the body, but also pull in the signals that radiate outside this limited realm
of self. It could be said that there is an extent to which we, in any moment, have
“tuned in” to what is beyond us. To the extent that we develop those skills of the
receiver, our levels of connectedness and feelings of purpose will occur.
Stephen C. Parkhill

Posted by: Mark Bahner | December 16, 2010 8:38 PM

Myers: "His first point is silly."

Kurzweil: "For starters, I said that we would be able to reverse-engineer the brain sufficiently to understand its basic principles of operation within two decades, not one decade, as Myers reports."

Myers: "I don't care."

So, you don't care about the facts, or misrepresenting what someone said. Interesting start to a post.

"I suggest that we need to have a combined strategy of digging into the brain from the perspectives of physiology, molecular biology, genetics, and development, and in all of those fields I see a long hard slog ahead."

"...I see a long hard slog ahead" is not a scientific prediction.

Ray Kurzweil has stated that within two decades it will be possible to reverse-engineer the human brain. What is YOUR prediction? That it will never be done? That it won't be done in this century? That it won't be done before 2050?

What is your prediction? Or can't you do any better than "...I see a long hard slog ahead"? (Which is not at all in conflict with Ray Kurzweil's prediction of reverse-engineering the human brain by 2030.)