Intractable problems?

Hmmmmmm....measuring gravity at the µm and sub-µm distance scale?

I say: Bose-Einstein condensate/atomic laser - in free fall, with some suitable geometry involving dense substances.
Just a metric problem.

Measuring how fast ultracold neutrons fall in a vacuum. You make 'em ultracold with a liquid helium bath, but getting them from that to the vertical vacuum chamber with essentially zero lateral velocity is a trick I haven't solved.

You can drop cold muons in a vacuum, but they decay in a very small fraction of an inch.

These come from a paper of mine about particles with imaginary component to rest mass -- traditionally used for decaying particles with exponent proportional to half-life, but also need to meaure if imaginary component affects coupling to gravity and other fields.

Others: distinguishing Higgs boson events from other things in LHC data.

Measuring quadrupole polarization of gravity waves.

Searching for cosmological background radiation signature of putative extra dimensions roilling up into Calabi-Yau manifolds early after Big Bang, making the universe look merely 4-D afterwards.

Mapping the space of all possible economic systems. (paper I had my 17-year-old son present for me at Wolfram conference 2006).

Searching the genome for evidence of previous genetic codes (from the putative RNA world and before).

Coupling the Human Metabolome Data Base to the Human Genome Data base with automated generation of the appropriate system of nonlinear differential equations and automated lookup of online literature (though Leroy Hood is more than halfway there).

High-resolution simulation of a single living cell.

This just scratches the surface of open hard problems I've posed in hundreds of papers.

I could go on... But what's the use? I don't have any grad students and post-docs to hand-off tough subproblems to. So I just publish hundreds, thousands (literally) of times and hope a few get picked up by others. Some do.

I don't think science is anywhere near solving the problem of how to make the person who took the last cup of coffee to make a new pot.

The coffee pot problem is indeed intractable, but it is a "soft-science" problem, more amenable to economic incentives and other distasteful instruments of social policy.

Hm: I thought combining Doppler converters with collimation could produce ultracold neutrons with very low transverse velocity spreads?

I was also under the impress the Higgs signature was expected to be relatively simple as these things go in proton colliders.

Measuring quadrupole polarization in gravitational radiation is easy, if you detect them with enough S/N in the first place. Two non-parallel L shaped detectors will independently extract the + and x polarization modes and these trivially reconstruct the quadrupole.

There are certainly ideas for searching the CMB for signatures of extra dimensions, though they at some level boil down to "looking for anomalous power at particular frequencies".

Don't even know where to begin mapping all possible economic systems - that may be a truly intractable problem, depending on the definition of an economic system and whether economics is powerful enough a system that you could do arithmetic with it...

The search for evidence of an RNA world or equivalent is definitely proceeding, although any evidence will be obscure at this stage. Still, there are some interesting hints in iRNA and in recent exploration of the detailed mechanism of transcription that seems to be hinting at something.

The linking of metabolism/protein production to the genome is happening as fast as they can soak up resources - and, I have to say, from what I've seen of the implications for our understanding of ourselves and ability to engineer ourselves, it is shit scary and totally fantastic.

Simulation of a cell is just technology limited, clearly doable with enough resources and time.

Not bad: clearly a couple of potentially hard problems there ;-)

Insightful comments, Steinn Sigurdsson.

I've spent my adult life (say ages 15-55) stretched at great tension between addiction to intractable problems and the compromise of publishing 10,000,000 words about teeny-weeny pieces of them, or very shallow approaches to attacking the unreachably deep parts of search space in them (word = 6 alphanumeric characters including spaces, divided by 6, as standard in book/magazine/newspaper metrics).

Too bad they don't give professorships and grants based on size of paper trail of search expended to date!

Results. Everyone wants results.

"Abandon Hope All Ye Who Enter, Unless Ye Have Good Data!"

See also the novel: "Hard Problems" by Ian Watson. I got mentioned in the acknowledgments.

Yeah, there is an aphorism that to get grants you have to have done the research already and then use the new grant to do what you want to do next to get the grant to retroactively do that.
There is a grain of truth to that.

Not, of course that I would ever do such a thing, except in so far as it is permitted by the disgression of the cognizant funding officer of the appropriate federal agency...

But, getting resources to do problems is only partially correlated with ability to do problems, and both are only partially correlated with ability to identify problems.
Some people are just good at all the parts, bastards. Others are lucky, or find synergistic collaborations - some of the best have good combinations of problem finders, problem solvers and resource acquirers.

In fact good science teams are like B-movie world war two squads: they have a hothead, a grandfather sergeant who has seen it all, and a scrounger; and a fresh faced lieutenant who may either die nobly or come-of-age and/or a fresh faced newbie grad student who serves as narrator.

BTW: don't know if you remember, but we had dinner at least twice, back when Brin gave annual talks at Caltech. Good crowd back then.

Steinn Sigurdsson: I do remember. I keep in touch with Brin. Had him give a keynote address at the 6th International Conference on Complex Systems, Boston, June 2006, where I also ran sessions with speakers including (to list SF-related) Dr. Stan Schmidt, Dr. Geoff Landis, Dr. Mary Turzillo, Dr. Marvin Minsky (hey, he co-authored a novel with Harry Harrison), and John Forbes Nash, Jr. ("A Beautiful Mind" was a science fiction movie, right?)

Whom do you recommend as Plenary Speakers for ICCS-2007?

World War Pi: Science Versus Ignorance.

Sarge: "Brooklyn, put down that knish. O'Reilly, shove that hip-flask back in your khakis. Sparks, get that packet-switched line. Radar, where the hell is that box of spare parts for the spectrometer? Gramps, where's heinlein when we need him? Okay, boys, let's charge out of the trench when I give the signal. C'mon, men, do you wanna live forever?"

Even statements like lets exclude from consideration problems that could be solved if only someone would fund a bigger (telescope/computer/accelerator/????) are too exclusive for the category. There are those problems for which a somewhat better/more expensive piece of equipment would be sufficient, then there are those for which with known methods only totally unrealistically better equipment would do the job. I would consider the latter to also be "grand challenge" caliber.

Ok: I disagree - for example, to resolve and image the disk of an Earth like planet that is, say, 100 light years away, is not possible with the technology we have right now.
But, we understand conceptually how to do it, and I don't think there are any fundamental difficulties in actually doing it - constructing 100 km optical interferometers with 100m individual optically smooth mirrors is not intractable, it is just hard.
Similarly, I think we know how to accelerate particles up to very high energies, until we reach fundamental limits where, for example, scattering off background photons becomes a limiting factor.
We do not know how to do so as efficiently as we would like, and we know the cost would be very very high, but conceptually we know just how to do it. It would just cost a lot.

A lot of our big science problems are limited by resources, not our lack of understanding. This is quite unlike the situation 50-100 years ago.

Of course it is the unknown unknowns that will get us.

I'm waiting for the In-Situ Galactic Halo Explorer (ISGHEX), myself. Unfortunately, I will have to wait a very long time.

I suppose that a probe capable of returning magnetic field information from the Earth's core would be too much to ask for? How about igniting a helium flash under controlled conditions? Accelerating a baseball to the Planck energy? Creating or stopping a hurricane?

Computing pi to the last digit...? ;)

ISGHEX we know how to do, it is just that with current technology it would take a gigayear or two, and probably a factor of more than a million increase in industrial capability.

Stevenson has a concept for a probe to the Earth's core - sounded conceivably doable, although I don't know that it'd be worth it.

We definitely think we know how to ignite a helium flash, all that requires is enough mass and some time. A more interesting issue is the scientific question of what determines the burnup in the nth helium flash and whence the structure along the horizontal branch!
That might be hard.

Baseball to Planck energies - you mean Planck energy per nucleon? 'Cause otherwise it is easy...
Tricky - we could probably get it to 0.0001 Planck energies, but then ablation would be
intrinsically limiting - could we dissociate it and just accelerate the bits?

Creating a hurricane is just engineering - might require some experimentation to get the right tweaking of water temperature and air circulation - stopping it is even easier, just put a large dissipative structure in its path.

Computing pi is mathematics. Not science. Boooring.

How about solving the general many-body problem of strongly interacting fermions (electrons)? That's a tough one, in large part because of the problem of antisymmetry of the total many-body wavefunction. While there are approximate approaches, the general strong correlation problem is unsolved. Heck, we don't even know if the Hubbard model, a very simplified model of the cuprate superconductors, actually has a superconducting state as its ground state.

What _is_ consciousness, what is the purpose of the mechanism, how is it implemented in the neural substrate, and how common is some form of it in different species?

That's more like it: consciousness is definitely one of the big ones - probably a harder problem than the set of all possible economic systems, maybe.
It should also be amenable to scientific analysis and work is being done on it, but it is not clear we are even asking the right questions, yet.

Strongly interacting many-body systems is another hard one, particularly if you allow long range forces or correlations, although the spin lattices are quite hard enough.
Although for those there is a cop out - we can always construct analog solutions, rather than model the system, just build it and see what it does. No predictive power, but it makes it more analogous to the issue of the helium flash.

If you're going to allow many-body systems then you should also allow 3-d non-local-thermal-equilibrium radiation magnetohydrodynamics. Yes, we could solve it with a big enough computer - _The Sun_ manages to solve the equations - but we're still stuck with varying degrees of slightly-screwy approximations.

Umm...I generally think that just about any scientific problem is solvable if enough money, time, and effort are invested in it. But in the meantime, the world is in sore need of a process by which stupidity and/or apathy could be converted into usable energy.

Isn't that what coffee is for?

Seriously though - I suspect some scientific problems are intrinsically intractable - that is we may not be able to solve them with arbitary resources and time. I'm not sure which problems might be like that, the most likely class are formal computational problems that reduce to NP, and certainly those harder than NP
Which of course begs a couple of questions - like whether P=NP and whether any "scientific" problem (as distinct from a toy math problem) is interestingly NP or more complex than NP.

There are clearly hard scientific problems out there, but I'm hard pressed to identify any that are intractable. At least any interesting ones.

There are definitely some problems where right now we just don't quite know how to approach, but we think we could if we were a bit cleverer - holography, information problem in black hole evolution, consciousness, consistent definition of and mechanism for abiotic origin of life, uniqueness of RNA/DNA life, uniqueness of CHON life etc
But all of the look doable in the finite future.

As for 3D nonLTE rad MHD.
Hah! This here very week I hope to look upon non-equilibrium special relativistic 3D MHD, and contemplate other peoples' hard work to make it fully relativistic with radiative transport.
Easy piecey.
Couple of clever grad students (or couple of dozen) and we're there. Ish.

Measuring how fast ultracold neutrons fall
in a vacuum. You make 'em ultracold with a
liquid helium bath, but getting them from
that to the vertical vacuum chamber with
essentially zero lateral velocity is a
trick I haven't solved.

Easy: just move the vacuum chamber with the same lateral velocity as the neutron :)

How to identify when a solution to a problem is "true." How to identify what "true" means.

Don't forget to consider:

Open Questions in Physics
John Baez
December 23, 2005
http://math.ucr.edu/home/baez/open.questions.html

While for the most part a FAQ covers the answers to frequently asked questions whose answers are known, in physics there are also plenty of simple and interesting questions whose answers are not known. Here we list some of these. We could have called this article Frequently Unanswered Questions, but the resulting acronym would have been rather rude.

Before you set about answering these questions on your own, it's worth noting that while nobody knows what the answers are, a great deal of of work has already been done on most of these subjects. So, do plenty of research and ask around before you try to cook up a theory that'll answer one of these and win you the Nobel prize! You'll probably need to really know physics inside and out before you make any progress on these.

The following partial list of open questions is divided into five groups:

* Condensed Matter and Nonlinear Dynamics
* Quantum Mechanics
* Cosmology and Astrophysics
* Particle Physics
* The Big Question

Yeah, Baez's page is a good resource, I'm also fond of
Joe's Questions: http://www.personal.psu.edu/users/j/m/jmp456/unsolvedproblems.htm

Baez reminds me of a big one no one mentioned yet - turbulence...

AT also raises an interesting if somewhat more metaphysical point.
I'd extend it and say "how do we know when a problems is true?"