Projection / Prediction

Posted by William M. Connolley on August 23, 2007
(19)
More »

I’m now hopelessly confused about the distinction between climate projection and prediction.

I used to be happy with what I thought was the case: that given the range in model results, and no good way of knowing the best, calling them predictions seemed too precise; so use a weaker word like projection instead. But.

The IPCC glossary says “A climate prediction or climate forecast is the result of an attempt to produce a most likely description or estimate of the actual evolution of the climate in the future, e.g. at seasonal, interannual or long-term time scales”

That isn’t a very good definition, because its near meaningless. Indeed, it appears to make the outcome dependent on the intention of the researcher(s) producing the runs.

And “A projection of the response of the climate system to emission or concentration scenarios of greenhouse gases and aerosols, or radiative forcing scenarios, often based upon simulations by climate models. Climate projections are distinguished from climate predictions in order to emphasise that climate projections depend upon the emission/concentration/ radiative forcing scenario used, which are based on assumptions, concerning, e.g., future socio-economic and technological developments, that may or may not be realised, and are therefore subject to substantial uncertainty.”

I read that as saying that all that distinguishes pred from proj is knowning the forcing scenario; if we knew future GHG (and solar, and volcanic) accurately, we would call them predictions. That doesn’t seem right either.

Over at RC, discussing the recent Smith 10-y forecasts, Gavin says “the kinds of simulations used in AR4 are all ‘projections’ i.e. runs that attempt to estimate the forced response of the climate to emission changes, but that don’t attempt to estimate the trajectory of the unforced ‘weather’.

This is at least a meaningful distinction – predictions attempt to forecast the actual trajectory of the ‘weather’. Or it would be meaningful if I was sure what ‘weather’ meant in this context. I presume not actual weather, but year-to-year fluctuations, including El Nino etc. But then thats not really “climate” prediction, since climate averages out the ‘weaher’.

Anyone have a more exact definition?

(19)
More »

Comments

  1. #1 Lubos Motl
    2007/08/23

    I agree with IPCC that a typical climate prediction these days is a subjective attempt of an excited, blinded, but ultimately confused person to create a piece of science fiction based on Freudian projections.

  2. #2 fergus
    2007/08/23

    Nothing like your blog, then, Lubos.:)

  3. #3 fragment
    2007/08/23

    It would be nice to clear it up succinctly for us lay folk. The scenario aspect is one feature that makes for projections, but perhaps the lack of detailed initial conditions is another? From reading in the blogosphere about the Smith et al work I was getting the impression that initialising with ocean heat content data was the main advance.

  4. #4 Gavin
    2007/08/23

    I don’t think this is as complex as you make it out to be. Basically, there are three different things going on and it makes sense to give them different names.

    Forecast:
    What you think will happen in the future (could be probabilistic), but with no conditionals. Used in weather forecasts, sales forecasts etc.

    Prediction
    A much broader category of scientific statement, that might apply to undiscovered information about the past, present or future, but that implies a complete specification of the circumstances under which X would be expected. The anticipated result of a well designed lab experiment is a prediction, the prediction of general relativity concerning Mercury etc. Predictions are the mainstay of the scientific method.

    Projection
    A conditional prediction about the future. i.e. if a certain set of circumstances come to pass, the climate will respond so. This is not a forecast since no probability for the specific scenario can be given, but it is a prediction since, should those circumstances occur, your prediction can be tested.

    Thus the AR4 runs were projections, the latest Smith et al paper talked about forecasts. A statement like ‘precip increases 1-3% per degree C warming’ is a prediction.

    Clearer?

  5. #5 James Annan
    2007/08/23

    I used to like Gavin’s projection/prediction distinction but I’m not sure it is universally used (by climate scientists). Eg Trenberth was grumbling about initialisation not so long ago, which isn’t relevant to the distinction.

  6. #6 Gavin
    2007/08/23

    I disagree, Initialisation is fundamentally part of a forecast, not a projection. Trenberth’s point was that without initial condition from observations, you couldn’t even begin to talk about proper forecasts. Of course, true forecasts can’t take you very far – a couple of weeks for the atmosphere, seasonal to potentially a few years using the ocean, but after that, all you have a projections.

  7. #7 Dano
    2007/08/23

    I agree with Gavin.

    In ecology, we presume to start from initial conditions and proceed from there. Projections in scenario analysis are formulated for different reasons and usually for different ideas (for management solutions), whereas predictions are to validate one idea.

    Best,

    D

  8. #8 James Annan
    2007/08/23

    Actually, on re-reading, you’ve (Gavin) effectively defined prediction and projection as the same thing – a (probabilistic) expectation conditional on xyz. If not, what is the distinction you are trying to make? I’m happy with the idea of some sort of unconditional (probabilistic) statement versus one that depends (perhaps implicitly) on other conditions (which could be converted to the first sort if one assigns probabilities to the conditions), but I’m not sure where a 3rd category can come in.

    I don’t see why a forecast is not permitted beyond a few years – I would be very confident in predicting/forecasting a climatological distribution for the weather on Christmas day 2015 in London, or the result of 100 coin tosses in 2050 :-)

  9. #9 Kooiti Masuda
    2007/08/24

    There is a semantic issue about “climate”.
    The Climate Prediction Center of U.S. NOAA makes prediction of year-to-year variability, e.g. the next El Nino event. I guess that this is a typical example of “climate prediction” in English-speeking meteorological community, though I remember it was called “long-range weather prediction” around 1980.
    In Japanese there is a word “tenko” (with long “o”) between “tenki” (weather) and “kiko” (climate), and what CPC does is certainly prediction of “tenko”, though Japan Meteorological Agency nowadays call it just (Japanese equivalent of) “seasonal prediction”. I learned that in German there is a word “Grosswetter” between “Wetter” and “Klima”.

    But perhaps this issue can be set aside if we consider prediction vs. projection of “climate change”.

    My understanding about that is similar to the paragraph “I read that…” of original posting.
    I think that climate change prediction must include some probabilistic estimate of GHG emissions. Thus it requires a kind of prediction about the society. But if the result is not sensitive to this part, the method of this may be crude. As I understand, some of my colleagues think that the pathway of climate in coming 30 years is not so sensitive to emission scenarios within plausible range of evolution of the society, so it is possible to make useful prediction by including the social system in a very crude way.

    Initialization of the state of the physical climate system (especially those parts of the ocean and the cryosphere whose response time may be decadal) is desirable for the climate change forecast, but I think that its priority is lower than the “enveloping” the emission pathways.

  10. #10 Roger Jones
    2007/08/24

    We had a go at these in Working Group II Chapter 3 in the TAR and Chapter 2 in AR4.

    From Box 2.1, projection is defined as:

    Projection. A projection is generally regarded as any description of the future and the pathway leading to it. However, here we define a projection as a model-derived estimate of future conditions related to one element of an integrated system (e.g., an emission, a
    climate, or an economic growth projection). Projections are generally less comprehensive than scenarios, even if the projected element is influenced by other elements. In addition, projections may be probabilistic, while scenarios do not ascribe likelihoods.

    Scenarios. A scenario is a coherent, internally consistent, and plausible description of a possible future state of the world. Scenarios are not predictions or forecasts (which indicate outcomes considered most likely), but are alternative images without ascribed likelihoods of how the future might unfold. They may be qualitative, quantitative,
    or both.

    I don’t mind the IPCC definition of the prediction or forecast because it implies that a forecast will be made fit for a particular purpose. Given the uncertainty with climate change, the researcher should clearly state what the forecast is fit for and why.

    The projection definition from the IPCC Glossary is messy – giving terms climate-specific definitions is problematic when they have wider useages (vulnerability and adaptation are two such terms), so we have tried to be as widely relevant in our definitions in Chapter 2 as possible.

    I take Gavin’s meaning of prediction and forecast – and we find in the literature many references to model predictions – however, these should not be mistaken for real world predictions. This is most often where the communication breaks down, because they are used interchangeably and specialist meanings overlap with the colloquial.

    Figure 2.4 is an attempt by the Chapter 2 team to characterise a range of such tools – they overlap and all have a valid place in climate research.

    These meanings are becoming more important because most adaptation has to deal with the whole of climate including variability, requiring properly initialised forecasts (or Bayesian methods accounting for all known variables that matter), whereas mitigation, especially when assessing the benefits of avoided damages, is more interested in the difference between impacts associated with reference and policy scenarios. The impacts are assessed using model projections of greenhouse gas scenarios.

    Any suggestions for clearer paths through this difficult terrain always welcome!

  11. #11 Fergus Brown
    2007/08/24

    Loath as I am to speak up in such exalted company, nonetheless, how about this?

    A prediction is a statement about future conditions which specifies what will happen under a given set of variables. It generally should have a strong truth value but is inferentially weak. For climate purposes, it could be considered as the projection with the highest probability.

    A projection is a statement about what could happen, given a range of variables which include at least one non-predictable (or stochastic) component, to which (Bayesian?) probabilities can be assigned. It should have a weaker truth value but stronger inference than a prediction.

    A scenario is the given set of conditions or variables which pertain to a projection or prediction. Some variables in a scenario could be ‘open’, but if so, a probability value or range of values must be assigned to them. Statements about scenarios contain a dubious truth value, as they are invariably conditional, but should, ideally, be based on sound observations and logical inferences (though I understand this is significantly different to the way the term is used in the IPCC, for example).

    I am not sure how to distinguish between a forecast and a prediction. Is it simply a matter of time span?

    Noticing Roger’s comment about the fitness of forecasts, I am wondering whether this has been done for the examples in the AR4?

    Regards,

  12. #12 William
    2007/08/24

    An interesting discussion, thank you all (with one possible exception :-)

    The IPCC defn equates climate forecasts and predictions. Gavin makes predictions conditional forecasts (at least I think that is what “implies a complete specification of the circumstances under which X would be expected” means). The IPCC can perhaps be forgiven for not taking too much care over this, since they make neither predictions or forecasts.

    But then Gavin defines projections as conditional predictions, which makes them conditional conditional forecasts. I still think this is a mess.

    Also, I’ll repeat my earlier point: the IPCC defn of proj says “…climate projections depend upon the emission/concentration/ radiative forcing scenario used… and are therefore subject to substantial uncertainty.” This appears to assert that most of the uncertainty comes from the emission scenario, which downplays model uncertainty far too strongly. Just for example (and this appears to be the issue that Hansen for one thinks is most importance), the amount of SLR expected from Greenland has an uncertainty dominated by our lack of understanding of the physics, not by emissions.

    Roger says “we find in the literature many references to model predictions – however, these should not be mistaken for real world predictions. This is most often where the communication breaks down, because they are used interchangeably and specialist meanings overlap with the colloquial” and he is likely right, but this is a Bad Thing because its going to confuse people. This can be justified if the specialist term “prediction” has a clearly defined meaning, but I’m not sure it has.

    Ah, and Gavin: if “I disagree, Initialisation is fundamentally part of a forecast, not a projection” is correct, would you modify your definitions to include it?

  13. #13 Luboš Motl
    2007/08/24

    All these distinctions are unphysical. A prediction in science is always the same thing and the additional words, forecast or projection, are only used to create fog and give a credibility to a prediction that wasn’t made sufficiently carefully.

    Concerning the terminological details, forecast is meant not to care about the probabilities of various outcomes or error margins. A projection is a prediction based on simple statistical methods such as linear regression and simple extrapolation: a projection is supposed to be a simple prediction or forecast that only uses a very small amount of numbers.

    Neither of these linguistic comments changes the criteria that determine whether a prediction is correct or not, justified or not, and whether it was formulated accurately enough or not. By quoting the alleged differences, people like Gavin Schmidt are only doing propaganda: they try to sell bad things as good things. Whether something is called a forecast, prediction, or projection has nothing to do with the question whether it’s trustworthy.

    And that’s the memo.

  14. #14 Gavin
    2007/08/24

    Maybe I wasn’t clear, each of the definitions are predictions in the very broad scientific sense. But since colloquially, prediction is synonymous with forecast (in the narrow sense defined above), it is worth defining projection as a special sub-class of prediction, conditional upon an uncertain future scenario and of course only good for the forced response.

    [Then I find it hard to fit this in with the distinction you made over at RC, about projections being distinguished by not following the tracjectory -W]

  15. #15 Luboš Motl
    2007/08/24

    Dear Gavin, what you write has a reasonable core and one may define words in any way she wants.

    But what really distinguishes projections from predictions (certainly in the context of economics) is not whether one assumes certain conditions – projections are used even if there is only one available scenario – but rather the fact that projections are extrapolations calculated from very simple models and formulae, most typically a linear extrapolation. In the context of the climate, projections are called projections because they assume a simple trend – i.e. a linear increase of CO2 in the atmosphere (or CO2 emissions per year).

    This linear extrapolation is arguably the reason why we call it a projection: we use a linear, straight beam of “light” to shine upon future.

  16. #16 Luboš Motl
    2007/08/24

    I wonder whether it is Gavin or William who has more courage to say that they actually agree with me, risking that they will look like traitors in the eyes of Michael Mann et al. ;-)

    [Fear not, Lubos, viewed from a scale that allows us to see the difference between you and me, Gavin and I are indistinguishable. Its like trying to look for extrasolar planets, only now is the discrimination becoming available... -W]

  17. #17 Luboš Motl
    2007/08/25

    Dear William, finally I can agree with you again. Both of you are living in a different galaxy and from this viewpoint, you’re indistinguishable.

  18. #18 mugwump
    2007/08/28

    The distinction between prediction and projection is straightforward:

    A projection is a conditional statement: X will happen if Y. It is ok for X to be probabilistic, eg X = “The distribution over delta T in 2100 is 3 +- 1.5 C” and Y = “CO2 doubles”.

    A prediction removes the conditional, usually by substituting Y with its most likely value, eg: “CO2 will double, therefore the distribution over delta T in 2100 is 3 +- 1.5 C”.

    If you like, a prediction is the maximum likelihood projection.

    At least, that’s how those of us in normal science would define it. Since climatologists like to redefine even the basic rules of science [1], I wouldn’t be at all surprised if you have your own kooky definition.

    [1] Eg, the recent argument by Gavin that unlike every other scientific field, climatology is better served by not providing enough information (source code) for people to reproduce analyses.

  19. #19 Isaias
    2011/02/18

    While predicting the weather depends critically on getting the initial state of the atmosphere right, predicting the climate does not. Which is not to say that climate prediction is easy. It’s not. sources: weathercast forecaster