Bad Math at The Weather Channel

Like this: Think of it as an index. If the index reaches maximum (which is 100) for a given spacetime (such as "The forcast for the Greenville Texas area for this afternoon) then if you go there you will get rained on. If the index is zero, you can pretty much count on it not raining there/then. As the number gets higher the chance of getting rained on goes up.

It isn't cumulative. The numbers don't have to add up.
If you plan to toss a coin tomorrow once an hour on the hour, the odds are 50% of getting heads each hour. Or, put another way, you have a 50% chance of having the coin come up heads tomorrow at any time you decide to toss it. sources: weathercast forecaster

It isn't cumulative. The numbers don't have to add up.
If you plan to toss a coin tomorrow once an hour on the hour, the odds are 50% of getting heads each hour. Or, put another way, you have a 50% chance of having the coin come up heads tomorrow at any time you decide to toss it.

Just to explain what I've done here, I've added the probability of rain in the first hour and rain in the second hour, and I have subtracted the overlapping probability of rain in both hours.

Would they still make these mistakes-after this information ?

Note also, that the temperature never gets up to 82 in the hourly forecast. I've seen that temperature discrepant by as much as 5 degrees.

But, this assumes all the hourly probabilities given are independent, which doesn't seem a very sensible assumption. If there's a high correlation between them maybe the numbers could work out.

to my understanding, precipitation predictions are based on the percent of area that will receive rain. IE, there is 100% chance of rain, but only 40% of the area will get it.

So with that in mind, the per hour prediction is "this hour, 40% of the area will get precipitation", etc etc

You're just trying to rub it in that it's nice and warm where you are and nice and cold and rainy where the rest of us are!!!

I have the same understanding as Uncle Bob. 40% chance of precipitation means that there's 100% chance of precipitation over 40% of the area covered by the forecast. This could be a different 40% of the area in the following hour.

You care enough to ask the question, but you don't care enough to spend 0.3 seconds Googling the answer?

http://www.srh.noaa.gov/ffc/html/pop.shtml

What does this "40 percent" mean? ...will it rain 40 percent of of the time? ...will it rain over 40 percent of the area?

The "Probability of Precipitation" (PoP) describes the chance of precipitation occurring at any point you select in the area.

How do forecasters arrive at this value?

Mathematically, PoP is defined as follows:

PoP = C x A where "C" = the confidence that precipitation will occur somewhere in the forecast area, and where "A" = the percent of the area that will receive measureable precipitation, if it occurs at all.
So... in the case of the forecast above, if the forecaster knows precipitation is sure to occur ( confidence is 100% ), he/she is expressing how much of the area will receive measurable rain. ( PoP = "C" x "A" or "1" times ".4" which equals .4 or 40%.)

But, most of the time, the forecaster is expressing a combination of degree of confidence and areal coverage. If the forecaster is only 50% sure that precipitation will occur, and expects that, if it does occur, it will produce measurable rain over about 80 percent of the area, the PoP (chance of rain) is 40%. ( PoP = .5 x .8 which equals .4 or 40%.)

In either event, the correct way to interpret the forecast is: there is a 40 percent chance that rain will occur at any given point in the area.

robert: well done. Glad someone was bored enough to look it up XD

Thanks everyone for all of the input. That's very informative. I agree with your points (individual hours not being independent, area being figured into the probability value) and that these would lessen the discrepancy between the daily and hourly forecasts. However, I don't think that this leaves quite a discrepancy. Firstly, although two adjacent hours would not be independent, times separated by a few hours probably are, especially given the transient nature of Texas thunderstorms. Also, if the area being considered is very, very large, then you could have a 40% chance of rain for each hour and a 40% chance of rain for the entire day. However, the area being covered in this forecast shouldn't be large enough for that to be the case.

Also, there are other discrepancies, similar to the one that Dave pointed out. For example, tonight there is a 60% chance of rain, but at midnight specifically there is a 65% chance of rain. That can't be right.

So, an important question has been answered here, and I now know that my assumption that a 40% chance of rain means that there is a 40% chance of there being rain at some point during that time period was mostly correct, this number also takes into account how much of the area is expected to receive precipitation, which I did not know. Clearly the situation is more complicated than how I describe it above, but it still appears to me that there's a discrepancy.

That is a good description, and of course correct, but how does one use this information?

Like this: Think of it as an index. If the index reaches maximum (which is 100) for a given spacetime (such as "The forcast for the Greenville Texas area for this afternoon) then if you go there you will get rained on. If the index is zero, you can pretty much count on it not raining there/then. As the number gets higher the chance of getting rained on goes up.

Thought of this way, the weather channel hourly is reasonable. You can try something outdoors, but have a backup plan. All day, across the forecast area.

But this does not fix the problem outlined in the post. Pick "my house" as the given point; if there is a 40% chance of rain during the day, and a 40% chance during any given hour, the numbers don't add up.

That said, I don't think this needs to be difficult. Most people looking up the weather forecast for the next day are thinking of some particular time (say maybe "Lunch hour - do I need an umbrella?"), so the 40% is the hourly chance for the hour which, by the Weather Channel's best guess, most people want to know about.

How the Weather Channel makes Probability of Precipitation (PoP) forecasts is unknown but it is probably a variant of how the National Weather Service makes them.

PoP is defined as "the likelihood, expressed as a percent, of a measurable precipitation event (1/100th of an inch) at any given point within the forecast area(s)".

More technically, the NWS runs a series of dynamical forecast models that move the atmosphere forward in time. Output from those models, such as relative humidity, sea level pressure, precipitable water, wind, moisture divergence, etc. along with geographic variables such as station location information, climate data and time-of-year are used in a multiple linear regression models to predict the probability of at least 0.01 inches of precipitation over some time period (usually 6-h increments). The regression models are updated regularly and are changed with the seasons.

In other words, the dynamical models are used to forecast the state of the atmosphere. Those future conditions are then statistically compared to past conditions to arrive at the PoP. So, a 40% chance of rain means that when past meteorological conditions (temp., rh, wind, etc) were similar to what is forecast, it rained 40% of the time.

A pdf of the technical document for the regression equations based on the NWS Eta model is here.

The prediction means that on all days with similar weather conditions, it will rain 40% of those days. So on all days which have the sort of incoming weather patterns, temperatures, barometric pressure, etc, that the day in question has, 4 out of 10 of them will see rain.

This is done by comparing past weather conditions over time. It's all based upon this historical data.

I think I understand this now --

Basically a 40 percent chance of rain means that 40 percent of an area will get at least X amount of rain during Y period.

Suppose Y is 24 hours, and you want to know the chance of rain during one hour. Now a 40 percent chance of rain during the hour in question means that 40 percent of that area will get at least X/24 amount of rain during that period (Y/24). I think if you pro-rate it this way the numbers all add up. And you could even have a 40 percent chance of rain for a day but 50 or 60 percent for a given hour in the same location.

You might be able to make a similar argument for temperature, but I'm not as sure about that.

"In either event, the correct way to interpret the forecast is: there is a 40 percent chance that rain will occur at any given point in the area."

There are two ambiguities here...one is: over what time period is this taking place? Although I don't think the NOAA does hourly forecasts, so they may dodge that problem. Secondly, whats the AREA? 40% chance of it raining over 100 square miles leaves me more likely to be dry than 100% chance of it raining over 1 square mile.

This has nothing to do with area! It's historical data.

http://www.livescience.com/environment/090420-rain-forecasts.html

~~~

If, for example, a forecast calls for a 20 percent chance of rain, many people think it means that it will rain over 20 percent of the area covered by the forecast. Others think it will rain for 20 percent of the time, said Susan Joslyn, a cognitive psychologist at the University of Washington who conducted the study.

The reality: "When a forecast says there is a 20 percent chance of rain tomorrow it actually means it will rain on 20 percent of the days with exactly the same atmospheric conditions," Joslyn explained.

Put another way, on that day there's an 80 percent chance there will be no rain anywhere in the forecast area.

~~~

joe: "the likelihood, expressed as a percent, of a measurable precipitation event (1/100th of an inch) at any given point within the forecast area(s)"

"Those future conditions are then statistically compared to past conditions to arrive at the PoP. So, a 40% chance of rain means that when past meteorological conditions (temp., rh, wind, etc) were similar to what is forecast, it rained 40% of the time."

Those two aren't the same thing -- maybe that's the problem with figuring out what the hell they mean. They give a gloss that doesn't match their method. The method is clear -- but it doesn't tell me the likelihood for a measurable precipitation event at any given point in the area -- it gives me a looser measure of "likelihood given the point data of meteorological conditions", not their evolution. In other words, you can have 40% every hour of the day given the methodology -- while still having 40% for the entire day; but if you're giving me "the likelihood it'll rain", that would be a non-self-consistent set that is mathematically verboten.

Wow, I didn't realize how confusing, it always made sense to me. You always have to turn the percentage around. A 40% chance of rain in a forecast area means that 60% of the time in the past with similar conditions NO rain fell ANYWHERE in the area.

There is no 40% "of the area" involved at all. If ANY RAIN falls ANYWHERE in the reporting area that is a positive outcome. So you could have 100% chance of rain and have the forecast be right and have no rain fall on you if the reporting area is big enough.

The issue with deriving a daily forecast from the hourly is still there. I don't know how they are getting the hourlies. They certainly are not independent, so you can't use any kind of combinator since the correspondence will change between every hour. You could fit all sorts of things to get the daily from the hourly depending on the assumptions.

It isn't cumulative. The numbers don't have to add up.

If you plan to toss a coin tomorrow once an hour on the hour, the odds are 50% of getting heads each hour. Or, put another way, you have a 50% chance of having the coin come up heads tomorrow at any time you decide to toss it.

That's true, Robert, but what's the chance that sometime tomorrow the coin will come up heads? If you tossed it once, it would be 50%. If you tossed it twice, it would be 75%. If you tossed it three times, it would be 87.5%. Once you've tossed it seven times, the probability of getting a coin heads up at least once is more than 99%.

#19: You're figuring the odds of getting one heads in seven tosses to be 99% but then the odds of one tails is seven tosses is 99% too. You can't treat the weather forecast like that because you'd have 99% certainty of rain and 99% certainty of not rain at the same time.

To maintain my coin analogy, I have to ask which of heads or tails comes up more often when you toss a coin seven times. In that case both heads and tails have equal 50% chance of winning (when considered prior to the first toss) regardless of how many tosses.

That's why a 50% rain chance every hour is also 50% for the whole day.

Can that be right?? It's confusing.

No, you're all wrong.

It's a statement - a prediction, followed by a percentage. The percentage governs how likely the statement is. So, if they say it will rain, they follow it by saying how confident they are: "We think it'll rain cats and dogs, and in fact, we're 40% confident that what we just said is true". At least, that's how I read things like this.

Robert, that's completely wrong (in #20). Presumably the specific measure is for there to be at least some rain during the day. Therefore rain in any hour should count as rain for the whole day, which ensures that the probability of rain for the day must always be greater than the probability for any single hour. Since the correlations between the various hours aren't likely to be perfect, the normal case should be for the daily rain probability to be significantly higher than the hourly.

The only way in which this measure makes any sense whatsoever is if it's taken as an average over the day, that is instead of 40% chance of precipitation during the day, we expect it to be raining roughly 40% of the time over the whole day.

I was wondering about the conditional probabilities while reading this. How does the observation that it didn't rain in the past hour count into the probability computation of the next hour?

If their PoP forecast changes in case it did rain in the past two hours, it means each hour the probability is re-computed based on the observations so far in the day.

Therefore rain in any hour should count as rain for the whole day, which ensures that the probability of rain for the day must always be greater than the probability for any single hour.

And now you know why those of us in the meteorology community have difficulty passing probabalistic information on to the public in any kind of meaningful way. If you were to ask 20 meteorologists what the probability of precipitation means, you'd probably get 21 different answers. Is it any wonder that the public doesn't really understand what PoPs are? BTW, Michael Hawkins in #12 has the best answer.

There's also a disconnect between what we can tell the public and what they need to hear. Imagine that you're sitting at home and want to do something like put a new roof on your house. You want to know whether it's going to rain today, yes or no. We tell you there's a 40% chance of it. There's a cost/loss issue now. Are you willing to tear off your existing roof if 4 times out of 10, rain is going to fall? How about if it's 6 times out of 10? 2 times out of 10?

One thing you aren't looking at correctly yet is that a weather "event", like a rain storm, will only happen once. When weather forcasters see conditions forming, they are not always sure exactly when and where or to what extent a storm will organize. I like the coin toss example from above, but with the following stipulations:

1) The coin will only be tossed one time. (real weather isn't 50/50, but that's irrelevant here) The point is that the "event" of tossing the coin only happens one time, and it either rains or it doesn't.

2) The coin is variable in size. It may be a very large coin that covers the entire region or a small coin with lots of holes in it.

3) The coin moves with variable speed and direction. It may arrive at various parts of the region at different times.

The forcast is a combination of those factors, and the hourly forcast is just a snapshot of how the overall forcast for the region changes over time. The rain storm isn't going to repeatedly happen every hour, so you shouldn't compound the likelyhood of rain at any given hour with the likelyhood of rain at any other hour. A weather front is a unique physical object or system. It exists in real time and space, in the physical world. It will travel over an area one time. The confidence of the forcast is based on (amongst other things) how sure they are of the time it will pass over, and how sure they are about whether it will produce rain when it passes over. Therefore, you can say that the percentage chance for rain increases and peaks at a given time of day, when it's most likely that the front will pass over, then later in the day it becomes increasingly less likely for the front to arrive, since it's likely already passed by then. The important part to remember is that a storm front is an actual physical object rather than a continuous stream of events that are all somewhat likely to happen or not.

I see no rational explanation for having a higher hourly chance than the overall chance for the day. Probably human error on that one.

it's obvious that your interpretation is wrong by just asking the simple question, what happens if we shorten the time intervals? The problem is you are assigning a different meaning to the numbers than was intended. The Weather channel information is written in a way that most people understand. The likely answer is that there is a 40% chance of rain during the day and there also exists an estimate of the statistical distribution expressing when rain is most likely to occur. The hourly graphic just illustrates the combination in a way that most people will quickly comprehend. The hours of 8, 9, 10, and 11 all fall within the period during which rain is most likely. The hours after that fall within other points in the distribution.

One thing worth considering is that these are not independent variables: If it's raining at 8 am, then odds are much higher that it's going to be raining at 9 am. And if it's not raining at 8 am, it's less likely to be raining at 9 am. So you can't just combine the 40% for 8 and the 40% for 9 and come up with 64%.

The above discussion is complicated by the introduction of a geographic area. Think about the forecast as being for a specific location (any one location in the area). A 40% chance of rain forecast for ANY time period for a given point means that 40% of the time that they make that forecast it rains at that location during the forecast time period. Statistically however, as was pointed out, a 40% chance from 8 to 9, 9 to 10, and 10 to 11 AND have a 40% chance for 8 to 11 are statistically inconsistent. But, if you have a POP of 40% from 8 to 11 and simply extract the number for each hour, it makes sense from a communications standpoint, if not a statistical one. The point about the POP being most accurately thought of as a rain index is the right one.

This seems to be what The Weather Channel is doing:

1. They assign one POP for the day and one POP for the night, using a definition that is the same as NOAA uses.

2. The summary description may restrict the POP to a portion of the day, such as afternoon showers.

3. In the hourly forecast, they assign the POP for the entire day (or night) to the hour when the rain is most likely. Then they assign the POP or a fraction of the POP to other times when the rain is equally likely or less likely, respectively.

It is generally dishonest to do this.

If you have a flight scheduled for 4 pm you want to know the probability of thunder storms at 4 pm. You don't want them automatically assigning the entire POP for the entire day to 4 pm because that could be very misleading in the event that the forecasted storms are actually predicted to be scattered randomly throughout the entire day.

In other words, I am retracting my earlier opinions. I was wrong.

I've noticed over the last few years that the wind direction and strength for a given day's prediction are based upon the 3 p.m. hourly values, rather than giving a description of changes expected through the day. Is it possible that this is what they're doing with the POP? I somehow doubt it, but it would resolve the problem if it is.

Ooops -- thought about that last post a bit more. This might explain the discrepancy, but actually makes the whole problem worse.

#26:

One thing you aren't looking at correctly yet is that a weather "event", like a rain storm, will only happen once. When weather forcasters see conditions forming, they are not always sure exactly when and where or to what extent a storm will organize.

This is correctly modeled by considering that the probabilities of rain in different hours are correlated. As I stated above, however, they're not going to be perfectly correlated, so the daily chance of rain should always be greater than the chance in any given hour.

One interpretation that seems to make all of the discrepancies go away is to take the Weather Channel's percentages as the probability that it will be raining over some small interval of time chosen at random from the interval that the percentage is given for.

So if you had a 40% chance of rain given for 8 am, that would mean that, if you choose a point in time between 8 am and 9 am, there's a 40% likelihood of it raining at that time (obviously we're choosing our times when the forecast is made, not at 8:30 when we see the sun shining). A 40% chance of rain for every hour in the day yields a 40% chance of rain for the day as a whole because, if you choose moments at random, you have a 40% chance of rain for every moment in the day. Depending on how they're rounding, I could see four 40s and eight 30s yielding a 40% chance for the day as a whole. This would also explain how the chance given for a single hour could be higher than the chance given for the day as a whole.

In short - maybe it's not the chance that it will rain at some time within the interval, but is instead the chance that it will be raining at any given time.

I note that you say that the "40% means there's a 40% chance of rain at 8 am exactly" interpretation is ludicrous, but why? Do you just mean that it doesn't actually rain for 40% of the day when they give a 40% chance? I don't find this that hard to believe. As long as we're not talking pouring rain over the whole area, but something like a drizzle over some subset of the area, that is.

Granted, I imagine that they'd put a 40% for the day if just one hour had a 40 and the rest had 10s, but I'm not seeing a forecast where they do that when I search random cities.

@24
That might happen if they update the PoP for the next day. I am assuming that the hourly PoP is being updated based on previous hour's observations whereas the PoP for the day is not. The PoP for the day can only be updated until the entire day has been observed (or defining day as past 24 hours).

Can it be that the day PoP is a prior probability and the hourly PoPs are posterior probabilities?

Forecasts for the next six hours determine POPs by what appears to be a completely different method.

If it's raining, the POP for the hour is 100% and likely the next hour is 100% also. Furthermore, they seem to use good math (or at least better math) if the rain hasn't started yet but is expected to start within six hours.

When you get past six hours in advance, they jump back to the other system where the POP for the entire day (or night) is assigned to the hour(s) with highest probability, and less probable hours get a fraction of the POP.

Surely whether the individual hours are independent or not depends on where you are? It seems reasonable that in Austin TX they are not, but here in the UK where it can drizzle on and off all day, it seems much more likely that they are independent.

BTW - the Met Office here appears to have leapt straight from "sunny intervals with a chance of showers" to animated predicted rainfall maps (http://news.bbc.co.uk/weather) without stopping at percentage probabilities in between!

I have always understood the prediction as meaning that every time the expected weather conditions occur precipitation will occur the stated percentage of the time. I have always understood further that the precise probability of precipitation at any particular point within the forecast area was not addressed at all.

Seems pretty simple, huh

I'm sorry if this has already been said. I just haven't had the patience to read all the responses... But what if the percentage is not a probability, but instead describes the percent of the surrounding land that will experience rain? E.g. 40% at 4 p.m. means that at 4:00 p.m. 2/5th of the surrounding land will be experiencing rain. It is interesting to noticed that at least in this example the percentages are bounded above by 40. I haven't checked this to see if the general principal is true.

I'm probably off base but the following explanation makes sense to me. Essentially I'm thinking of the POP as a continuous function of time. As a storm arises POP(t) increases. When it hits 50%, say, it means that any particular location is 50% likely to be experiencing rain at that moment.

This explains the discrepancy since now the daily POP number is the *maximum* of the POP curve over that day. The hourly POP number is the max of the POP curve over the hour.

I asked a meteorologist I know about this issue, and this is what he said:

I can't speak for The Weather Channel's hourly forecast, but I know that ours is pulled directly from a computer model, and I assume theirs is as well. The National Weather Service also produces an hourly forecast product. Here is the current one for Houston.

Regardless of whether or not this make sense from a statistician's perspective, the idea is to convey what hours of the forecast period have the greatest chance of rain. In this example, the National Weather Service forecaster is communicating that it's safe bet we'll have rain Friday afternoon.

And you are correct in your understanding about the rain chance. So many people make decisions based on the chance of rain, yet so many still do not have a proper understanding of what it means. As a stand alone number, it says nothing about how much of the forecast area will get rain (although that is factored in), how long it will rain, or how heavy the rain will be. It simply tells you what the chance is that at least 1/100th of an inch of rain will fall on your head for a given time period.

There is a 40% chance of rain at 8 am. If it rains at 8 am, there is a 75% chance that the rain will last longer than an hour.

If days were two hours long, that would match up perfectly to an overall 40% chance of rain as the maximum of all the hourly values.

My basic point is that the numbers are not independent coin flips, but a way to describe overall weather conditions.

This rant is based on the premise that "40% chance" of rain means "there is a 40% probability that it will rain." If that premise was true, then the rant would be correct.

But that premise is not entirely true. When meterologists say '40% chance of rain' that doesn't mean what you think it does. Most of the uncertainty comes from them not being able to nail down the location the air masses will move to. In other words, its the chaos of predicting wind direction over the coming days that is he biggest factor for why the predictions are fuzzy, rather than being unable to nail down whether it will rain. So often that "40%" comes from a calculation similar to this: "There is a 90% chance that this one particular mass of air located hundreds of miles away from you will form rain two days from now, but only a 44% chance that it will be located over your city when that happens. If that 90% chance of rain does happen, there's still a 56% chance that it will be hitting some other city miles away. Therefore the chance that YOU will get this rain is 44% of 90% = 40%."

Therefore once the rainstorm starts, the probabilities change. People who are already being rained on are likely to continue being rained on, and people whom the storm has missed are likely to continue to be missed by it, since the majority of that unpredictability of the forecast two days ago was due to not knowing the location the storm would hit.

I have always understood further that the precise probability of precipitation at any particular point within the forecast area was not addressed at all.

Can you spot the problem?

Sure: Austin is too damn humid!

I take the forcast to mean there is a 40% chance of rain on my place during the morning hours.

read #28. The hourly values are not independent values. Its not like a coin flip, it can be like taping 24 coins together and flipping them all together. The chances that a coin will be a head will be 50%, but if one coin is a head, they will all be heads.

See #30 above. I'm pretty sure if the forecast said "40% chance of rain in the morning, 60% in the afternoon" that looking at an hourly forecast is just taking the % for the hour in the time period. a snapshot or sample of the forecast that covers a larger time period.

In other words if the forecast was for partly cloudy skies in the morning and i asked you what is forecast for 10am you would say partly cloudy skies.