Issued 420 pm CST Fri Nov 16 2012
We hear all the good jokes about our accuracy. "You meteorologists are the only people that are only right half of the time and still get paid."
However, you might have noticed that our accuracy is actually pretty good and has been improving steadily in recent years. The following should give you an idea of how accurate we were in southern Wisconsin last November versus the previous 7 Novembers and how much error you can expect on a typical temperature or precipitation forecast.
High Temperature: The chart below shows average error for High Temperature Forecasts in southern Wisconsin for the month of November in 2011 (green), while the average of all forecasts in Novembers from 2004 to 2010 is shown in red. This chart shows you several things: 1) We were better nearly across the board, with the exception of Day 4 High Temperature forecasts which were about the same. 2) As you probably have noticed, we are more accurate on Day 1 with average errors around 2 degrees, but that error gradually increases to about 5 degrees on Day 7. 3) On days 3, 6, and 7, our accuracy has improved by about 1 day. In other words, our accuracy on Day 7 is now about what it used to be on Day 6.
Low Temperature: The chart below shows average error for Low Temperature Forecasts in southern Wisconsin for the month of November in 2011 (green), while the average of all forecasts in Novembers from 2004 to 2010 is shown in red. This chart shows significant improvement across the board but especially in the Day 4 to 7 extended forecast period: 1) Much like High Temperature, we are more accurate on Day 1 with an average error of 3 degrees, and that increases to about 6 degrees on Day 7, 2) Our accuracy improved by 1 to almost 2 days. Our Day 4 forecasts are now slightly better than our Day 3 forecasts used to be, and our Day 7 forecast was almost as good as our Day 5 forecasts in the past.
Probability of Precipitation (PoP): The following chart is more complex, and I won't attempt to explain it here. But it shows the same type of information as the average error for temperature. The Brier Score measures Precipitation forecast accuracy, and there has been a large improvement there as well. The Brier Score for November 2011 (green) is well below the average for the past 7 years (red). A perfect score, just like in temperature forecasts, would be zero. Our skill here has improved by a solid 2 days over the past. Notice that our Day 7 PoP forecast is more accurate now than our Day 5 forecast used to be, and our Day 4 forecast now beats the old Day 2 forecasts.
One last chart shows how well calibrated or reliable our PoP (Probability of Precipitation) forecasts are. An example of perfect calibration/reliability on a 50% chance of precipitation would be that if we added up all the forecasts of 50% and compared to what happened, half of the forecasts would have rain or snow and half would be dry.
So the charts below show, along the bottom, the PoP forecasts from 0 to 100 percent. Then, along the left axis, is the amount of times it actually rained or snowed given a particular category of PoP. The diagonal line from lower left to upper right represents perfect reliability or calibration. So ideally we want our forecast close to that line. Anything above the line has a dry bias (it rains more often that we predict), and below the line is a wet bias (it rains less often than we predict).
There has been little change in our reliability for November, as we continue to have a dry bias with PoP forecasts above 40%. What that means is that when we forecast a 50% PoP, it rained or snowed 65% of the time. When we forecast 60%, it rained/snowed 85% of the time. The good news is that the number of times that we forecast high PoPs of 50% or more increased by about ten times, yet we still had a dry bias. You may have noticed that in the extended time periods, we forecast relatively high PoPs of 50% or higher more often and it tends to be more accurate.
Jeff Craven, Science and Operations Officer
National Weather Service Milwaukee/Sullivan WI