A Study of FWC Guidance at Three New Sites in Wyoming
Alan W. Ross and Jeff Colton
National Weather Service Forecast Office
Model Output Statistics (MOS) guidance from the Nested Grid Model (NGM) has been used for 30 years by the National Weather Service (NWS) (Glahn and Lowry 1972). In May 1994, three new NGM MOS guidance sites were added in western Wyoming. The locations were Jackson (JAC), Big Piney (BPI), and Lake Yellowstone (P60), which are located in high mountain valleys (Figure 1). MOS guidance is just one of the tools used by NWS meteorologists to prepare temperature forecasts for populated areas in western Wyoming. This study will examine the verification statistics of the NGM MOS at the three new Wyoming sites for the summer and fall of 1994. The focus is on the maximum and minimum temperature guidance, to detect, for example, any biases that may have been present.
Figure 1. Zone Map of Wyoming indicating the locations of Big Piney, Jackson and Lake Yellowstone. The three sites are located in the west-central and northwest parts of the state.
Daily maximum and minimum temperatures were recorded for each day during this study. The data was then put through a series of computations that helped to determine any biases that may have been present. The summer season (June through August) and the fall season (September through November) NGM MOS guidance minimum and maximum temperatures were compared against the actual minimum and maximum temperatures for the three locations. The only exception was the minimum temperature at Jackson (JAC). NGM MOS guidance does not compute a low temperature for Jackson. Jackson was only a 16-hour data station (6 a.m.-10 p.m.) when the new NGM MOS guidance for that location was created. Therefore, a minimum temperature forecast was not created. The data was compiled for the 0000 UTC and 1200 UTC model runs, and then stratified into the four 12-hour forecast periods that are routinely output. For this study, the perfect forecast will be defined as the NGM MOS correctly forecasting the exact temperature during the specified 12-hour period in which the forecast was valid. Whenever the observed temperature
fell below the forecasted value (by even 1°F), the NGM MOS guidance was considered to be overforecasting the temperatures (actual air temperature cooler than predicted by MOS). Whenever the temperature rose above the forecasted value, the NGM MOS guidance was considered to be underforecasting (actual air temperature warmer than predicted by MOS) the temperatures.
Some of the more significant results found during the summer season of statistical calculations included the forecast bias, the percentage of underforecasting and overforecasting, and the average errors. Figure 2a shows, by period, the percentage of time that the NGM MOS overforecasted and underforecasted the low temperature at Big Piney, Wyoming. Figure 3a displays the average error, in degrees Fahrenheit, of overforecasting and underforecasting at Big Piney by period. Figure 4a and Figure 5a show this same format, but for Lake Yellowstone (P60).
Figure 6a depicts, by period, the percentage of time that the NGM MOS overforecasted and underforecasted the high temperature at Big Piney. Figure 7a displays the average error in degrees Fahrenheit by period for Big Piney. Figure 8a and Figure 9a show these results for Lake Yellowstone while Figure 10a and Figure 11a display the results for Jackson.
Using the same method of statistical calculations for the fall season, Figure 2b shows, by period, the percentage of time that the NGM MOS overforecasted and underforecasted the minimum temperature at Big Piney, Wyoming. Figure 3b displays the average error, in degrees Fahrenheit, of overforecasting and underforecasting at Big Piney by period. Figure 4b and Figure 5b show the results for Lake Yellowstone.
Figure 6b shows, by period, the percentage of time that the NGM MOS overforecasts and underforecasts the high temperature at Big Piney. Figure 7b displays the average temperature error in degrees Fahrenheit by period for Big Piney. Figure 8b and Figure 9b depict this presentation for Lake Yellowstone, while Figure 10b and Figure 11b display the results for Jackson. A mechanical failure of the Lake Yellowstone (P60) observation site occurred from November 9 through the 30. However, the feeling was that the lack of data did not significantly influence the overall outcome of the statistics for this site.
For the minimum temperature forecasts in the summer season (June through August), there was a slight tendency for the NGM MOS to overforecast the low temperatures in all periods at Big Piney and Lake Yellowstone (Figures 2a and 4a). The average error in overforecasting the low temperature for the two locations was very similar for all periods (Figures 3a and 5a). Usually, one would expect to see a gradual increase in error magnitude from the first through the fourth period. This expected trend did occur in a few instances, but not at the frequency that one would expect.
For the NGM MOS maximum temperature forecasts in the summer season, there was a considerable bias toward underforecasting the high at all three locations (Figures 6a, 8a, 10a). The MOS forecast for Big Piney had the highest percentage of underforecasting (Figure 6a). One reason for the larger error in forecasting the 1994 summer season highs may lie in the fact that the summer was the fifth warmest in the last 100 years in western Wyoming. The average temperature was 1 to 3 above normal in this region for the summer period as reported by the Weekly Climate Bulletin (Sept. 1994).
The average error in underforecasting was quite high for Big Piney and Lake Yellowstone (Figures 7a and 9a). In fact, at Lake Yellowstone, the average error, when underforecasting both maximum and minimum temperatures were significantly larger than the average error when overforecasting. In addition, the magnitude of the error was again consistent for all periods at these two locations. Error values at Jackson (Figure 11a) were less, but they were still significant.
The results of this study were compared to a similar verification study of NGM MOS guidance from the summer season in 1992 for Cheyenne, Casper, Sheridan, Lander, and Rock Springs (Harrison and Weiland 1994). Both studies agree that an underforecasting bias of maximum temperatures does exist in the summer season. However, the minimum temperature forecast bias for these two studies was different. In this work, the NGM MOS guidance showed a slight tendency to overforecast the minimum temperatures (Figures 2a and 3a). The Harrison and Weiland study indicated that the NGM MOS guidance tended to underforecast minimum temperatures.
Interestingly, the magnitude of forecast errors did not significantly increase for the later periods of each forecast. In other words, forecast errors did not increase as one progressed toward the 48-hour forecast. Usually, the MOS guidance errors fluctuated little from the original errors discovered in the first period. This may be attributed to the statistical approach used to generate NGM MOS guidance.
For the minimum temperature forecasts in the fall season (September through November), there was a solid tendency for the NGM MOS to overforecast the low temperatures in all periods at Big Piney (Figure 2b). At Lake Yellowstone (Figure 4b), similar comparisons revealed only slight tendencies. A slight tendency to overforecast the low was observed in the second through the fourth periods, while the first period trended toward underforecasting the low temperature.
The average error in overforecasting the low temperature for the two locations was similar for all periods (Figures 3b and 5b). However, when the guidance did underforecast the lows, the average error, in degrees Fahrenheit, tended to increase over time, as during the summer season.
For the NGM MOS maximum temperature forecasts in the fall season, there was a considerable bias toward underforecasting the high at Big Piney and Lake Yellowstone (Figures 6b, 8b). However at Jackson, a slight tendency to overforecast the high temperature was discovered (Figure 10b). In the second period, a slight underforecasting did occur. This trend to overforecast the highs at Jackson in the fall was a sharp turnaround from the strong underforecasting bias of the summer maximums.
The average error in underforecasting the highs at Big Piney and Lake Yellowstone (Figures 7b, 9b) was fairly similar. There was a greater error in the later periods, as one would expect. At Jackson (Figure 11b), the average overforecast error was quite high for the second through the fourth period.
There is a bias of overforecasting the minimum temperature at two of the new NGM MOS guidance locations in Wyoming (Big Piney and Lake Yellowstone) during the first six months of usage (June through November). In addition, the NGM MOS guidance for the new locations exhibits a bias of underforecasting the maximum temperature at Big Piney and Lake Yellowstone during the six-month period. For Jackson, the high temperatures in the summer season had an underforecast bias, while in the fall season the bias turned to slight overforecasting.
Studies of the winter and spring seasons will be carried out to see if the current trends of overforecasting minimum temperatures and underforecasting maximum temperatures continue. In addition, the temporal trends will be closely monitored.
The results of this study may provide some helpful insight in forecasting the maximum and minimum temperatures at these new sites. In addition, a local adaptive computer program could be created on one of the office's workstations that would display up to the day results on how MOS is doing, compared to the actual forecast.
DOC, NOAA, 1994: Weekly Climate Bulletin, September 21, No. 94/38. Climate Analysis Center.
Glahn, H.R., and D. A. Lowry, 1972: The Use of Model Output Statistics (MOS) in Objective Weather Forecasting. J. Appl. Meteor., 11, 1203-1211.
Harrison, J., and M. Weiland, 1994: FWC Guidance in Wyoming. Central Region Applied Research Paper, 12,12-14.