NOAA

National Weather Service Weather Forecast Office

Dodge City, KS

• Home
• Site Map
• News
• Organization
• Search for: NWS All NOAA 

P1.5 17th Conference on Weather Analysis and Forecasting, 13-17 September 1999, Denver, Colorado

VERIFICATION OF EXTENDED PERIOD FORECASTS AT NWS DODGE CITY KS

Scott M. Reiter⁽¹⁾ and Steven M. Hunter

NOAA/National Weather Service

Dodge City, KS

1. INTRODUCTION

NWS Weather Forecast Office (WFO) Dodge City had been producing 3-5 day extended forecasts for nearly 18 months when our users recommended that we expand these forecasts to include days 6 and 7. This raised questions regarding forecast skill for this period. Could we forecast as well as current medium range models? If so, how much could we improve on them? To answer this, we developed a verification scheme using Quick Basic programs. A spreadsheet was incorporated to do various mathematical and statistical calculations. We began forecasts for days 6 and 7 on April 15^th 1998, but it was not until August 1^st 1998 that they were appended to zone forecasts. We compared forecasts with those of the Global Spectral Medium Range Forecast (MRF) model-based objective statistical guidance FMR. Information on the MRF model may be found in Kanamitsu (1989) and Kanamitsu et al. (1991). Livingston and Schaefer (1990) evaluated MRF performance in the 3-5 day period.

Since the FMR for Dodge City KS (FMRDDC) is issued once per day based on the 0000 UTC model run, only one forecast per day was verified. This study also compares forecaster performance to climatology. The comparison period was one year - May 1^st 1998 through April 30^th 1999. Verification of temperatures and probability of precipitation (POP) was for Dodge City only.

---

2. METHOD

Initially, data were manually entered into a spreadsheet. This included manual recording of FMR data on the sheet and forecaster information. Later, to aid ease of entry, a coded format was implemented for the forecaster to enter his/her temperatures and POP forecast values. Finally a Quick Basic program was written that automatically extracted FMR and forecaster information and then wrote it to a file. The data were imported into the spreadsheet and analyzed.

The FMR forecasts high temperatures for the period 1200 UTC through 2400 UTC and lows for 0000 UTC through 1200 UTC. Forecasters also made their temperature prognostications for these periods.

POP forecasts from the FMR are based on the 24 hour day starting at 0000 UTC and ending at 2400 UTC, and were averaged to the nearest ten percent to coincide with forecaster resolution constraints. Measured precipitation was recorded for within this period. Measurable precipitation is defined as an amount greater than a trace, or 0.25mm. Precipitation amounts less than this were set to zero.

Forecaster, FMR and climatology temperature errors were calculated for each forecast period, separated for lows and highs. Forecaster improvement over the FMR was calculated by subtracting the absolute error of the forecaster from absolute error from the FMR, dividing the result by the maximum error between the two:

% IMPRV = {ERR_FMR - ERR_FCSTR} / {MAX ERR _{(TFMR,TFCSTR)} } x 100

We used the same method for calculating forecaster improvement over climatology.

The data were stratified by month and season. Seasonal definitions were summer (June-August), fall (September-November), winter (December-February), and spring (March-May). Since the principal experiment was for days 6-7, the data for this period were analyzed separately from those of days 3-5.

We performed a paired Student's T test on the error data sets to determine if there were significant differences between them. The Student's T test compares population means and yields a probability that their differences did not occur due to chance. The populations in this case consist of FMR errors and forecaster errors. The test used error data from the spreadsheet. Both populations had an equal number of members and a two-tailed test was applied. A probability of less than 5 percent was deemed significant.

Separate POP averages were calculated for days with measurable precipitation and those days without such precipitation at Dodge City. Averages for both POPs and temperatures were calculated for the combined forecast period, day 3-7. Results were partitioned by month and season.

---

3. RESULTS

3.1 Forecaster vs FMR temperatures

Overall forecaster temperature improvement over the FMR for the 12 month period was -1.1%, i.e., a slight degradation vs the guidance. The Student's T test showed this to be a statistically significant figure. The figure for low temperatures was -0.2% and it was -1.6% for highs. Only the value for highs was found to be significant, however.

During the summer season for days 3-5, (Fig. 1) the forecaster improved over FMR output by 10.9% for high temperatures, but there was negative improvement for lows (-7.1%). Values for days 6-7 were not statistically significant.

During the fall and winter seasons, forecasters significantly improved over the FMR for low temperatures in both 3-5 day and 6-7 day periods. For high temperatures in the 6-7 day period there was a significant -2.7% degradation in fall and -3.4% in winter.

For the day 3-5 period in spring, there was a -4.4% degradation for lows and a 1.7% improvement for highs. There were no significant differences for days 6-7.

3.2 Forecaster vs Climatology Temperatures

Overall forecaster temperature improvement over climatology for days 3-7 was 21%, with 17.8% for lows and 23% for highs. All these were statistically significant. Figure 2 shows days 3-5 and 6-7 error comparisons vs climatology. This exhibits a large variation in forecaster improvement over climatology for days 3-5 and only small improvements for days 6-7. Differences for day 3-5 low for summer and day 3-5 high for fall and spring were not significantly different. However, for days 6-7, all differences were significant except for highs in winter and spring.

3.3 POP forecasts

The FMR consistently output slightly higher POP's than the forecaster in each season for days on which precipitation occurred. On days with no precipitation, however, the forecaster provided smaller POPs (Fig. 3). The average FMR POP values were close to climatology for precipitation-free days. Forecasters consistently generated POP's less than climatology on such days during all seasons, even more so during winter and spring.

---

4. SUMMARY AND CONCLUSIONS

The data show that WFO Dodge City's 6-7 day temperature forecasts were significantly better than the FMR for just low temperatures during fall and winter. These improvements were 2.5% and 5.9%, respectively. All other 6-7 day forecasts were either degradations or not different with statistical significancy.

The lack of improvement in the summer is not surprising, since mesoscale atmospheric motions become increasingly important during that season. These motions are highly unpredictable in the extended period.

We would expect improvement to occur in the winter, since medium-range models continue to improve, providing guidance for the extended period that may be modified by forecaster knowledge of local weather patterns. This expectation must be tempered by the fact that migratory synoptic-scale pressure systems in the cool season produce larger temperature deviations from climatology, yielding greater possibilities for error. The results presented here for winter are mixed, with improvement for lows and degradation for highs (a trend repeated in the fall). We are uncertain how to interpret this finding.

Another factor that might impact this study would be a long period of abnormal temperatures. For example, 1999 had one of the warmest Februaries on record. As a result, the forecasters and the FMR had a cold bias during that month.

The FMR incorporates a progressive bias toward climatology values from day 1 through day 7, with an 80% bias at day 7. Fig. 2 shows that there is much less forecaster improvement over climatology in days 6-7 relative to days 3-5. This we take as an indication that forecasters are more closely following the guidance for the longer period, which results in the proximity to climatology.

The FMR also incorporates an increasing bias toward climatology in its POP forecasts. The consistently lower (than FMR) POPs by the forecaster suggests either that the forecasters were not swayed by the FMR forecast or that he/she is less confident in mentioning precipitation in late periods. In either event, the forecasters' dry forecasts verify better than the guidance in these periods.

This study represents a first foray into day 6-7 forecast verification for an NWS field office. The sampling period of one year is so small that we do not recommend drawing any far-reaching conclusions about the viability of forecasting for such an extended period. We plan to extend the data collection period for at least two more years to obtain a more representative sample, one less influenced by any "anomalies" of the one year evaluated. We also wish to stratify the results by individual forecasters, to identify any significant biases therein.

We recognize that Dodge City's weather is different from many other U.S. climate regimes. Its high plains location, near the center of the country only 430 km from the Rocky Mountains, gives it a very continental climate characterized by frequent rapid meteorological changes. We endorse analogous verification tests at NWS offices in other climates.

Finally, assuming we have identified any deficiencies in forecaster performance at long periods, we offer possible remedies. Many forecasters require much more training on long wave patterns that are critical in determining the weather in such periods. We also feel that more information on medium range model biases would help improve forecasts.

Acknowledgments. Thanks to Art Meunier of WFO Riverton WY and Larry Ruthi and Ryan McCammon of WFO Dodge City. We are also indebted to Tom Hicks of CWSU Fort Worth TX.

REFERENCES

Kanamitsu, M., 1989: Description of the NMC global data assimilation and forecast system. Wea. And Forecasting, 4, 335-342.

------, J.C. Albert, K.A. Campana, P.M. Caplan, D.G. Deaven, M. Iredell, B. Katz, H.L. Pan, J. Sela, and G.H. White, 1991: Recent changes implemented into the global forecast system at NMC. Wea. and Forecasting, 6, 425-435.

Livingston, R.L. and J.T. Schaefer, 1990: On Medium-Range model guidance and the 3-5 day extended forecast. Wea. and Forecasting, 6, 361-376.

Mendenhall, W., 1971: Introduction to Probability and Statistics. Duxbury Press, Belmont, CA, USA, Third Ed.

---

1. Corresponding author address: Scott M. Reiter, National Weather Service, 104 Airport Road, Dodge City, KS, 67801; email Scott.Reiter@noaa.gov