Dichotomous Weather Forecasts Score Research and a New Measure of Score

Luo Yang; Zhao Wei; Zhai Jingqiu

Vol.20, NO.2, 2009

Article Contents

Article Navigation > Journal of Applied Meteorological Science > 2009 > 20(2): 129-136

Luo Yang, Zhao Wei, Zhai Jingqiu. Dichotomous weather forecasts score research and a new measure of score. J Appl Meteor Sci, 2009, 20(2): 129-136.

Citation:

Luo Yang, Zhao Wei, Zhai Jingqiu. Dichotomous weather forecasts score research and a new measure of score. J Appl Meteor Sci, 2009, 20(2): 129-136.

Luo Yang, Zhao Wei, Zhai Jingqiu. Dichotomous weather forecasts score research and a new measure of score. J Appl Meteor Sci, 2009, 20(2): 129-136.

Citation:

Luo Yang, Zhao Wei, Zhai Jingqiu. Dichotomous weather forecasts score research and a new measure of score. J Appl Meteor Sci, 2009, 20(2): 129-136.

PDF( 420 KB)

Dichotomous Weather Forecasts Score Research and a New Measure of Score

1.
Meteorological Center of PLA 61741 Army, Beijing 100081
2.
Meteorological Department of PLA 63880 Army, Luoyang 471003

Received Date: 2008-04-08
Rev Recd Date: 2008-12-11
Publish Date: 2009-04-30

Abstract

Abstract

The significance of forecast estim ate and the principles are discussed. It is assumed that the scores are objective; and moreover they can objectively reflect the forecast level.The scores should be comparable, guiding forecast in the right direction. Several usual methods of dichotomous forecasts score are analyzed, revealing that accuracy and critical success index (CSI) in different areas are incomparable due to the influence of event possibility. It shows that the true skill statistic (TSS) is approach to the probability of detection (POD) when forecasting rare events.When events do not appear but false alarms are made, TSS can't be calculated. Heidke skill score and Girbet skill score make up for the above weaknesses. The three skill scores are all obtained by comparing forecast with random ones, hence there are (n₁₁n₂₂-n₁₂n₂₁) in the three formulas. It can be used as the discriminant for forecast skill. When (n₁₁n₂₂-n₁₂n₂₁)> 0, it indicates that the forecast level is better than that of random forecast, otherwise, it will be worse than it. On the basis of the relationship between event probability and the difficulty of forecast, a new method of score weight is considered and proposed. The essay points out the exiting problems are resulted from improper score weight, leading the score result unreliable and not comparable. The new method of score is based on CSI, and combines CSI of the two event forecasts. The focus of score is laid upon estimating the one with smaller probability in the two events.The principles of forecast score are fulfilled. By comparative analysis, the new method is proved to be superior to other methods, especially on estimating rare events. They can reflect the forecast level and changes more accurately.The advantages are as follows :With the increase of samples, the new score tends to be more stable than other scores in the rare events fo recast, thus leading to a rapid judgement for forecast level. When forecast level is improved, the new score will be able to reflect it correctly and distinctly. The new score is objective, just and real, and is compatible for different seasons and regions. So it is a uniform standard in forecast score.
- forecast estimate;
- skill score;
- score weight;
- probability;
- comparable

FullText(HTML)

References(26)

Figures and Tables

图 1 事件A的评分权重分布示意图

Fig. 1 Score weight distribution of Event A

DownLoad: Full-Size Img PowerPoint

图 2 S_FT与I_CSA, 相互关系示意图

Fig. 2 Interrelation for S_FTtoI_CSAand

DownLoad: Full-Size Img PowerPoint

表 1 两类预报列联表

Table 1 Contingency table for dichotomous (yes/no) forecasts

表 2 两类预报评价方法

Table 2 Verification methods for dichotomous (yes/no) forecasts

表 3 Finley预报列联表

Table 3 Contingency table for Finley's forecasts

表 4 预报实例及评分研究

Table 4 Examples of forecast and score research

表 5 预报水平变化时各种评对比分析

Table 5 Comparative analysis of scores at changed forecast level

表 6 各类评分特性分析

Table 6 Characteristic analysis of scores

References

[1]	Hughes P.The great leap forward.Weatherwise, 1994, 47: 22-27.
[2]	Whitnah D R.A History of the United States Weather Bureau. Illinois:University of Illinois Press, 1961: 1-267.
[3]	Burton J.Robert Fitzroy and the early history of the Meteorological Office.BrJ Hist Sci, 1986, 19: 147-176. doi: 10.1017/S0007087400022949
[4]	Finley J P. Tornado predictions. Amer Meteor J, 1884, 1: 85-88. http://science.sciencemag.org/content/ns-4/80/126
[5]	Murphy A H.The Finley affair : A signal event in the history of forecast verification. Wea Forecasting, 1996, 11: 3-20. doi: 10.1175/1520-0434(1996)011<0003:TFAASE>2.0.CO;2
[6]	Gilbert G K.Finley's tornado predictions.Amer Meteor J, 1884, 1: 166-172.
[7]	Palmer W C, Allen R A. Note on the Accuracy of Forecasts Concerning the Rain Problem.US Weather Bureau, 1949:1-4.
[8]	Donaldson R J, Dyer R M, Krauss M J. An Objective Evaluator of Techniques for Predicting Severe Weather Events. 9th Conf Severe Local Storms, Norman, Oklahoma, Amer Meteor Soc, 1975:321-326. http://citeseerx.ist.psu.edu/showciting?cid=3237109
[9]	Schaefer J T. The critical success index as an indicator of warning skill. Wea Forecasting, 1990, 5:570-575. doi: 10.1175/1520-0434(1990)005<0570:TCSIAA>2.0.CO;2
[10]	Black T L.The new NMC mesoscale eta model:Description and forecast examples. Wea Forecasting, 1994, 9: 265-278. doi: 10.1175/1520-0434(1994)009<0265:TNNMEM>2.0.CO;2
[11]	Peirce C S.The numerical measure of the success of prediction. Science, 1884, 4:453-454. http://citeseerx.ist.psu.edu/showciting?cid=1558742
[12]	Hanssen A W, Kuipers W J A. On the relationship between the frequency of rain and various meteorological parameters. Mededeelingen en Verhandelingen, 1965, 81:2-15. http://www.worldcat.org/title/on-the-relationship-between-the-frequency-of-rain-and-various-meteorological-parameters-with-reference-to-the-problem-of-objective-forecasting/oclc/37281102
[13]	Murphy A H, Daan H.Forecast Evaluation∥Murphy A H, Katz R W.Probability, Statistics, and Decision Making in the Atmospheric Sciences. Westview : Westview Press, 1985: 379-437.
[14]	Flueck J A. A Study of Some Measures of Forecast Verification. 10th Conf Probability and Statistics in Atmospheric Science, Edmonton, Alberta, Amer Meteor Soc, 1987: 69-73.
[15]	Doolittle M H.The verification of predictions.Amer Meteor J, 1885, 2:327-329.
[16]	Doolittle M H.Association ratios. Bull Philos Soc Washington, 1888, 10:83-87;94-96.
[17]	Heidke P.Berechnung des Erfolges und der Gǜte der Windstärkevorhersagen im Sturmwarnungdienst (Calculation of the success and goodness of strong wind forecasts in the storm warning service). Geogr Ann Stockholm, 1926, 8: 301-349.
[18]	Murphy A H, Winkler R L.A general framework for forecast verification.Mon Wea Rev, 1987, 115:1330-1338. doi: 10.1175/1520-0493(1987)115<1330:AGFFFV>2.0.CO;2
[19]	Murphy A H. What is a good forecast? An essay on the nature of goodness in weather forecasting.Wea Forecasting, 1993, 8:281-293. doi: 10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2
[20]	张军, 葛军, 田俊杰, 等.概率天气预报及其应用.北京:气象出版社, 1998: 127-137.
[21]	丁金才.天气预报评分方法评述.南京气象学院学报, 1995, 18(1):143-150. http://www.cnki.com.cn/Article/CJFDTOTAL-NJQX501.021.htm
[22]	新田尚, 立平良三, 市桥英辅.宁松, 译.最新天气预报技术.北京:气象出版社, 1997: 145-146.
[23]	Brownlee K A.Statistical Theory and Methodology in Science and Engineering (2nd ed). John Wiley and Sons, 1965.
[24]	Doswell C A Ⅲ, Davies-Jones R P, Keller D L.On summary measures of skill in rare event forecasting based on contingency tables. Wea Forecasting, 1990, 5:576-585. doi: 10.1175/1520-0434(1990)005<0576:OSMOSI>2.0.CO;2
[25]	赵有娟, 赵康林, 王小虎.涿州地区冬半年辐射雾的预报.军事气象水文, 2005, (6) : 32-33.
[26]	宋伟, 雷显飞, 宋作义.东北平原夏季雷雨预报.航空气象, 2005, (2): 31-34.