FSS-based Evaluation on Convective Weather Forecasts in North China from High Resolution Models

Tang Wenyuan; Zheng Yongguang; Zhang Xiaowen

doi:10.11898/1001-7313.20180501

Vol.29, NO.5, 2018

Turn off MathJax

Article Contents

Article Navigation > Journal of Applied Meteorological Science > 2018 > 29(5): 513-523

Tang Wenyuan, Zheng Yongguang, Zhang Xiaowen. FSS-based evaluation on convective weather forecasts in North China from high resolution models. J Appl Meteor Sci, 2018, 29(5): 513-523. DOI: 10.11898/1001-7313.20180501.

Citation:

Tang Wenyuan, Zheng Yongguang, Zhang Xiaowen. FSS-based evaluation on convective weather forecasts in North China from high resolution models. J Appl Meteor Sci, 2018, 29(5): 513-523. DOI: 10.11898/1001-7313.20180501.

Citation:

PDF( 28350 KB)

FSS-based Evaluation on Convective Weather Forecasts in North China from High Resolution Models

DOI: 10.11898/1001-7313.20180501

National Meteorological Center, Beijing 100081

Received Date: 2018-04-13
Rev Recd Date: 2018-06-25
Publish Date: 2018-09-30

Abstract

Abstract

High resolution numerical model has a certain ability to predict the structure and evolution characteristics of the convection system, however, it is not easy to exploit the advantage of high resolution numerical model forecast using traditional verification metrics. Fuzzy verification is a recently developed and popular spatial verification method used for high resolution numerical model. It compares characteristics of the adjacent area of the corresponding point in prediction and observation fields to assess the accuracy of the forecast. Fuzzy verification considers a certain space and time uncertainty instead of completely accurate matching between prediction and observation.The Method of Fraction Skill Score, which is a kind of fuzzy verification, is used to evaluate the convective weather forecast ability in North China from 3 different high-resolution models(including Rapid Analysis and Forecast System GRAPES_Meso, GRAPES_3 km and East China Regional Numerical Model). Seven convective cases in North China with different organization modes caused by different weather systems from July to September 2017 are selected to evaluate the prediction ability of the small and medium scale convection system of high resolution models. Results show that the fraction skill score (FSS) can achieve valuable assessment information when the model prediction has a certain displacement and intensity of deviation. Another dominant advantage of FSS is that it can examine the model forecast skill scale, referring to the smallest window scale over which the forecast output contains useful information. The forecast skill scale is different from the scale of weather system, it represents the scale of spatial displacement deviation. The prediction of radar echo intensity from three models is weaker than the observation, East China Regional Numerical Model is the closest to the observation with echoes below 44 dBZ, and the intensity deviation from GRAPES_3 km is the smallest with echoes above 44 dBZ. Those performances lead to difference of FSS curve between GRAPES_3 km and East China Regional Numerical Model in the case of lower threshold and higher threshold. In order to evaluate the spatial displacement deviation of the model, the frequency (percentile) threshold is adopted, which represents the size change of the verification object. With increasing percentile threshold, GRAPES_3 km forecast skill scale basically maintains at 150-200 km, but the forecast skill scale of East China Regional Numerical Model increases gradually from 100 km to 400 km against the percentile threshold, which suggests that GRAPES_3 km model is more capable of predicting the small scale convective events.
- meso-scale model;
- fuzzy verification;
- fraction skill score;
- severe convective weather

FullText(HTML)

References(25)

Figures and Tables

图 1 用于理想试验构建的雷达回波带

(a)实况, (b)预报1, (c)预报2, (d)预报3

Fig. 1 Radar echo band for ideal experiment

(a)observation, (b)Forecast 1, (c)Forecast 2, (d)Forecast 3

DownLoad: Full-Size Img PowerPoint

图 2 不同阈值条件下理想试验FSS评分随窗区格点数的变化曲线

(a)阈值为35 dBZ, (b)阈值为50 dBZ, (c)阈值为第75百分位数

Fig. 2 Graphs of FSS against neighborhood length using thresholds of 35 dBZ(a), 50 dBZ(b) and the 75th percentile(c)

DownLoad: Full-Size Img PowerPoint

图 3 2017年9月21日20:00华北飑线过程雷达回波

(a)实况，(b)GRAPES_Meso 12 h时效预报，(c)GRAPES_3 km 12 h时效预报, (d)华东区域模式12 h时效预报

Fig. 3 Radar reflectivity of North China squall line at 2000 BT 21 Sep 2017

(a)observation, (b)12 h forecast from GRAPES_Meso, (c)12 h forecast from GRAPES_3 km, (d)12 h forecast from East China Regional Numerical Model

DownLoad: Full-Size Img PowerPoint

图 4 2017年8月5日14:00华北对流过程雷达回波

(a)实况，(b)GRAPES_Meso 6 h时效预报，(c)GRAPES_3 km 6 h时效预报，(d)华东区域模式6 h时效预报

Fig. 4 Radar reflectivity of North China convection case at 1400 BT 5 Aug 2017

(a)observation, (b)6 h forecast from GRAPES_Meso, (c)6 h forecast from GRAPES_3 km, (d)6 h forecast from East China Regional Numerical Model

DownLoad: Full-Size Img PowerPoint

Fig. 5 Radar echo intensity of observation and models against percentile values at 2000 BT 21 Sep 2017 during the North China squall line process

DownLoad: Full-Size Img PowerPoint

图 6 FSS评分随窗区尺度变化曲线(取第95百分位数阈值)

(a)2017年9月21日08:00起报12 h时效预报，(b)8月5日08:00起报6 h时效预报

Fig. 6 FSS against neighborhood length(threshold using the 95th percentile)

(a)12 h forecast initiating from 0800 BT 21 Sep 2017, (b)6 h forecast initiating from 0800 BT 5 Aug 2017

DownLoad: Full-Size Img PowerPoint

图 7 不同阈值条件下FSS评分随窗区尺度变化曲线

(a)30 dBZ，(b)40 dBZ，(c)50 dBZ，(d)55 dBZ，(e)第75百分位数，(f)第90百分位数，(g)第95百分位数，(h)第99百分位数

Fig. 7 FSS against neighborhood length using different thresholds

(a)30 dBZ, (b)40 dBZ, (c)50 dBZ, (d)55 dBZ, (e)the 75th percentile, (f)the 90th percentile, (g)the 95th percentile, (h)the 99th percentile

DownLoad: Full-Size Img PowerPoint

Fig. 8 Bias against threshold

DownLoad: Full-Size Img PowerPoint

Fig. 9 Forecast skill scale against percentile threshold

DownLoad: Full-Size Img PowerPoint

Table 1 Information of severe convective weather cases in 2017

时间	模式起报时间	影响天气系统	过程特点
07-11T19:00	08:00起报11 h时效	东北冷涡，地面冷锋	团状回波，雷暴大风、冰雹
07-14T22:00	08:00起报14 h时效	500 hPa短波槽，低层切变线	分散强回波，短时强降水、雷暴大风
07-15T16:00	08:00起报8 h时效	地面倒槽	分散强回波，局地短时强降水
07-21T20:00	08:00起报12 h时效	副热带高压，低层切变线	分散强回波，局地短时强降水
07-23T08:00	20:00起报12 h时效	副热带高压，低层切变线	分散对流，局地短时强降水
08-05T14:00	08:00起报6 h时效	500 hPa槽前，低层切变线	线性对流，局地雷暴大风、短时强降水
09-21T20:00	08:00起报12 h时效	蒙古冷涡，地面锋面	飑线，雷暴大风为主

DownLoad: Download CSV

References

[1]	王宁, 王婷婷, 张硕, 等.东北冷涡背景下一次龙卷过程的观测分析.应用气象学报, 2014, 25(4):463-469. doi: 10.3969/j.issn.1001-7313.2014.04.009
[2]	段亚鹏, 王东海, 刘英."东方之星"翻沉事件强对流天气分析及数值模拟.应用气象学报, 2017, 28(6):666-677. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20170603&flag=1
[3]	陈淑琴, 章丽娜, 俞小鼎, 等.浙北沿海连续3次飑线演变过程的环境条件.应用气象学报, 2017, 28(3):357-368. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20170309&flag=1
[4]	何立富, 陈涛, 周庆亮, 等.北京"7.10"暴雨β-中尺度对流系统分析.应用气象学报, 2007, 18(5):655-665. doi: 10.3969/j.issn.1001-7313.2007.05.010
[5]	Sun Juanzhen, Xue Ming, Wilson J W, et al.Use of NWP for nowcasting convective precipitation:Recent progress and challenges.Bull Amer Meteor Soc, 2014, 95(3):409-426. doi: 10.1175/BAMS-D-11-00263.1
[6]	郑永光, 张小玲, 周庆亮, 等.强对流天气短时临近预报业务技术进展与挑战.气象, 2010, 36(7):33-42. http://d.old.wanfangdata.com.cn/Periodical/qx201007007
[7]	郑永光, 周康辉, 盛杰, 等.强对流天气监测预报预警技术进展.应用气象学报, 2015, 26(6):641-657. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20150601&flag=1
[8]	王金成, 龚建东, 邓莲堂.GNSS反演资料在GRAPES_Meso三维变分中的应用.应用气象学报, 2014, 25(6):654-668. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20140602&flag=1
[9]	漆梁波.高分辨率数值模式在强对流天气预警中的业务应用进展.气象, 2015, 41(6):661-673. http://d.old.wanfangdata.com.cn/Periodical/qx201506001
[10]	郑永光, 薛明, 陶祖钰.美国NOAA试验平台和春季预报试验概要.气象, 2015, 41(5):598-612. http://d.old.wanfangdata.com.cn/Periodical/qx201505008
[11]	黄丽萍, 陈德辉, 邓莲堂, 等.GRAPES_Meso V4.0主要技术改进和预报效果检验.应用气象学报, 2017, 28(1):25-37. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20170103&flag=1
[12]	许晨璐.公里尺度GRAPES_Meso模式的动力物理性能评估与分析.北京:中国气象科学研究院, 2017.
[13]	Ebert E E, Mcbride J L.Verification of precipitation in weather systems:Determination of systematic errors.J Hydrology, 2000, 239(1):179-202. http://cn.bing.com/academic/profile?id=c791dd182c74b2779dbaf43509e0d284&encoded=0&v=paper_preview&mkt=zh-cn
[14]	Brown B G, Bullock R R, David C A, et al.New Verification Approaches for Convective Weather Forecasts.Conference on Preprints, 2004, 68(2):3.D.4-1-3.D.4-10. https://www.researchgate.net/publication/254584299_NEW_VERIFICATION_APPROACHES_FOR_CONVECTIVE_WEATHER_FORECASTS
[15]	Davis C, Brown B, Bullock R. Object-based verification of precipitation forecasts.Part Ⅰ:Methods and application to mesoscale rain areas.Mon Wea Rev, 2006, 134(7):1772-1784. doi: 10.1175/MWR3145.1
[16]	Davis C A, Brown B G, Bullock R, et al.The Method for Object-based Diagnostic Evaluation (MODE) applied to WRF forecasts from the 20055 NSSL/SPC Spring program.Wea Forecasting, 2009, 24(5):1252-1267. doi: 10.1175/2009WAF2222241.1
[17]	Duda J D, Gallus W A J.The impact of large-scale forcing on skill of simulated convective initiation and upscale evolution with convection-allowing grid spacings in the WRF.Wea Forecasting, 2013, 28(4):994-1018. doi: 10.1175/WAF-D-13-00005.1
[18]	符娇兰, 代刊.基于CRA空间检验技术的西南地区东部强降水EC模式预报误差分析.气象, 2016, 42(12):1456-1464. doi: 10.7519/j.issn.1000-0526.2016.12.003
[19]	薛春芳, 潘留杰.基于MODE方法的日本细网格模式降水预报的诊断分析.高原气象, 2016, 35(2):406-418. http://d.old.wanfangdata.com.cn/Periodical/gyqx201602013
[20]	Robert N M, Lean H W.Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events.Mon Wea Rev, 2008, 136(1):78-97. doi: 10.1175/2007MWR2123.1
[21]	Ebert E E.Neighborhood verification:A strategy for rewarding close forecasts.Wea Forecasting, 2009, 24(6):1498-1510. doi: 10.1175/2009WAF2222251.1
[22]	Ebert E E.Fuzzy verification of high-resolution gridded forecasts:A review and proposed framework.Meteor Appl, 2010, 15(1):51-64. http://cn.bing.com/academic/profile?id=be03a985db29899619998cc605b87080&encoded=0&v=paper_preview&mkt=zh-cn
[23]	潘留杰, 张宏芳, 陈小婷, 等.基于邻域法的高分辨率模式降水的预报能力分析.热带气象学报, 2015, 31(5):632-642. http://d.old.wanfangdata.com.cn/Periodical/rdqxxb201505006
[24]	李佰平, 戴建华, 张欣, 等.三类强对流天气临近预报的模糊检验试验与对比.气象, 2016, 42(2):129-143. http://d.old.wanfangdata.com.cn/Periodical/qx201602001
[25]	郜彦娜, 何立富.2011年7月12-20日华北冷涡阶段性特征.应用气象学报, 2013, 24(6):704-713. doi: 10.3969/j.issn.1001-7313.2013.06.007