设为首页
加入收藏
Evaluation of Weather Forecasts from AI Big Models over East Asia
-
摘要: 针对人工智能气象大模型的500 hPa位势高度、2 m气温、10 m风速、降水以及热带气旋路径等, 从定性和定量两个角度进行评估。结果表明:从定性角度出发, FuXi、Pangu和GraphCast 3个大模型均会响应热带异常加热, 其中Pangu与GraphCast响应强度接近, FuXi响应较弱。从定量角度出发, FuXi整体展现出更高的预报能力, 其最大可用预报日数超过9.75 d, Pangu和GraphCast分别为8.75 d和8.5 d。在2 m气温预报中, FuXi的时间异常相关系数为0.48~0.91, Pangu和GraphCast分别为0.43~0.91和0.38~0.83。此外, 采用TS(threat score)评分对FuXi和GraphCast降水预报进行评估, FuXi在晴雨、小雨和中雨预报中更具优势, 其预报技巧分别为0.22~0.41、0.15~0.24和0.06~0.22, GraphCast在大雨预报中展现更强能力。针对2019年7月29日华北暴雨和2020年8月16—17日乐山暴雨两次极端降水个例进行分析, FuXi和GraphCast均可提前8 d预报降水的空间分布, 但在降水量级预报中存在偏差, 随着预报时效减小, 偏差也逐渐减小。在热带气旋路径预报中, Pangu展现更高精度。Abstract: Reliable medium-range weather forecasts are crucial for both science and society. Although weather predictions primarily rely on numerical weather models, the artificial intelligence (AI) weather big models have shown potential for accurate weather forecasts with reduced computational costs. However, prediction skills of big models remain uncertain, particularly in East Asia, which limits further application of weather AI models. To systematically evaluate predictive capabilities of Pangu, FuXi, and GraphCast models over East Asia, their prediction results are focusing on 500 hPa geopotential height, 2 m air temperature, 10 m wind speed, precipitation, and track of tropical cyclones.ECMWF reanalysis V5 (ERA5) datasets are utilized to provide the initial conditions for big models, and to assess their predictive skill. Additionally, precipitation observations and China Meteorological Administration tropical cyclone datasets are utilized to access big models as well. FuXi shows the highest forecasting skills among 3 big models for 500 hPa geopotential height. The forecast from FuXi is reliable for up to 9.75 days, while the forecasts from Pangu and GraphCast are reliable for 8.75 days and 8.5 days, respectively. For 2 m air temperature forecasting, FuXi presents higher skills with an averaged temporal anomaly correlation coefficient (TCC) ranging from 0.48 to 0.91, while TCCs of Pangu and GraphCast are 0.43-0.91 and 0.38-0.83, respectively. Among 3 models, only FuXi and GraphCast provide precipitation forecasts. FuXi shows higher prediction skill compared to GraphCast in forecasting precipitation, light rain, and moderate rain; however, GraphCast has advantage in heavy rain forecast. As the lead time increases, the threat scores (TSs) of FuXi for rainfall, light rainfall and moderate rainfall are 0.22-0.41, 0.15-0.24 and 0.06-0.22, respectively. The model demonstrates higher skill in the northern and southeastern regions of China. For predicting the track of cyclones, Pangu model demonstrates superior predictive skill. As the lead time increases from 6 hours to 240 hours, biases of Pangu's prediction track increase from 17.5 km to 1850 km.The study focuses on the prediction skill of various AI big models through TCC, spatial anomaly correlation coefficient, and TS. Generally, the performance of FuXi is superior for most elements. And reasonable evaluation of AI model is helpful for the development of AI models.
-
图 1 冬季500 hPa位势高度对黑色实线所示区域内0.1 K的异常热带加热在不同积分时间(5 d、10 d和20 d) 的响应
Fig. 1 500 hPa geopotential height responsed to tropical anomalous heating of 0.1 K within region outlined by the black line integrated for 5 d, 10 d and 20 d in winter
图 2 东亚区域500 hPa位势高度预报空间异常相关系数随时效变化
Fig. 2 Spatial anomaly correlation coefficients of 500 hPa geopotential height forecast over East Asia
图 3 不同时效(5 d和10 d) 500 hPa位势高度的时间异常相关系数和标准化均方根误差
Fig. 3 Temporal anomaly correlation coefficients and normalized root mean square errors of 500 hPa geopotential height forecast over East Asia at lead times of 5 d and 10 d
图 4 不同时效(5 d和10 d) 2 m气温与10 m风速预报偏差
Fig. 4 Error distributions of 2 m temperature and 10 m wind speed forecasts at lead times of 5 d and 10 d
图 5 不同时效(5 d和10 d) 2 m气温预报的时间异常相关系数和标准化均方根误差
Fig. 5 Temporal anomaly correlation coefficients and normalized root mean square errors of 2 m temperature forecast at lead times of 5 d and 10 d
图 6 不同时效(5 d和10 d) 10 m风速预报的时间异常相关系数和标准化均方根误差
Fig. 6 Temporal anomaly correlation coefficients and normalized root mean square errors of 10 m wind speed forecast at lead times of 5 d and 10 d
图 7 FuXi和GraphCast晴雨、小雨、中雨和大雨的TS评分随预报时效变化
Fig. 7 TS of rainfall, light rainfall, moderate rainfall, and heavy rainfall forecast based on FuXi and GraphCast at different lead times
图 8 FuXi和GraphCast不同时效预报与观测强降水(大于7.5 mm)
Fig. 8 Heavy rainfall (greater than 7.5 mm) forecast and observation from FuXi and GraphCast at different lead times
图 9 基于台站观测数据的2019年7月29日华北暴雨总降水量与2020年8月16—17日乐山暴雨总降水量
Fig. 9 Heavy rainfall observations of North China on 29 Jul 2019 and of Leshan from 16 Aug to 17 Aug in 2020
图 10 2019年7月29日华北暴雨FuXi和GraphCast不同起报时刻降水预报偏差
Fig. 10 Rainfall forecast biases initiated at different time based on FuXi and GraphCast during heavy rainfall of North China on 29 Jul 2019
图 11 2020年8月16—17日乐山暴雨FuXi和GraphCast不同起报时刻降水预报偏差
Fig. 11 Rainfall forecast biases initiated at different time based on FuXi and GraphCast during heavy rainfall of Leshan from 16 Aug to 17 Aug in 2020
-
[1] 沈学顺, 王建捷, 李泽椿, 等.中国数值天气预报的自主创新发展.气象学报, 2020, 78(3):451-476.Shen X S, Wang J J, Li Z C, et al. China's independent and innovative development of numerical weather prediction. Acta Meteor Sinica, 2020, 78(3): 451-476. [2] 曾庆存. 天气预报——由经验到物理数学理论和超级计算. 物理, 2013, 42(5): 300-314.Zeng Q C. Weather forecast-From empirical to physico-mathematical theory and super-computing system engineering. Physics, 2013, 42(5): 300-314. [3] Ebert-Uphoff I, Hilburn K. The outlook for AI weather prediction. Nature, 2023, 619(7970): 473-474. doi: 10.1038/d41586-023-02084-9 [4] 马雷鸣. 天气预报中的人工智能技术进展. 地球科学进展, 2020, 35(6): 551-560.Ma L M. Development of artificial intelligence technology in weather forecast. Adv Earth Sci, 2020, 35(6): 551-560. [5] 曾小团, 邹晨曦, 范娇, 等. 基于GRU深度学习的短时临近降水预报订正方法. 应用气象学报, 2024, 35(5): 513-525.Zeng X T, Zou C X, Fan J, et al. Short-term precipitation correction based on GRU deep learning. J Appl Meteor Sci, 2024, 35(5): 513-525. [6] 李丹, 林文, 刘群, 等. 机器学习在人工增雨效果统计检验中的应用. 应用气象学报, 2024, 35(1): 118-128.Li D, Lin W, Liu Q, et al. Application of machine learning to statistical evaluation of artificial rainfall enhancement. J Appl Meteor Sci, 2024, 35(1): 118-128. [7] 尹晓燕, 胡志群, 郑佳锋, 等. 利用深度学习填补双偏振雷达回波遮挡. 应用气象学报, 2022, 33(5): 581-593.Yin X Y, Hu Z Q, Zheng J F, et al. Filling in the dual polarization radar echo occlusion based on deep learning. J Appl Meteor Sci, 2022, 33(5): 581-593. [8] Wu P, Chang X T, Yuan W Y, et al. Fast data assimilation(FDA): Data assimilation by machine learning for faster optimize model state. J Comput Sci, 2021, 51. DOI: 10.1016/J.JOCS.2021.101323. [9] Bonavita M, Laloyaux P. Machine learning for model error inference and correction. J Adv Model Earth Syst, 2020, 12(12). DOI: 10.1029/2020MS002232. [10] Zhang H, Wang Y Q, Chen D D, et al. Temperature forecasting correction based on operational GRAPES-3 km model using machine learning methods. Atmosphere, 2022, 13(2). DOI: 10.3390/atmos13020362. [11] 谢舜, 孙效功, 张苏平, 等. 基于SVD与机器学习的华南降水预报订正方法. 应用气象学报, 2022, 33(3): 293-304.Xie S, Sun X G, Zhang S P, et al. Precipitation forecast correction in South China based on SVD and machine learning. J Appl Meteor Sci, 2022, 33(3): 293-304. [12] 胡莹莹, 庞林, 王启光. 基于深度学习的7~15 d温度格点预报偏差订正. 应用气象学报, 2023, 34(4): 426-437.Hu Y Y, Pang L, Wang Q G. Application of deep learning bias correction method to temperature grid forecast of 7-15 days. J Appl Meteor Sci, 2023, 34(4): 426-437. [13] Nearing G, Cohen D, Dube V, et al. Global prediction of extreme floods in ungauged watersheds. Nature, 2024, 627(8004): 559-563. doi: 10.1038/s41586-024-07145-1 [14] 庄潇然, 郑玉, 王亚强, 等. 基于深度学习的融合降水临近预报方法及其在中国东部地区的应用研究. 气象学报, 2023, 81(2): 286-303.Zhuang X R, Zheng Y, Wang Y Q, et al. A deep learning-based precipitation nowcast model and its application over East China. Acta Meteor Sinica, 2023, 81(2): 286-303. [15] 张林, 吴蕾, 李峰, 等. 基于深度学习的天气雷达异常数据识别技术. 应用气象学报, 2023, 34(6): 694-705.Zhang L, Wu L, Li F, et al. Indentification of weather radar abnormal data based on deep learning. J Appl Meteor Sci, 2023, 34(6): 694-705. [16] 袁凯, 李武阶, 庞晶. 基于决策树算法的鄂东地区冰雹识别技术. 应用气象学报, 2023, 34(2): 234-245.Yuan K, Li W J, Pang J. Hail identification technology in eastern Hubei based on decision tree algorithm. J Appl Meteor Sci, 2023, 34(2): 234-245. [17] 赵琳娜, 卢姝, 齐丹, 等. 基于全连接神经网络方法的日最高气温预报. 应用气象学报, 2022, 33(3): 257-269.Zhao L N, Lu S, Qi D, et al. Daily maximum air temperature forecast based on fully connected neural network. J Appl Meteor Sci, 2022, 33(3): 257-269. [18] 刘海知, 徐辉, 包红军, 等. 机器学习分类算法在降雨型滑坡预报中的应用. 应用气象学报, 2022, 33(3): 282-292.Liu H Z, Xu H, Bao H J, et al. Application of machine learning classification algorithm to precipitation-induced landslides forecasting. J Appl Meteor Sci, 2022, 33(3): 282-292. [19] Reichstein M, Camps-Valls G, Stevens B, et al. Deep learning and process understanding for data-driven Earth system science. Nature, 2019, 566(7743): 195-204. doi: 10.1038/s41586-019-0912-1 [20] 熊国玉, 祖繁, 包云轩, 等. 基于LSTM和先验知识的高速公路路面温度预报. 应用气象学报, 2024, 35(1): 68-79.Xiong G Y, Zu F, Bao Y X, et al. Expressway pavement temperature forecast based on LSTM and prior knowledge. J Appl Meteor Sci, 2024, 35(1): 68-79. [21] Zhang Y C, Long M S, Chen K Y, et al. Skillful nowcasting of extreme precipitation with NowcastNet. Nature, 2023, 619(7970): 526-532. doi: 10.1038/s41586-023-06184-4 [22] Bi K F, Xie L X, Zhang H H, et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature, 2023, 619(7970): 533-538. doi: 10.1038/s41586-023-06185-3 [23] Chen L, Zhong X H, Zhang F, et al. FuXi: A cascade machine learning forecasting system for 15-day global weather forecast. NPJ Clim Atmos Sci, 2023, 6. DOI: 10.1038/s41612-023-00512-1. [24] Lam R, Sanchez-Gonzalez A, Willson M, et al. Learning skillful medium-range global weather forecasting. Science, 2023, 382(6677): 1416-1421. doi: 10.1126/science.adi2336 [25] Weyn J A, Durran D R, Caruana R. Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere. J Adv Model Earth Syst, 2020, 12(9). DOI: 10.1029/2020MS002109. [26] Rasp S, Thuerey N. Data-driven medium-range weather prediction with a resnet pretrained on climate simulations: A new model for weather bench. J Adv Model Earth Syst, 2021, 13(2). DOI: 10.1029/2020MS002405. [27] 肖海霞, 张峰, 王亚强, 等. 基于生成对抗网络和卫星数据的云图临近预报. 应用气象学报, 2023, 34(2): 220-233.Xiao H X, Zhang F, Wang Y Q, et al. Nowcasting of cloud images based on generative adversarial network and satellite data. J Appl Meteor Sci, 2023, 34(2): 220-233. [28] Kurth T, Subramanian S, Harrington P, et al. FourCastNet: Accelerating Global High-Resolution Weather Forecasting Using Adaptive Fourier Neural Operators//Proceedings of the Platform for Advanced Scientific Computing Conference. Davos Switzerland. ACM, 2023: 1-11. [29] Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010. [30] Zhou J, Cui G Q, Hu S D, et al. Graph neural networks: A review of methods and applications. AI Open, 2020, 1: 57-81. doi: 10.1016/j.aiopen.2021.01.001 [31] Ben Bouallègue Z, Clare M C A, Magnusson L, et al. The rise of data-driven weather forecasting: A first statistical assessment of machine learning-based weather forecasts in an operational-like context. Bull Am Meteor Soc, 2024, 105(6): E864-E883. doi: 10.1175/BAMS-D-23-0162.1 [32] Hersbach H, Bell B, Berrisford P, et al. The ERA5 global reanalysis. Quart J R Meteor Soc, 2020, 146(730): 1999-2049. doi: 10.1002/qj.3803 [33] Sun S, Shi C X, Pan Y, et al. Applicability assessment of the 1998-2018 CLDAS multi-source precipitation fusion dataset over China. J Meteor Res, 2020, 34(4): 879-892. doi: 10.1007/s13351-020-9101-2 [34] Ying M, Zhang W, Yu H, et al. An overview of the China Meteorological Administration tropical cyclone database. J Atmos Ocean Technol, 2014, 31(2): 287-301. [35] Lu X Q, Yu H, Ying M, et al. Western North Pacific tropical cyclone database created by the China Meteorological Administration. Adv Atmos Sci, 2021, 38(4): 690-699. [36] Zeng J H, Yuan X, Ji P, et al. Effects of meteorological forcings and land surface model on soil moisture simulation over China. J Hydrol, 2021, 603. DOI: 10.1016/j.jhydrol.2021.126978. [37] Hakim G J, Masanam S. Dynamical tests of a deep learning weather prediction model. Artif Intell Earth Syst, 2024, 3(3). DOI: 10.1175/AIES-D-23-0090.1. [38] Hoskins B J, Karoly D J. The steady linear response of a spherical atmosphere to thermal and orographic forcing. J Atmos Sci, 1981, 38(6): 1179-1196. [39] 沈学顺, 陈起英, 孙健, 等. 中央气象台全球中期数值预报业务系统的发展. 气象, 2021, 47(6): 645-654.Shen X S, Chen Q Y, Sun J, et al. Development of operational global medium-range forecast system in National Meteorological Centre. Meteor Mon, 2021, 47(6): 645-654. [40] 中国气象局. GB/T 38308—2019天气预报检验台风预报. 北京: 中国标准出版社, 2019: 2-3.China Meteorological Administration. Weather Forecast Verification-Typhoon forecast. GB/T 38308-2019. Beijing: Standards Press of China, 2019: 2-3. [41] Jiang Q, Li W Y, Fan Z D, et al. Evaluation of the ERA5 reanalysis precipitation dataset over Chinese Mainland. J Hydrol, 2021, 595. DOI: 10.1016/j.jhydrol.2020.125660. -
下载:
计量
- 摘要浏览量: 372
- HTML全文浏览量: 71
- PDF下载量: 139
- 被引次数: 0
