Visibility Forecast Based on PhyDNet-ATT Deep Learning Algorithm

Zhu Yuying; Zheng Yu; Zhang Bei

doi:10.11898/1001-7313.20240603

Vol.35, NO.6, 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Applied Meteorological Science > 2024 > 35(6): 667-679

Zhu Yuying, Zheng Yu, Zhang Bei. Visibility forecast based on PhyDNet-ATT deep learning algorithm. J Appl Meteor Sci, 2024, 35(6): 667-679. DOI: 10.11898/1001-7313.20240603.

Citation:

Zhu Yuying, Zheng Yu, Zhang Bei. Visibility forecast based on PhyDNet-ATT deep learning algorithm. J Appl Meteor Sci, 2024, 35(6): 667-679. DOI: 10.11898/1001-7313.20240603.

Zhu Yuying, Zheng Yu, Zhang Bei. Visibility forecast based on PhyDNet-ATT deep learning algorithm. J Appl Meteor Sci, 2024, 35(6): 667-679. DOI: 10.11898/1001-7313.20240603.

Citation:

Zhu Yuying, Zheng Yu, Zhang Bei. Visibility forecast based on PhyDNet-ATT deep learning algorithm. J Appl Meteor Sci, 2024, 35(6): 667-679. DOI: 10.11898/1001-7313.20240603.

PDF( 6899 KB)

Visibility Forecast Based on PhyDNet-ATT Deep Learning Algorithm

DOI: 10.11898/1001-7313.20240603

Zhu Yuying^{1, 2
,
,},
Zheng Yu^{1, 2},
Zhang Bei^{1, 2}

1.
Nanjing Joint Institute for Atmospheric Sciences, Nanjing 210041
2.
Key Laboratory of Transportation Meteorology, CMA, Nanjing 210041

Received Date: 2024-07-04
Rev Recd Date: 2024-09-03
Publish Date: 2024-11-30

Abstract

Abstract

NWP (numerical weather prediction) and statistical methods still have limitations in forecasting low visibility. Therefore, enhancing nowcasting techniques is crucial for ensuring the safety of daily life and industrial activities. A short-term visibility forecast model in Jiangsu Province PhyDNet-ATT-VIS is established based on PhyDNet-ATT (physical dynamics network with attention) deep learning method, using high spatial and temporal resolution ground observations and NWP data. Ground observations contain 1 min average horizontal visibility from automatic weather stations in Jiangsu Province, as well as its upstream regions of Anhui Province, Henan Province, and Shandong Province. NWP data are derived from PWAFS (Precision Weather Analysis and Forecast System) gridded meteorological forecast data. Temporal resolution and spatial resolutions of PWAFS forecast products are 1 h and 3 km, respectively. Due to PhyDNet-ATT-VIS's outstanding ability to handle nonlinear problems, visibility forecast from 6 h to 18 h with a spatial resolution of 3 km and a temporal resolution of 1 h is achieved, and model results are tested and evaluated. The visibility forecasting product of ECMWF (European Centre for Medium-Range Weather Forecasts) is selected to evaluate and compare the forecasting skills with that of PhyDNet-ATT-VIS. The initial forecast time of ECMWF is 0800 BT and 2000 BT every day, with the time interval of 3 h and the spatial resolution of 0.125°×0.125°. Compared to ECMWF visibility product, the rootmean square error (RMSE) and mean absolute error (MAE) of PhyDNet-ATT-VIS are reduced by 201% and 310%, respectively. Across different visibility levels, the probability of detection (POD) significantly improves, while the false alarm ratio (FAR) substantially decreases. Threat score (TS) of the forecast demonstrates a clear advantage, although the model's ability to predict low visibility (defined as visibility less than 0.2 km) in 15 h to 18 h range still requires further improvement. In terms of spatial distribution, the forecast error for visibility in lake and coastal areas is significantly lower than that in other regions for NWP. For PhyDNet-ATT-VIS, the error in areas with dense observation sites is significantly lower than that in regions with sparse observation sites. Compared to ECMWF model, PhyDNet-ATT-VIS can more accurately predict key characteristic parameters, such as affected areas, intensity, onset, and dissipation, in both regional and local fog processes. This study can provide a reliable and operationally replicable reference for improving short-term forecasting skills in visibility.
- visibility nowcast;
- deep learning;
- fog

FullText(HTML)

References(43)

Figures and Tables

Fig. 1 Distribution of mean absolute error and root mean square error for different forecast models

DownLoad: Full-Size Img PowerPoint

Fig. 2 Visibility forecast assessment for PhyDNet-ATT-VIS and ECMWF

DownLoad: Full-Size Img PowerPoint

Fig. 3 TS for PhyDNet-ATT-VIS and ECMWF at different levels of visibility

DownLoad: Full-Size Img PowerPoint

Fig. 4 TS for PhyDNet-ATT-VIS at different levels of visibility with different lead times

DownLoad: Full-Size Img PowerPoint

Fig. 5 Observed and predicted visibility for fog case on 28 Dec 2020 initiated at 2000 BT 27 Dec 2020

DownLoad: Full-Size Img PowerPoint

Fig. 6 Visibility test for PhyDNet-ATT-VIS and ECMWF for fog case on 28 Dec 2020

DownLoad: Full-Size Img PowerPoint

Fig. 7 Observed and predicted visibility for fog case on 22 Nov 2023 initiated at 2000 BT 21 Nov 2023

DownLoad: Full-Size Img PowerPoint

Table 1 Visibility assessment threshold

能见度/km	能见度等级
(1, 10]	轻雾
(0.5, 1]	大雾
(0.2, 0.5]	浓雾
(0, 0.2]	强浓雾

DownLoad: Download CSV

Table 2 Forecasting effectiveness in testing sample

预报模型	均方根误差/km	平均绝对偏差/km	相关系数
PhyDNet-ATT-VIS预报	2.43	1.46	0.94
ECMWF产品	7.31	5.98	0.4
PWAFS产品	10.87	8.58	0.06

DownLoad: Download CSV

References

[1]	Zhang X Y, Sun J Y, Wang Y Q, et al. Factors contributing to haze and fog in China. Chinese Sci Bull, 2013, 58(13): 1178-1187.
[2]	Gultepe I, Tardif R, Michaelides S C, et al. Fog research: A review of past achievements and future perspectives. Pure Appl Geophys, 2007, 164(6): 1121-1159.
[3]	Chen H S, Xu Y H, Gao Z Q, et al. Visibility forecast in Jiangsu Province based on the GCN-GRU model. Sci Rep, 2024. DOI: 10.1038/s41598-024-61572-8.
[4]	Peláez-Rodríguez C, Pérez-Aracil J, Casanova-Mateo C, et al. Efficient prediction of fog-related low-visibility events with machine learning and evolutionary algorithms. Atmos Res, 2023, 295. DOI: 10.1016/j.atmosres.2023.106991.
[5]	Kim B Y, Belorid M, Cha J W. Short-term visibility prediction using tree-based machine learning algorithms and numerical weather prediction data. Wea Forecasting, 2022, 37(12): 2263-2274. doi: 10.1175/WAF-D-22-0053.1
[6]	Ding Y H, Liu Y J. Analysis of long-term variations of fog and haze in China in recent 50 years and their relations with atmospheric humidity. Science China(Earth Sciences), 2014, 44(1): 37-48.
[7]	Liu Q, Sheng L F, Wang Y X, et al. Quantitative study of the effect of meteorological variables on the long-term variation of visibility in China. Clim Environ Res, 2016, 21(1): 47-55.
[8]	Feng L, Tian H. Progress in fog prediction. J Nanjing Univ Inf Sci Technol(Nat Sci Ed), 2014, 6(1): 74-81.
[9]	Gao S H, Qi Y L, Zhang S B, et al. Initial conditions improvement of sea fog numerical modeling over the Yellow Sea by Using cycling 3DVAR. Part Ⅰ: WRF numerical experiments. Periodical of Ocean University of China, 2010, 40(10): 1-9.
[10]	Quan J N, Xu X D, Jia X C, et al. Multi-scale processes in severe haze events in China and their interactions with aerosols: Mechanisms and progresses. Chinese Sci Bull, 2020, 65(9): 810-824.
[11]	Zhang G C. The progress of fog forecast operation in China. Adv Meteor Sci Tech, 2016, 6(2): 42-48.
[12]	Gao R Z, Li X, Shi X M, et al. Comparative study on three algorithms of the visibility in Qingdao offshore areas based on WRF model. J Mar Meteor, 2018, 38(2): 28-35.
[13]	Zhao X J, Xu J, Zhang Z Y, et al. Beijing regional environmental meteorology prediction system and its performance test of PM_2.5 concentration. J Appl Meteor Sci, 2016, 27(2): 160-172.
[14]	Wang T J, Jiang F, Deng J J, et al. Urban air quality and regional haze weather forecast for Yangtze River Delta region. Atmos Environ, 2012, 58: 70-83. doi: 10.1016/j.atmosenv.2012.01.014
[15]	Hou M L, Wang H, Zhao T L, et al. A modeling study of the visibility and PBL key meteorological elements during a heavy fog-haze episode in Beijing-Tianjin-Hebei of China. Chinese J Atmos Sci, 2017, 41(6): 1177-1190.
[16]	Zhao X J, Li Z M, Xu J. Modification and performance tests of visibility parameterizations for haze days. Environ Sci, 2019, 40(4): 1688-1696.
[17]	Wang Y, Wu Z J, Ma N, et al. Statistical analysis and parameterization of the hygroscopic growth of the sub-micrometer urban background aerosol in Beijing. Atmos Environ, 2018, 175: 184-191. doi: 10.1016/j.atmosenv.2017.12.003
[18]	Jiang J, Zhang G P, Gao J B. Main influencing factors of visibility in Beijing. J Appl Meteor Sci, 2018, 29(2): 188-199. doi: 10.11898/1001-7313.20180206
[19]	Zhang S P, Ren Z P. The influence of the thermal effect of underlaying surface on the spring sea fog over the Yellow Sea: Observations and numerical simulations. Acta Meteor Sinica, 2010, 68(4): 439-449.
[20]	Schlatter T W, Helms D, Reynolds D, et al. A Phenomenological Approach to the Specification of Observational Requirements. A Report to the Office of Science and Technology. National Weather Service, NOAA, 2005.
[21]	Hu Y Y, Pang L, Wang Q G. Application of deep learning bias correction method to temperature grid forecast of 7-15 days. J Appl Meteor Sci, 2023, 34(4): 426-437. doi: 10.11898/1001-7313.20230404
[22]	Yuan K, Li W J, Pang J. Hail identification technology in eastern Hubei based on decision tree algorithm. J Appl Meteor Sci, 2023, 34(2): 234-245. doi: 10.11898/1001-7313.20230209
[23]	Han N F, Yang L, Chen M X, et al. Machine learning correction of wind, temperature and humidity elements in Beijing-Tianjin-Hebei Region. J Appl Meteor Sci, 2022, 33(4): 489-500. doi: 10.11898/1001-7313.20220409
[24]	Liu D W, Mu H Z, He Q S, et al. A low visibility recognition algorithm based on surveillance video. J Appl Meteor Sci, 2022, 33(4): 501-512. doi: 10.11898/1001-7313.20220410
[25]	Zhao L N, Lu S, Qi D, et al. Daily maximum air temperature forecast based on fully connected neural network. J Appl Meteor Sci, 2022, 33(3): 257-269.
[26]	Xie S, Sun X G, Zhang S P, et al. Precipitation forecast correction in South China based on SVD and machine learning. J Appl Meteor Sci, 2022, 33(3): 293-304.
[27]	Huang W, Niu R Y. The medium-term multi-model integration forecast experimentation for heavy rain based on support vector machine. Meteor Mon, 2017, 43(9): 1110-1116.
[28]	Ren P, Chen M X, Cao W H, et al. Error analysis and correction of short-term numerical weather prediction under complex terrain based on machine learning. Acta Meteor Sinica, 2020, 78(6): 1002-1020.
[29]	Baldwin M E, Kain J S, Lakshmivarahan S. Development of an automated classification procedure for rainfall systems. Mon Wea Rev, 2005, 133(4): 844-862. doi: 10.1175/MWR2892.1
[30]	McGovern A, Gagne D J 2nd, Williams J K, et al. Enhancing understanding and improving prediction of severe weather through spatiotemporal relational learning. Mach Learn, 2014, 95(1): 27-50. doi: 10.1007/s10994-013-5343-x
[31]	Zhong J T, Zhang X Y, Gui K, et al. Robust prediction of hourly PM_2.5 from meteorological data using LightGBM. Natl Sci Rev, 2021, 8(10). DOI: 10.1093/nsr/nwaa307.
[32]	Le Guen V, Thome N. Disentangling physical dynamics from unknown factors for unsupervised video prediction. IEEE, 2020. DOI: 10.48550/arXiv.2003.01460.
[33]	Yuan K, Li W J, Li M, et al. Examination and evaluation of four machine deep learning algorithms for radar echo nowcasting in Wuhan Region. Meteor Mon, 2022, 48(4): 428-441.
[34]	Ma Z F, Zhang H, Liu J. A survey of precipitation nowcasting based on deep learning. Comput Eng Sci, 2023, 45(10): 1731-1753. doi: 10.3969/j.issn.1007-130X.2023.10.003
[35]	Li X, Zou X L, Zhuge X Y, et al. Improved himawari-8/AHI radiance data assimilation with a double cloud detection scheme. J Geophys Res Atmos, 2020, 125(13). DOI: 10.1029/2020JD03631.
[36]	Su X, Liu M, Kang Z M, et al. Verification of short-range torrential rain forecast during the 2020 Jiangsu main flood season. Meteor Mon, 2022, 48(3): 357-371.
[37]	Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need//11th Conference on Neural Information Processing Systems(NIPS), 2017. DOI: 10.48550/arXiv.1706.03762.
[38]	Zhou Y K, Zhu B, Han Z W, et al. Analysis of visibility characteristics and connecting factors over the Yangtze River Delta Region during winter time. China Environ Sci, 2016, 36(3): 660-669. doi: 10.3969/j.issn.1000-6923.2016.03.005
[39]	Liu X, Zheng Y, Zhuang X R, et al. Spatiotemporal convolutional approach for the short-term forecast of hourly heavy rainfall probability integrating numerical weather predictions and surface observations. Wea Forecasting, 2024, 39(3): 597-612. doi: 10.1175/WAF-D-23-0068.1
[40]	Zhang Z Y, Zhao X J, Xiong Y J, et al. The fog/haze medium-range forecast experiments based on dynamic statistic method. J Appl Meteor Sci, 2018, 29(1): 57-69.
[41]	Yin M X, Luo R T, Chen R Q, et al. Research on short-impending forecast of near-coast visibility for Leizhou peninsula based on GRU neural network. J Trop Meteor, 2023, 39(2): 267-275.
[42]	Zhuang X R, Zheng Y, Wang Y Q, et al. A deep learning-based precipitation nowcast model and its application over East China. Acta Meteor Sinica, 2023, 81(2): 286-303.
[43]	General Administration of Quality Supervision, Inspection and Quarantine of the People's Republic of China. Standardization Administration of the People's Republic of China. Grade of Fog Forecast. GB/T 27964-2011. Beijing: Standards Press of China, 2012.