Nowcasting of Cloud Images Based on Generative Adversarial Network and Satellite Data

Xiao Haixia; Zhang Feng; Wang Yaqiang; Tang Fei; Zheng Yu

doi:10.11898/1001-7313.20230208

Vol.34, NO.2, 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Applied Meteorological Science > 2023 > 34(2): 220-233

Xiao Haixia, Zhang Feng, Wang Yaqiang, et al. Nowcasting of cloud images based on generative adversarial network and satellite data. J Appl Meteor Sci, 2023, 34(2): 220-233. DOI: 10.11898/1001-7313.20230208.

Citation:

Xiao Haixia, Zhang Feng, Wang Yaqiang, et al. Nowcasting of cloud images based on generative adversarial network and satellite data. J Appl Meteor Sci, 2023, 34(2): 220-233. DOI: 10.11898/1001-7313.20230208.

Citation:

PDF( 22549 KB)

Nowcasting of Cloud Images Based on Generative Adversarial Network and Satellite Data

DOI: 10.11898/1001-7313.20230208

Xiao Haixia^{1, 2},
Zhang Feng^{2, 3},
Wang Yaqiang^{4
,
,},
Tang Fei¹,
Zheng Yu¹

1.
Key Laboratory of Transportation Meteorology of China Meteorological Administration, Nanjing Joint Institute for Atmospheric Sciences, Nanjing 210041
2.
Shanghai Qi Zhi Institute, Shanghai 200232
3.
Department of Atmospheric and Oceanic Sciences, Intitute of Atmospheric Sciences, Fudan University, Shanghai 200438
4.
Chinese Academy of Meteorological Sciences, Beijing 100081

Received Date: 2022-10-01
Rev Recd Date: 2022-12-15
Publish Date: 2023-03-31

Abstract

Abstract

Satellite cloud images contain abundant information, which can reflect daily weather conditions. Nowcasting based on cloud images can strengthen the application of satellite data in the early warning and forecasting of severe weather. At present, the cloud images predicted by most nowcasting methods based on artificial intelligence are not accurate enough and the lead time is limited. Thus, it's necessary to improve the accuracy and period validity of cloud images in nowcasting. Using the infrared cloud image data of Fengyun-4A (FY-4A) and the generative adversarial network (GAN) method, an infrared cloud image extrapolation nowcasting model is proposed. The cloud images in the next 3 hours in East China are predicted by the proposed model, and the spatial resolution of predicted cloud images is 4 km and the temporal resolution is 1 hour. The results show that the evaluation values of SSIM (structural similarity), PSNR (peak signal to noise ratio) and RMSE (root mean square error) predicted by the proposed GAN-based cloud images extrapolation model are 0.75, 20.92 and 10.00 K, respectively. In addition, the MAE (mean absolute error), MSE (mean squared error), and SSIM are chosen as loss function and analyzed, aiming to verify the rationality of the loss function in the generator. Comparative experiments of different loss functions show that it is reasonable and effective to choose SSIM combined with MAE as the loss function. Furthermore, to verify the effectiveness of the proposed GAN-based model, the prediction results are compared with those of the optical flow method and the TrajGRU model with the GAN-based model. The experimental results show that the cloud image extrapolation model based on GAN has the superior prediction performance, with the highest SSIM and PSNR, and the lowest RMSE within 1-3 h of cloud images nowcasting. The observational examples show that the cloud images predicted by the proposed model can well describe the movement, development and dissipation trend of clouds. Meanwhile, the experiments obtain accurate prediction performance on the intensity, location and shape of clouds in the study region. It indicates that the cloud image extrapolation model based on GAN is rational and feasible, which can be effectively applied to the meteorological business to monitor the occurrence and movement of clouds and warn the occurrence of severe weather in advance, and can provide an important reference for weather forecasting.
- satellite cloud image;
- nowcasting;
- deep learning;
- generative adversarial network

FullText(HTML)

References(52)

Figures and Tables

图 1 本文研究的华东区域

(红色虚线框所示)

Fig. 1 Study area in East China

(within the red dashed box)

DownLoad: Full-Size Img PowerPoint

Fig. 2 Prediction performance of diverse cloud image extrapolation models at different lead times

DownLoad: Full-Size Img PowerPoint

Fig. 3 Observed cloud images in the past 5 hours from 0500 UTC to 0900 UTC on 23 Jun 2020(inputs of models)

DownLoad: Full-Size Img PowerPoint

Fig. 4 Comparison of the observed and the predicted cloud images in the next 3 hours from 1000 UTC to 1200 UTC on 23 Jun 2020

DownLoad: Full-Size Img PowerPoint

Fig. 5 The same as in Fig. 3, but from 0300 UTC to 0700 UTC on 15 Jun 2020

DownLoad: Full-Size Img PowerPoint

Fig. 6 The same as in Fig. 4, but from 0800 UTC to 1000 UTC on 15 Jun 2020

DownLoad: Full-Size Img PowerPoint

Fig. 7 The same as in Fig. 3, but from 1000 UTC to 1400 UTC on 13 Jun 2020

DownLoad: Full-Size Img PowerPoint

Fig. 8 The same as in Fig. 4, but from 1500 UTC to 1700 UTC on 13 Jun 2020

DownLoad: Full-Size Img PowerPoint

Table 1 Performance of the cloud image extrapolation model trained with different loss functions in the generator

损失函数	评估指标
损失函数	相对误差绝对值/%	均方根误差/K	结构相似性	峰值信噪比
均方误差	157.80	10.48	0.69	20.50
平均误差绝对值	155.75	10.11	0.74	20.87
结构相似性	158.43	10.36	0.74	20.63
结构相似性结合平均误差绝对值	151.64	10.00	0.75	20.92

DownLoad: Download CSV

Table 2 Predictive performance of the generative adversarial network model for the cloud images at different lead times using different loss functions

损失函数	评估指标
	结构相似性			峰值信噪比
	1 h	2 h	3 h	1 h	2 h	3 h
平均误差绝对值	0.79	0.74	0.71	23.25	20.44	18.92
均方误差	0.75	0.69	0.61	22.99	19.53	18.98
结构相似性	0.79	0.74	0.71	22.92	20.17	18.82
结构相似性结合平均误差绝对值	0.80	0.75	0.72	23.21	20.51	19.03

DownLoad: Download CSV

Table 3 Prediction performance of different models based on testing set

模型	评估指标
模型	相对误差绝对值%	均方根误差/K	结构相似性	峰值信噪比
生成对抗网络	151.64	10.00	0.75	20.92
轨迹门控循环单元模型	172.84	11.10	0.72	19.99
光流法	186.35	11.97	0.66	19.22

DownLoad: Download CSV

References

[1]	Zhang C, Liu J, Li X G, et al. A cloud classification method based on information fusion of visible and infrared images. J Meteor Environ, 2018, 34(1): 82-90. doi: 10.3969/j.issn.1673-503X.2018.01.010
[2]	Wu X J, Zhu X X, Mao Z Y, et al. Algorithm design of convective precipitation monitoring and early warning service using FY-2 infrared data. Journal of Marine Meteorology, 2019, 39(3): 1-10. https://www.cnki.com.cn/Article/CJFDTOTAL-SDQX201903001.htm
[3]	Rojas Y, Minder J R, Campbell L S, et al. Assessment of GPM IMERG satellite precipitation estimation and its dependence on microphysical rain regimes over the mountains of south-central Chile-ScienceDirect. Atmos Res, 2021, 253: 105454. doi: 10.1016/j.atmosres.2021.105454
[4]	Qin H, Zheng F Q, Wu L Q. The interaction between intensity and rainfall of Typhoon Rammasun(1409). J Appl Meteor Sci, 2022, 33(4): 477-488. doi: 10.11898/1001-7313.20220408
[5]	Zheng Q, Mao C Y, Ding L H, et al. Comparison of cloud characteristics between Typhoon Lekima(1909) and Typhoon Yagi(1814). J Appl Meteor Sci, 2022, 33(1): 43-55. doi: 10.11898/1001-7313.20220104
[6]	Gao S Z, Zhang S J, Lü X Y, et al. Circulation characteristics and thermal and dynamic conditions 48 hours before typhoon formation in South China Sea. J Appl Meteor Sci, 2021, 32(3): 272-288. doi: 10.11898/1001-7313.20210302
[7]	Zhang Z S, Liu M, Hu C Y, et al. Prediction of typhoon grade with time series typhoon satellite images based on Seq2Seq and Attention. Remote Sensing Information, 2020, 35(4): 16-22. doi: 10.3969/j.issn.1000-3177.2020.04.003
[8]	Zou G L, Hou Q, Zheng Z S, et al. Classification of typhoon grade based on satellite cloud image and deep learning. Remote Sensing Information, 2019, 34(3): 1-6. https://www.cnki.com.cn/Article/CJFDTOTAL-YGXX201903001.htm
[9]	Ren S L, Fang X, Lu N M, et al. Recognition method of the Tibetan Plateau vortex based on meteorological satellite data. J Trop Meteor, 2019, 30(3): 345-359. doi: 10.11898/1001-7313.20190308
[10]	Wang Y F, Qi Y B, Li Q, et al. Macro and micro characteristics of a fog process in Changbai Mountain in summer. J Appl Meteor Sci, 2022, 33(4): 442-453. doi: 10.11898/1001-7313.20220405
[11]	Ma R Y, Zheng D, Yao W, et al. Thunderstorm feature dataset and characteristics of thunderstorm activities in China. J Appl Meteor Sci, 2021, 32(3): 358-369. doi: 10.11898/1001-7313.20210308
[12]	Ren S L, Niu N, Qin D Y, et al. Extreme cold and snowstorm event in North America in February 2021 based on satellite data. J Appl Meteor Sci, 2022, 33(6): 696-710. doi: 10.11898/1001-7313.20220605
[13]	Wang L, Zhou Y Q, Cai M, et al. Study on correlation between cloud characteristic parameters and precipitation in North China. Meteor Environ Sci, 2019, 42(3): 9-16. https://www.cnki.com.cn/Article/CJFDTOTAL-HNQX201903002.htm
[14]	Wang S H, Han Z G, Yao Z G, et al. Analysis on cloud vertical structure over China and its neighborhood based on CloudSat data. Plateau Meteor, 2011, 30(1): 38-52. https://www.cnki.com.cn/Article/CJFDTOTAL-GYQX201101006.htm
[15]	Genkova I S, Pachedjieva B, Ganev G, et al. Cloud motion estimation from METEOSAT images using time mutability method. International Society for Optics and Photonics, 1999, 3571: 297-301.
[16]	Chen M X, Wang Y C, Yu X D. Improvement and application test of TREC algorithm for convective storm nowcast. J Trop Meteor, 2007, 18(5): 690-701. http://qikan.camscma.cn/article/id/200705105
[17]	Zhang L J, Cheng M H, Tian F Y. Application of artificial neural networks and support vector machine in rainfall forecasting. Plateau Meteor, 2010, 29(4): 982-991. https://www.cnki.com.cn/Article/CJFDTOTAL-GYQX201004018.htm
[18]	Liu K F, Zhang R, Sun Z B. A cloud movement short-time forecast based on crosscorrelation. J Image Graph, 2006, 11(4): 586-591. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB200604023.htm
[19]	Vukicevic T, Greenwald T, Zupanski M, et al. Mesoscale cloud state estimation from visible and infrared satellite radiances. Mon Wea Rev, 2004, 132(12): 3066-3077.
[20]	Das S K, Chanda B, Mukherjee D P. Prediction of Cloud for Weather Now-casting Application Using Topology Adaptive Active Membrane//International Conference on Pattern Recognition and Machine Intelligence. Springer, 2009: 303-308.
[21]	Goswami B, Bhandari G. Automatically Adjusting Cloud Movement Prediction Model from Satellite Infrared Images//2011 Annual IEEE India Conference, 2012: 1-4.
[22]	Liu Y A, Wei M, Gao W, et al. Short-term automatic forecast algorithm of severe convective cloud identification using FY-2 IR images. J Remote Sens, 2012, 16(1): 79-92. https://www.cnki.com.cn/Article/CJFDTOTAL-YGXB201201007.htm
[23]	Morf H. Sunshine and cloud cover prediction based on Markov processes. Sol Energy, 2014, 110: 615-626.
[24]	Liang L W, Yang X H, Yin J, et al. The new short-term cloud forecast method. Plateau Meteor, 2015, 34(4): 1186-1190. https://www.cnki.com.cn/Article/CJFDTOTAL-GYQX201504030.htm
[25]	Sakaino H. Spatio-temporal image pattern prediction method based on a physical model with time-varying optical flow. IEEE Trans Geosci Remote Sens, 2013, 51(5): 3023-3036.
[26]	Chow C W, Belongie S, Kleissl J. Cloud motion and stability estimation for intra-hour solar forecasting. Sol Energy, 2015, 115: 645-655.
[27]	Tian L, Li X, Ye Y, et al. A generative adversarial gated recurrent unit model for precipitation nowcasting. IEEE Geoscience and Remote Sensing Letters, 2019, 17(4): 601-605.
[28]	Liu N, Xiong A Y, Zhang Q, et al. Development of basic dataset of severe convective weather for artificial intelligence training. J Appl Meteor Sci, 2021, 32(5): 530-541. doi: 10.11898/1001-7313.20210502
[29]	Yuan K, Li W J, Li M, et al. Examination and avaluation of four machine deep learning algorithms for radar echo nowcasting in Wuhan Region. Meteor Mon, 2022, 48(4): 428-441. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXX202204004.htm
[30]	Shi X, Gao Z, Lausen L, et al. Deep learning for precipitation nowcasting: A benchmark and a new model. Computer Science-Computer Vision and Pattern Recognition, 2017, arXiv. 1706.03458.
[31]	Pham T Q D, Hoang T V, Van Tran X, et al. Fast and accurate prediction of temperature evolutions in additive manufacturing process using deep learning. J Intell Manuf, 2022: 1-19. DOI: 10.1007/s10845-021-01896-8.
[32]	Wei J, Huang W, Li Z Q, et al. Estimating 1-km-resolution PM_2.5 concentrations across China using the space-time random forest approach. Remote Sens Environ, 2019, 231: 111221.
[33]	Kristiani E, Lin H, Lin J R, et al. Short-term prediction of PM_2.5 using LSTM deep learning methods. Sustainability, 2022, 14(4): 2068.
[34]	Zhang J, Liu P, Zhang F, et al. CloudNet: Ground-based cloud classification with deep convolutional neural network. Geophys Res Lett, 2018, 45(16): 8665-8672.
[35]	Hensel S, Rinov M B, Koch M, et al. Evaluation of deep learning-based neural network methods for cloud detection and segmentation. Energies, 2021, 14(19): 6156.
[36]	He R, Guan Z Y, Jin L. A short-term cloud forecast model by neural networks. Trans Atmos Sci, 2010, 33(6): 725-730. https://www.cnki.com.cn/Article/CJFDTOTAL-NJQX201006010.htm
[37]	Tan C, Feng X, Long J, et al. FORECAST-CLSTM: A New Convolutional LSTM Network for Cloudage Nowcasting. 2018 IEEE Visual Communications and Image Processing(VCIP), 2018.
[38]	Su X, Li T, An C, et al. Prediction of short-time cloud motion using a deep-learning model. Atmosphere, 2020, 11(11): 1151.
[39]	Lee J H, Lee S S, Kim H G, et al. MSCIP Net: Multichannel satellite image prediction via deep neural network. IEEE Trans Geosci Remote Sens, 2019, 58(3): 2212-2224.
[40]	Tao R Z. Cloud Detection and Precipitation Extrapolation Forecast in Tibet Based on Fengyun-4 Satellite Images. Nanjing: Nanjing University of Information Science & Technology, 2021.
[41]	Qu J H, Zhang L, Lu Q F, et al. Characterization of bias in FY-4A advanced geostationary radiation imager observations from ERA5 background simulations using RTTOV. Acta Meteor Sinica, 2019, 77(5): 911-922. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB201905009.htm
[42]	Tang F, Zhuge X, Zeng M, et al. Applications of the advanced radiative transfer modeling system(ARMS) to characterize the performance of Fengyun-4A/AGRI. Remote Sens, 2021, 13(16): 3120.
[43]	Gao Y, Cai M, Cao Z Q, et al. Environmental conditions and cloud macro and micro features of "21·7" extreme heavy rainfall in Henan Province. J Appl Meteor Sci, 2022, 33(6): 682-695. doi: 10.11898/1001-7313.20220604
[44]	Chyi D, He L F, Wang X M, et al. Fine observation characteristics and thermodynamic mechanisms of extreme heavy rainfall in Henan on 20 July 2021. J Appl Meteor Sci, 2022, 33(1): 1-15. doi: 10.11898/1001-7313.20220101
[45]	Chen Y Z, Lin L X, Wang R, et al. A study on the artificial intelligence nowcasting based on generative adversarial networks. Trans Atmos Sci, 2019, 42(2): 311-320. https://www.cnki.com.cn/Article/CJFDTOTAL-NJQX201902014.htm
[46]	Shi X, Chen Z, Wang H, et al. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Computer Science-Computer Vision and Pattern Recognition, 2015, 28: 802-810.
[47]	Jing J, Li Q, Peng X. MLC-LSTM: Exploiting the spatiotemporal correlation between multi-level weather radar echoes for echo sequence extrapolation. Sensors, 2019, 19(18): 3988.
[48]	Kingma D P, Ba J. Adam: A method for stochastic optimization. Computer Science-Machine Learning, 2014, arXiv: 1412.6980.
[49]	Ruder S. An overview of gradient descent optimization. Comput Sci-Machine Learning, 2016, arXiv: 1609.04747.
[50]	Zhou W, Bovik A C, Sheikh H R, et al. Image quality assessment: From error visibility to structural similarity. IEEE Trans Image Process, 2004, 13: 600-612.
[51]	Zhao H, Gallo O, Frosio I, et al. Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging, 2016, 3(1): 47-57.
[52]	Ayzel G, Heistermann M, Winterrath T. Optical flow models as an open benchmark for radar-based precipitation nowcasting(rainymotion v0.1). Geosci Model Dev, 2019, 12(4): 1387-1402.