Indentification of Weather Radar Abnormal Data Based on Deep Learning

Zhang Lin; Wu Lei; Li Feng; Li Yan; Shi Lijuan; Sun Kangyuan

doi:10.11898/1001-7313.20230605

Vol.34, NO.6, 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Applied Meteorological Science > 2023 > 34(6): 694-705

Zhang Lin, Wu Lei, Li Feng, et al. Indentification of weather radar abnormal data based on deep learning. J Appl Meteor Sci, 2023, 34(6): 694-705. DOI: 10.11898/1001-7313.20230605.

Citation:

Zhang Lin, Wu Lei, Li Feng, et al. Indentification of weather radar abnormal data based on deep learning. J Appl Meteor Sci, 2023, 34(6): 694-705. DOI: 10.11898/1001-7313.20230605.

Zhang Lin, Wu Lei, Li Feng, et al. Indentification of weather radar abnormal data based on deep learning. J Appl Meteor Sci, 2023, 34(6): 694-705. DOI: 10.11898/1001-7313.20230605.

Citation:

Zhang Lin, Wu Lei, Li Feng, et al. Indentification of weather radar abnormal data based on deep learning. J Appl Meteor Sci, 2023, 34(6): 694-705. DOI: 10.11898/1001-7313.20230605.

PDF( 2470 KB)

Indentification of Weather Radar Abnormal Data Based on Deep Learning

DOI: 10.11898/1001-7313.20230605

Zhang Lin^{1, 2},
Wu Lei^{1
,
,},
Li Feng¹,
Li Yan^{1, 3},
Shi Lijuan¹,
Sun Kangyuan²

1.
CMA Meteorological Observation Center, Beijing 100081
2.
Key Laboratory of Transportation Meteorology of CMA, Nanjing Joint Institute for Atmospheric Sciences, Nanjing 210041
3.
Chinese Academy of Meteorological Sciences, Beijing 100081

Received Date: 2023-06-06
Rev Recd Date: 2023-09-26
Publish Date: 2023-11-27

Abstract

Abstract

The world's largest weather radar observation network which consists of 236 weather radars is built up in China. The quality control of weather radar data becomes an indispensable part in operation as data grow. In real-time operation of CMA Meterological Observation Center, the abnormal data caused by radar hardware fault or calibration problem usually leads to a bad image, and the problem directly affects the quality of weather radar data, quantitative estimation of precipitation and analysis of the weather system. At present, the abnormal data are processed by artificial corrigendum in real-time operation, so that it does not affect the application of follow-up data. In recent years, artificial intelligence technology has developed rapidly, and deep learning algorithms are used to build convolutional neural network models to extract image features from abnormal data such as radar hardware failures and electromagnetic interference. According to characteristics of the abnormal data, two kinds of abnormal dataset are established. The first kind is labeled data for those samples with whole abnormal picture or precipitation. The second kind is the electromagnetic interference in fixed direction, which is easily mixed with precipitation, and it is used to label the samples for pixels. Based on the convolution pre-training network model (ResNet), the R-ResNet model is built with the first kind of abnormal data. The model evaluation shows that the R-ResNet model achieves more than 99% accuracy in both training and test datasets, and the prediction results of the model in test datasets are all consistent with the label data. For the second kind of abnormal data, based on the image semantic segmentation network model LinkNet, a group of encoders and decoders are extended, and a hop-link structure is added, and ResNet50 is used as the encoder structure. The R-LinkNet model is constructed and the accuracy is over 98% on the training and test datasets, with intersection over union of 83.4% and 83.2%, respectively. Two models can be used to monitor the abnormal data of the national radar in real-time operation, so that abnormal data can be corrected automatically and the workload of the manual on duty can be reduced greatly.
- deep learning;
- neural network;
- abnormal data;
- model

FullText(HTML)

References(40)

Figures and Tables

Fig. 1 Proportion of weather radar data quality problems in China in 2020

DownLoad: Full-Size Img PowerPoint

图 2 全国各型号天气雷达电磁干扰占比及个例

(a)各型号天气雷达电磁干扰占比，(b)2022年7月16日01:25:58山东滨州雷达电磁干扰(相邻距离圈间隔为50 km，下同)

Fig. 2 Proportion of electromagnetic interference of various types of weather radar

(a)proportion of electromagnetic interference, (b)a case of electromagnetic interference of Binzhou radar in Shandong at 012558 BT 16 Jul 2022 (the distance between adjacent rings is 50 km, similarly hereinafter)

DownLoad: Full-Size Img PowerPoint

图 3 全国各型号天气雷达设备故障、标定异常数据占比及个例

(a)各型号天气雷达设备故障、标定异常数据占比，(b)2022年4月11日11:21:10新疆图木舒克雷达异常数据

Fig. 3 Percentage of abnormal data of equipment failure and calibration of various types of weather radar in China

(a)proportion of abnormal data of equipment failure and calibration, (b)a case of abnormal data of Tumxuk radar in Xinjiang at 112110 BT 11 Apr 2022

DownLoad: Full-Size Img PowerPoint

图 4 第2类异常数据原始图像与标签数据

(a)2022年7月20日08:15:02黑龙江黑瞎子岛雷达观测图像，(b)标签数据

Fig. 4 Observation image and label data of the second kind of abnormal data

(a)observation image of Heixiazi Island radar in Heilongjiang at 081502 BT 20 Jul 2022, (b)label data

DownLoad: Full-Size Img PowerPoint

Fig. 5 Accuracy and loss of ResNet50 model on the first kind of abnormal data

DownLoad: Full-Size Img PowerPoint

Fig. 6 Accuracy and loss of R-LinkNet model on the second kind of abnormal data

DownLoad: Full-Size Img PowerPoint

图 7 R-LinkNet分离黑龙江黑瞎子岛雷达观测电磁干扰和降水回波

Fig. 7 R-LinkNet model discriminate electromagnetic interference and precipitation of Heixiazi Island radar in Heilongjiang

(a)observation image at 200334 BT 8 Jul 2022, (b)image after quality control by R-LinkNet model at 200334 BT 8 Jul 2022, (c)observation image at 100335 BT 5 Aug 2022, (d)image after quality control by R-LinkNet model at 100335 BT 5 Aug 2022, (e)observation image at 090233 BT 8 Sep 2022, (f)image after quality control by R-LinkNet model at 090233 BT 8 Sep 2022

DownLoad: Full-Size Img PowerPoint

Table 1 Accuracy of ResNet models (unit:%)

指标	数据	ResNet18	ResNet50	ResNet101
A_cc	训练集	99.92	99.98	99.99
A_cc	测试集	100	100	100
L_os	训练集	0.1	0.1	0.1
L_os	测试集	0.1	0.1	0.1

DownLoad: Download CSV

Table 2 Accuracy, loss and intersection over union of model (unit:%)

数据	A_cc		L_os		I_ou
数据	LinkNet	R-LinkNet	LinkNet	R-LinkNet	LinkNet	R-LinkNet
训练集	97.8	98.6	0.7	0.4	73.8	83.4
测试集	97.8	98.6	0.7	0.4	73.5	83.2

DownLoad: Download CSV

Table 3 Evaluation of R-LinkNet model accuracy (unit:%)

测试时间	R-LinkNet模型
测试时间	7月	8月	9月
7月	98.6	98.5	98.3
8月	98.2	98.6	98.5
9月	98.0	98.3	98.6

DownLoad: Download CSV

References

[1]	Zhao R J, Liu L P, Zhang J. The quality control method of erroneous radar echo data generated by hardware fault. J Appl Meteor Sci, 2015, 26(5): 578-589. doi: 10.11898/1001-7313.20150507
[2]	Shao N, Pei C, Liu C C, et al. Automatic idenfication system of abnormal radar echoes based on image processing technology. Meteor Sci Technol, 2013, 41(6): 993-997. doi: 10.3969/j.issn.1671-6345.2013.06.004
[3]	Jiang Y, Liu L P, Zhuang W. Statistical characteristics of clutter and improvements of ground clutter identification technique with Doppler weather radar. J Appl Meteor Sci, 2009, 20(2): 203-213. http://qikan.camscma.cn/article/id/20090210
[4]	Li F, Liu L P, Wang H Y, et al. Identification of ground clutter with C-band Doppler weather radar. J Appl Meteor Sci, 2014, 25(2): 158-167. http://qikan.camscma.cn/article/id/20140205
[5]	Liu L P, Wu L L, Yang Y M. Development of fuzzy-logical two-step ground clutter detection algorithm. Acta Meteor Sinica, 2007, 65(2): 252-260. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB200702010.htm
[6]	Wen H, Liu L P, Zhang Y. Improvements of ground clutter identification algorithm for Doppler weather radar. Plateau Meteor, 2017, 36(3): 736-749. https://www.cnki.com.cn/Article/CJFDTOTAL-GYQX201703014.htm
[7]	Wen H, Zhang L J, Liang H H, et al. Radial interference echo identification algorithm based on fuzzy logic for weather radar. Acta Meteor Sinica, 2020, 78(1): 116-127. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202001010.htm
[8]	Tan X, Liu L P, Fan S R. Statistical characteristics of sea clutter and its identification with the CINRAD. Acta Meteor Sinica, 2013, 71(5): 962-975. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB201305015.htm
[9]	Leng L, Huang X Y, Yang H P, et al. Recognition and application of Doppler weather radar clear air echoes. Meteor Sci Technol, 2012, 40(4): 534-541. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ201204005.htm
[10]	Chen Y J, Liang H H, Zhang L J. Characteristic analysis reflectivity factor of clear air echoes of Doppler weather radars. Meteor Sci Technol, 2022, 50(3): 303-313. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ202203001.htm
[11]	Ye F, Liang H H, Wen H, et al. Evaluation of homogeneity of new generation weather radar. Meteor Sci Technol, 2020, 50(3): 322-330. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ202003002.htm
[12]	Li F, Liu L P, Wang H Y, et al. Identification of non-precipitation meteorological echoes with Doppler weather radar. J Appl Meteor Sci, 2012, 23(2): 147-158. http://qikan.camscma.cn/article/id/20120203
[13]	Xiao Y J, Wan Y F, Wang J, et al. Study of an automated Doppler radar velocity dealiasing algorithm. Plateau Meteor, 2012, 31(4): 1119-1128. https://www.cnki.com.cn/Article/CJFDTOTAL-GYQX201204028.htm
[14]	Yang C, Liu L P, Hu Z Q, et al. An algorithm for chaos radial velocity identifying and processing in C-band Doppler radars running in the dual PRF mode. Acta Meteor Sinica, 2013, 70(4): 875-886. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB201204026.htm
[15]	Wu C, Liu L P, Yang M L, et al. Key technologies of hydrometeor classification and mosaic algorithm for X-band polarimetric radar. J Appl Meteor Sci, 2021, 32(2): 200-216. doi: 10.11898/1001-7313.20210206
[16]	Tang L, Zhang L, Langston C, et al. A physically based precipitation-nonprecipitation radar classifier using polarimetric and environmental data in a real-time national system. Wea Forecasting, 2014, 29(5): 1106-1119.
[17]	Park H S, Ryzhkov A V, Zrnić D S, et al. The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to an MCS. Wea Forecasting, 2009, 24(3): 730-748.
[18]	Jiang Y, Xu Q, Zhang P F, et al. Using WSR-88D polarimetric data to identify bird-contaminated Doppler velocities. Adv Meteor, 2013, 1: 1-13.
[19]	Zhu Y M, Ma S Q, Yang L, et al. Recognition and analysis of biological echo using WSR-88D dual-polarization weather radar in Nanhui of Shanghai. Meteor Environ Sci, 2019, 42(3): 118-128. https://www.cnki.com.cn/Article/CJFDTOTAL-HNQX201903016.htm
[20]	Zhang L, Yang H P. Non-precipitation identification technique on S-band WSR-88D polarization weather radar. Meteor Month, 2018, 44(5): 665-675. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXX201805007.htm
[21]	Zhang L, Li F, Wu L, et al. Non-precipitation identification technique for CINRAD/SAD dual polarimetric weather radar. J Appl Meteor Sci, 2022, 33(6): 724-735. doi: 10.11898/1001-7313.20220607
[22]	Zhang L, Li F, Feng W Y, et al. Research of data quality analysis and bias correction on mobile X-band dual-polarization weather radar. Meteor Mon, 2021, 47(3): 337-347. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXX202103007.htm
[23]	Yin X Y, Hu Z Q, Zheng J F, et al. Filling in the dual polarization radar echo occlusion based on deep learning. J Appl Meteor Sci, 2022, 33(5): 581-593. doi: 10.11898/1001-7313.20220506
[24]	Li Y, Chen H L. Review of machine learning approaches for modern agrometeorology. J Appl Meteor Sci, 2020, 31(3): 257-266. doi: 10.11898/1001-7313.20200301
[25]	Zhang Y F, Feng Z Z, Liu B. Lightning nowcasting early warning model based on convolutional neural network. Meteor Mon, 2021, 47(3): 373-380. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXX202103010.htm
[26]	Zhou K H, Zheng Y G, Han L, et al. Advances in application of machine learning to severe convective weather monitoring and forecasting. Meteor Mon, 2021, 47(3): 274-289. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXX202103002.htm
[27]	Huang J W, Cai R H, Yao R, et al. Application of deep learning method to discrimination and forecasting of precipitation type. Meteor Mon, 2021, 47(3): 317-326. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXX202103005.htm
[28]	Jin Z Q, Wang X M, Bao Y S, et al. Squall line identification method based on convolution neural network. J Appl Meteor Sci, 2021, 32(5): 580-591. doi: 10.11898/1001-7313.20210506
[29]	Nawal H, Timothy D, Sebastian T, et al. A neural network quality-control scheme for improved quantitative precipitation estimation accuracy on the UK weather radar network. J Atmos Ocean Technol, 2021, 38(6): 1157-1172.
[30]	Huangfu J, Hu Z Q, Zheng J F, et al. A study on polarization radar quantitative precipitation estimation using deep learning. Acta Meteor Sinica, 2022, 80(4): 565-577. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202204006.htm
[31]	Zhao L N, Lu S, Qi D, et al. Daily maximum air temperature forecast based on fully connected neural network. J Appl Meteor Sci, 2022, 33(3): 257-269. doi: 10.11898/1001-7313.20220301
[32]	Liu H Z, Xu H, Bao H J, et al. Application of machine learning classification algorithm to precipitation-induced landslides forecasting. J Appl Meteor Sci, 2022, 33(3): 282-292. doi: 10.11898/1001-7313.20220303
[33]	Zhang F Y, Zhao M, Zhou Y Z, et al. Detection of diseased takifugu rubripes based on ResNet50 and transfer learning. Fishery Modern, 2021, 48(4): 51-60. https://www.cnki.com.cn/Article/CJFDTOTAL-HDXY202104007.htm
[34]	Riyue G H. PyTorch Deep Learning Concise Actual Combat. Beijing: Tsinghua University Press, 2022.
[35]	Zheng Z Y, Liang B W, Gu S Y. TensorFlow: A Framework for Google Deep Learning in Actual Combat. 2nd Ed. Beijing: Publishing House of Electronics Industry, 2018.
[36]	Chen Y, Dang S W, Nie L, Loop detection algorithm based on resnet model. Intelligent Computer Appl, 2022, 12(8): 196-199. https://www.cnki.com.cn/Article/CJFDTOTAL-DLXZ202208039.htm
[37]	Abhishek C, Culurciello E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation. arXiv Preprint arXiv, 2017. DOI: 10.1109/VCIP.2017.8305148.
[38]	Liu H T. Research on Remote Sensing Road Extraction Method Based on Improved Linknet. Harbin: Harbin Engineering University, 2021.
[39]	Zhang Y. Research on Image Semantic Segmentation Algorithm Based on Deep Learning. Chengdu: Southwest University of Science and Technology, 2021.
[40]	Yang X B. Road and Small Buliding Extraction Methods Based on Semantic Segmentation in Remote Sensing Image. Wuhan: Huazhong University of Science and Technology, 2019.