A Low Visibility Recognition Algorithm Based on Surveillance Video

Liu Dongwei; Mu Haizhen; He Qianshan; Shi Jun; Wang Yadong; Wu Xueqin

doi:10.11898/1001-7313.20220410

Vol.33, NO.4, 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Applied Meteorological Science > 2022 > 33(4): 501-512

Liu Dongwei, Mu Haizhen, He Qianshan, et al. A low visibility recognition algorithm based on surveillance video. J Appl Meteor Sci, 2022, 33(4): 501-512. DOI: 10.11898/1001-7313.20220410.

Citation:

Liu Dongwei, Mu Haizhen, He Qianshan, et al. A low visibility recognition algorithm based on surveillance video. J Appl Meteor Sci, 2022, 33(4): 501-512. DOI: 10.11898/1001-7313.20220410.

Liu Dongwei, Mu Haizhen, He Qianshan, et al. A low visibility recognition algorithm based on surveillance video. J Appl Meteor Sci, 2022, 33(4): 501-512. DOI: 10.11898/1001-7313.20220410.

Citation:

Liu Dongwei, Mu Haizhen, He Qianshan, et al. A low visibility recognition algorithm based on surveillance video. J Appl Meteor Sci, 2022, 33(4): 501-512. DOI: 10.11898/1001-7313.20220410.

PDF( 1935 KB)

A Low Visibility Recognition Algorithm Based on Surveillance Video

DOI: 10.11898/1001-7313.20220410

1.
Shanghai Ecological Forecasting and Remote Sensing Center, Shanghai 200030
2.
Shanghai Meteorological Information and Technology Support Center, Shanghai 200030
3.
Shanghai Key Laboratory of Meteorology and Health, Shanghai 200030

Received Date: 2022-02-21
Rev Recd Date: 2022-05-23
Publish Date: 2022-07-13

Abstract

Abstract

Low visibility has significant influences on highways, ferries, civil aviation, and other modes of transportation, and the visibility observation of meteorological departments is not dense enough to meet the monitoring needs of low visibility weather. Using existing video surveillance equipment to extract visibility data can save a significant amount of money on visibility instrument deployment and maintenance, improve data density, and provide finer data support for traffic and urban safety operations. Based on video live image conversion, a simple convolutional neural network classification approach is suggested to extract visibility levels. The algorithm assumes that the video devices are installed horizontally and have an open view, and it creates a new fixed-size image by dividing the original video image into horizontal chunks and extracting the gradient, color saturation, and brightness information from each horizontal chunk. A simple convolutional neural network is used to learn and develop a visibility level recognition model from the converted images. The model is trained by 29668 video images of Yangshan Port Weather Station in Shanghai from September 2019 to December 2020, and then tested with 5757 video images from January to May in 2021. The comparison indicates the recognition model generated with this technique has a greater accuracy than the recognition model built directly with AlexNet network. The model has an overall accuracy of 87.99% during daytime and 81.32% during nighttime when the observed visibility is classified into five levels of fog-free, light fog, fog, dense fog, and thick fog according to the fog forecasting level. The model's identification ability for no fog and light fog is high. However, because the scenery becomes nearly indistinguishable once dense fog appears at night, the model's recognition ability for dense fog level at night is poor, and it is easy to categorize it as a fog level mistakenly. Taking 1000 meters as the criterion of low visibility weather, the algorithm's accuracy is 96.18% during daytime and 96.14% during nighttime. The algorithm features a quick learning rate and ease of application, making it suitable for low visibility video image recognition in most open-field scenarios. The model is applied during a radiation fog in Shanghai on 13 April 2021. The video images of the sparse area of the automatic weather station installation are collected for visibility identification, and the visibility distribution map formed together with the existing automatic weather station visibility meter data is more complete and accurate, which demonstrates that the model established by this algorithm can effectively compensate the problem of insufficient density of the existing automatic station visibility meter data, and has certain application value in meteorological operations.
- low visibility;
- image recognition;
- algorithm;
- convolutional neural network

FullText(HTML)

References(38)

Figures and Tables

Fig. 1 Distribution of video scene scenery at Yangshan Port Weather Station at 1400 BT 16 Jan 2021

DownLoad: Full-Size Img PowerPoint

图 2 裁剪后白天和夜间原始图像和梯度图个例

(a)白天原始图像，(b)白天梯度图，(c)夜间原始图像，(d)夜间梯度图

Fig. 2 Daytime and nighttime examples of cropped original images and gradient maps

(a)daytime original image, (b)daytime gradient map, (c)nighttime original image, (d)nighttime gradient map

DownLoad: Full-Size Img PowerPoint

图 3 白天不同能见度等级下处理后的典型图像

(a)无雾，(b)轻雾，(c)雾，(d)大雾，(e)浓雾

Fig. 3 Typical images after processing at various levels of visibility during daytime

(a)no fog,(b)light fog,(c)fog,(d)dense fog,(e)thick fog

DownLoad: Full-Size Img PowerPoint

图 4 夜间不同能见度等级下处理后的典型图像

(a)无雾，(b)轻雾，(c)雾，(d)大雾，(e)浓雾

Fig. 4 Typical images with various visibility levels after processing during nighttime

(a)no fog,(b)light fog,(c)fog,(d)dense fog,(e)thick fog

DownLoad: Full-Size Img PowerPoint

Fig. 5 Examples of failed recognition when the visibility meter observation level is fog

DownLoad: Full-Size Img PowerPoint

图 6 2021年4月13日05:00能见度分布图(▲为能见度仪位置，$ \bullet $为视频设备位置)

(a)仅用能见度仪数据，(b)能见度仪数据和4个站点视频图像能见度解析数据

Fig. 6 The visibility distribution at 0500 BT 13 Apr 2021 (▲ is the visibility meter position,$ \bullet $ is the video position)

(a)using visibility meter data only, (b)using visibility meter data and 4 video images visibility analysis data

DownLoad: Full-Size Img PowerPoint

Table 1 The standard for visibility classification

分类编号	能见度范围/m	能见度等级名称
0	大于10000	无雾
1	1001~10000	轻雾
2	501~1000	雾
3	201~500	大雾
4	51~200	浓雾
5	0~50	强浓雾

DownLoad: Download CSV

Table 2 The total number of original and modified samples in each category

等级	分类编号	白天		夜间
等级	分类编号	原有样本量	调整后样本量	原有样本量	调整后样本量
无雾	0	10537	10537	9597	9597
轻雾	1	4997	4997	4159	4159
雾	2	54	6912	131	7546
大雾	3	78	6630	165	7574
浓雾	4	85	7225	165	7574

DownLoad: Download CSV

Table 3 Image visibility recognition accuracy at various degrees of visibility

时段	项目	自动气象站观测能见度等级
时段	项目	无雾	轻雾	雾	大雾	浓雾
白天	样本量	561	1812	77	244	160
白天	准确率/%	88.59	92.72	76.62	59.43	81.25
夜间	样本量	727	1534	88	366	186
夜间	准确率/%	85.01	85.92	56.82	95.90	12.37

DownLoad: Download CSV

Table 4 Failed recognition of image visibility level

时段	自动气象站观测能见度等级	识别错误样本量
时段	自动气象站观测能见度等级	无雾	轻雾	雾	大雾	浓雾
白天	无雾		33	0	0	0
	轻雾	63		9	2	0
	雾	0	98		25	0
	大雾	0	0	7		30
	浓雾	0	0	3	7
夜间	无雾		113	0	0	0
	轻雾	108		10	1	0
	雾	0	96		7	0
	大雾	1	7	28		163
	浓雾	0	0	3	7

DownLoad: Download CSV

References

[1]	China Meteorological Administration. Specifications for Surface Meteorological Observation. Beijing: China Meteorological Press, 2003.
[2]	Jiang J, Zhang G P, Gao J B. Main influencing factors of visibility in Beijing. J Appl Meteor Sci, 2018, 29(2): 188-199. doi: 10.11898/1001-7313.20180206
[3]	Zhang H, Shi C E, Yang J, et al. Microphysical characteristics of fog with different intensities and their relationship with visibility in Shouxian County. Chinese J Atmos Sci, 2021, 45(6): 1217-1231. https://www.cnki.com.cn/Article/CJFDTOTAL-DQXK202106005.htm
[4]	Si P, Gao R X. A comparative evaluation on automatic and manual observations of fog and haze in Tianjin. J Appl Meteor Sci, 2015, 26(2): 240-246. doi: 10.11898/1001-7313.20150212
[5]	Xia D, Wu Z Q, Tan H B, et al. Analysis and correction of visibility measured by automatic observing system in Guangdong. Meteor Sci Technol, 2014, 42(1): 70-74. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ201401011.htm
[6]	Ding Y H, Liu Y J. Long term variation characteristics of fog and haze and their relationship with atmospheric humidity in China in recent 50 years. Sci China(Earth Sci), 2014, 44(1): 37-48. https://www.cnki.com.cn/Article/CJFDTOTAL-JDXK201401005.htm
[7]	Pan B F, Wang W, Li L, et al. Analysis of season of formation and characteristics of pollution about fog or haze at key cities in autumn and winter in China. Environment and Sustainable Development, 2013(1): 33-36. doi: 10.3969/j.issn.1673-288X.2013.01.007
[8]	Song H J, Gao Y Y, Chen Y Z. Traffic visibility estimation based on dynamic camera calibration. Chinese Journal of Computers, 2015, 38(6): 1172-1187. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201506007.htm
[9]	Li J, Ge J Q, Chen Q M. Visibility detection algorithm for road video. Computer Engineering, 2009, 35(16): 175-177. doi: 10.3969/j.issn.1000-3428.2009.16.063
[10]	Lü W T, Tao S C, Liu Y F, et al. Measurement of meteorological visibility based on digital camera technology-double brightness difference method and experimental research. Chinese J Atmos Sci, 2004, 28(4): 559-568. doi: 10.3878/j.issn.1006-9895.2004.04.08
[11]	Chen Z Z, Chen Q M. Video contrast visibility detection algorithm and its implementation based on camera self-calibration. Journal of Electronics & Information Technology, 2010, 32(12): 2907-2912. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX201012021.htm
[12]	Li B, Dong R, Chen Q M. Visibility detection based on traffic video contrast analysis without artificial markers. Journal of Computer-aided Design & Computer Graphics, 2009, 21(11): 1575-1582. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJF200911010.htm
[13]	Guo S S, Qi W X, Qi Y. Video visibility measurement method based on dark channel prior. Computer & Digital Engineering, 2014, 42(4): 694-697. doi: 10.3969/j.issn1672-9722.2014.04.036
[14]	Hu P, Yang X D. An algorithm for fast detecting expressway visibility. Journal of Highway and Transportation Research and Development, 2017, 34(4): 115-122. https://www.cnki.com.cn/Article/CJFDTOTAL-GLJK201704018.htm
[15]	Lu T S, Yang J, Deng M, et al. A visibility estimation method based on digital total-sky images. J Appl Meteor Sci, 2018, 29(6): 701-709. doi: 10.11898/1001-7313.20180606
[16]	Gao J J, Tian H, Wu H, et al. A visibility detection method using daytime road images based on brightness contrast and dark channel prior principle. Meteor Sci Technol, 2019, 47(3): 386-396. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ201903003.htm
[17]	Li Y, Chen H L. Review of machine learning approaches for modern agrometeorology. J Appl Meteor Sci, 2020, 31(3): 257-266. doi: 10.11898/1001-7313.20200301
[18]	Jin Z Q, Wang X M, Bao Y S, et al. Squall line identification method based on convolution neural network. J Appl Meteor Sci, 2021, 32(5): 580-591. doi: 10.11898/1001-7313.20210506
[19]	Yang L, Nan G Q, Chen M X, et al. The construction and comparison of high resolution precipitation type prediction models based on three machine learning methods. Acta Meteor Sinica, 2021, 79(6): 1022-1034. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202106008.htm
[20]	Mi Q C, Gao X N, Li Y, et al. Application of deep learning method to drought prediction. J Appl Meteor Sci, 2022, 33(1): 104-114. doi: 10.11898/1001-7313.20220109
[21]	Zhao L N, Lu S, Qi D, et al. Daily maximum air temperature forecast based on fully connected neural network. J Appl Meteor Sci, 2022, 33(3): 257-269. doi: 10.11898/1001-7313.20220301
[22]	Zhang Y B, Chen M X, Han L, et al. Multi-element deep learning fusion correction method for numerincal weather prediction. Acta Meteor Sinica, 2022, 80(1): 153-167. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202201011.htm
[23]	Liu H Z, Xu H, Bao H J, et al. Application of machine learning classification algorithm to precipitation-induced landslides forecasting. J Appl Meteor Sci, 2022, 33(3): 282-292. doi: 10.11898/1001-7313.20220303
[24]	Shi Y L, Wang B, Bu F. Atmospheric visibility measurement based on image feature. Journal of Nanjing University of Science and Technology, 2018, 42(5): 532-559. https://www.cnki.com.cn/Article/CJFDTOTAL-NJLG201805007.htm
[25]	Miao K C, Wang C H, Zhang Y L, et al. Road visibility estimation method based on AlexNet algorithm. Computer and Modernization, 2019, 286(6): 87-91. doi: 10.3969/j.issn.1006-2475.2019.06.015
[26]	General Administration of Quality Supervision, Inspection and Quarantine of the People's Republic of China, Standardization Administration. GB/T 27964-2011. Grade of Fog Forecast. Beijing: Standards Press of China, 2012.
[27]	Zhou F Y, Jin L P, Dong J. Review of convolutional neural network. Chinese Journal of Computers, 2017, 191(11): 27-29. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201706001.htm
[28]	Zhao H Y, Wu L H, Shi Y J, et al. Moving target detection method based on HSV color space. Modern Electronics Technique, 2013, 36(12): 45-48. doi: 10.3969/j.issn.1004-373X.2013.12.014
[29]	Fan W D, Zhou S B, Xin P C. Target tracking of HSV color space characteristic and Kalman filer. Computer Engineering and Applications, 2011, 47(13): 169-173. doi: 10.3778/j.issn.1002-8331.2011.13.048
[30]	Yuan C L, Xiong Z L, Zhou X H, et al. Study of infrared image edge detection based on Sobel operator. Laser & Nerared, 2009, 39(1): 85-87. doi: 10.3969/j.issn.1001-5078.2009.01.023
[31]	Ketkar N. Introduction to Keras//Deep Learning with Python. Berkeley: Apress, 2017: 97-111.
[32]	Wang X J, Guo G D. Comparative study of re-sampling methods for imbalanced data sets. Microcomputer Information, 2011, 27(12): 155-157. https://www.cnki.com.cn/Article/CJFDTOTAL-WJSJ201112064.htm
[33]	Zhou Z H. Machine Learning. Beijing: Tsinghua University Press, 2016.
[34]	Tang N, Lv Y. Median filering image denoising method based on noise pixel detection. Microcomputer and Application, 2015, 34(5): 35-38. doi: 10.3969/j.issn.1674-7720.2015.05.013
[35]	Cressman G P. An operational objective analysis system. Mon Wea Rev, 1959, 87(10): 367-374. doi: 10.1175/1520-0493(1959)087<0367:AOOAS>2.0.CO;2
[36]	Deng L. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 2012, 29(6): 141-142. doi: 10.1109/MSP.2012.2211477
[37]	Liu N, Xiong A Y, Zhang Q, et al. Development of basic dataset of severe convective weather for artificial intelligence training. J Appl Meteor Sci, 2021, 32(5): 530-541. doi: 10.11898/1001-7313.20210502
[38]	Long Y, Xia G S, Li S, et al. On creating benchmark dataset for aerial image interpretation: Reviews, guidances and million-AID. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 4205-4230. doi: 10.1109/JSTARS.2021.3070368