设为首页
加入收藏
A Low Visibility Recognition Algorithm Based on Surveillance Video
-
摘要: 为了利用大量视频监控设备提高能见度数据采集密度，提出一种基于实景图像转换的、采用简单卷积神经网络分类提取能见度等级的算法。该算法假设视频设备水平安装且具备开阔视野, 对原始视频图像进行水平分块，提取各分块的梯度、饱和度和亮度信息组成新的图像，基于简单卷积神经网络建模。采用2019年9月—2020年12月上海洋山港气象站29668张视频图像进行训练，建立识别模型，并采用2021年1—5月5757张视频图像对模型进行测试。采用该算法建立的模型参考雾的预报等级(GB/T 27964—2011)将能见度分为5个等级进行检验，白天准确率为87.99%，夜间准确率为81.32%，优于直接采用AlexNet模型。对1000 m以下低能见度天气的识别准确率达95%以上。利用现有的视频摄像头，可有效弥补气象站点能见度仪数据不足的问题，在气象业务上有一定的应用价值。Abstract: Low visibility has significant influences on highways, ferries, civil aviation, and other modes of transportation, and the visibility observation of meteorological departments is not dense enough to meet the monitoring needs of low visibility weather. Using existing video surveillance equipment to extract visibility data can save a significant amount of money on visibility instrument deployment and maintenance, improve data density, and provide finer data support for traffic and urban safety operations. Based on video live image conversion, a simple convolutional neural network classification approach is suggested to extract visibility levels. The algorithm assumes that the video devices are installed horizontally and have an open view, and it creates a new fixed-size image by dividing the original video image into horizontal chunks and extracting the gradient, color saturation, and brightness information from each horizontal chunk. A simple convolutional neural network is used to learn and develop a visibility level recognition model from the converted images. The model is trained by 29668 video images of Yangshan Port Weather Station in Shanghai from September 2019 to December 2020, and then tested with 5757 video images from January to May in 2021. The comparison indicates the recognition model generated with this technique has a greater accuracy than the recognition model built directly with AlexNet network. The model has an overall accuracy of 87.99% during daytime and 81.32% during nighttime when the observed visibility is classified into five levels of fog-free, light fog, fog, dense fog, and thick fog according to the fog forecasting level. The model's identification ability for no fog and light fog is high. However, because the scenery becomes nearly indistinguishable once dense fog appears at night, the model's recognition ability for dense fog level at night is poor, and it is easy to categorize it as a fog level mistakenly. Taking 1000 meters as the criterion of low visibility weather, the algorithm's accuracy is 96.18% during daytime and 96.14% during nighttime. The algorithm features a quick learning rate and ease of application, making it suitable for low visibility video image recognition in most open-field scenarios. The model is applied during a radiation fog in Shanghai on 13 April 2021. The video images of the sparse area of the automatic weather station installation are collected for visibility identification, and the visibility distribution map formed together with the existing automatic weather station visibility meter data is more complete and accurate, which demonstrates that the model established by this algorithm can effectively compensate the problem of insufficient density of the existing automatic station visibility meter data, and has certain application value in meteorological operations.
-
Key words:
- low visibility;
- image recognition;
- algorithm;
- convolutional neural network
-
图 1 2021年1月16日14：00洋山港气象站视频场景景物分布
Fig. 1 Distribution of video scene scenery at Yangshan Port Weather Station at 1400 BT 16 Jan 2021
图 2 裁剪后白天和夜间原始图像和梯度图个例
(a)白天原始图像，(b)白天梯度图，(c)夜间原始图像，(d)夜间梯度图
Fig. 2 Daytime and nighttime examples of cropped original images and gradient maps
(a)daytime original image, (b)daytime gradient map, (c)nighttime original image, (d)nighttime gradient map
图 3 白天不同能见度等级下处理后的典型图像
(a)无雾，(b)轻雾，(c)雾，(d)大雾，(e)浓雾
Fig. 3 Typical images after processing at various levels of visibility during daytime
(a)no fog,(b)light fog,(c)fog,(d)dense fog,(e)thick fog
图 4 夜间不同能见度等级下处理后的典型图像
(a)无雾，(b)轻雾，(c)雾，(d)大雾，(e)浓雾
Fig. 4 Typical images with various visibility levels after processing during nighttime
(a)no fog,(b)light fog,(c)fog,(d)dense fog,(e)thick fog
图 5 自动气象站能见度仪观测等级为雾时识别错误个例
Fig. 5 Examples of failed recognition when the visibility meter observation level is fog
图 6 2021年4月13日05:00能见度分布图(▲为能见度仪位置，$ \bullet $为视频设备位置)
(a)仅用能见度仪数据，(b)能见度仪数据和4个站点视频图像能见度解析数据
Fig. 6 The visibility distribution at 0500 BT 13 Apr 2021 (▲ is the visibility meter position,$ \bullet $ is the video position)
(a)using visibility meter data only, (b)using visibility meter data and 4 video images visibility analysis data
表 1 能见度分类标准
Table 1 The standard for visibility classification
分类编号 能见度范围/m 能见度等级名称 0 大于10000 无雾 1 1001~10000 轻雾 2 501~1000 雾 3 201~500 大雾 4 51~200 浓雾 5 0~50 强浓雾 
下载: 导出CSV
表 2 不同类型原有样本量和调整后样本量
Table 2 The total number of original and modified samples in each category
等级 分类编号 白天 夜间 原有样本量 调整后样本量 原有样本量 调整后样本量 无雾 0 10537 10537 9597 9597 轻雾 1 4997 4997 4159 4159 雾 2 54 6912 131 7546 大雾 3 78 6630 165 7574 浓雾 4 85 7225 165 7574
下载: 导出CSV
表 3 不同能见度等级下的图像能见度识别准确率
Table 3 Image visibility recognition accuracy at various degrees of visibility
时段 项目 自动气象站观测能见度等级 无雾 轻雾 雾 大雾 浓雾 白天 样本量 561 1812 77 244 160 准确率/% 88.59 92.72 76.62 59.43 81.25 夜间 样本量 727 1534 88 366 186 准确率/% 85.01 85.92 56.82 95.90 12.37
下载: 导出CSV
表 4 图像能见度等级识别错误样本分布
Table 4 Failed recognition of image visibility level
时段 自动气象站观测能见度等级 识别错误样本量 无雾 轻雾 雾 大雾 浓雾 白天 无雾 33 0 0 0 轻雾 63 9 2 0 雾 0 98 25 0 大雾 0 0 7 30 浓雾 0 0 3 7 夜间 无雾 113 0 0 0 轻雾 108 10 1 0 雾 0 96 7 0 大雾 1 7 28 163 浓雾 0 0 3 7
下载: 导出CSV
-
[1] 中国气象局.地面气象观测规范.北京: 气象出版社, 2003.China Meteorological Administration. Specifications for Surface Meteorological Observation. Beijing: China Meteorological Press, 2003. [2] 姜江, 张国平, 高金兵. 北京大气能见度的主要影响因子. 应用气象学报, 2018, 29(2): 188-199. doi: 10.11898/1001-7313.20180206Jiang J, Zhang G P, Gao J B. Main influencing factors of visibility in Beijing. J Appl Meteor Sci, 2018, 29(2): 188-199. doi: 10.11898/1001-7313.20180206 [3] 张浩, 石春娥, 杨军, 等. 寿县不同强度雾的微物理特征及其与能见度的关系. 大气科学, 2021, 45(6): 1217-1231. https://www.cnki.com.cn/Article/CJFDTOTAL-DQXK202106005.htmZhang H, Shi C E, Yang J, et al. Microphysical characteristics of fog with different intensities and their relationship with visibility in Shouxian County. Chinese J Atmos Sci, 2021, 45(6): 1217-1231. https://www.cnki.com.cn/Article/CJFDTOTAL-DQXK202106005.htm [4] 司鹏, 高润祥. 天津雾和霾自动观测与人工观测的对比评估. 应用气象学报, 2015, 26(2): 240-246. doi: 10.11898/1001-7313.20150212Si P, Gao R X. A comparative evaluation on automatic and manual observations of fog and haze in Tianjin. J Appl Meteor Sci, 2015, 26(2): 240-246. doi: 10.11898/1001-7313.20150212 [5] 夏冬, 吴志权, 谭浩波, 等. 广东省能见度自动观测系统资料评估分析与订正. 气象科技, 2014, 42(1): 70-74. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ201401011.htmXia D, Wu Z Q, Tan H B, et al. Analysis and correction of visibility measured by automatic observing system in Guangdong. Meteor Sci Technol, 2014, 42(1): 70-74. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ201401011.htm [6] 丁一汇, 柳艳菊. 近50年我国雾和霾的长期变化特征及其与大气湿度的关系. 中国科学(地球科学), 2014, 44(1): 37-48. https://www.cnki.com.cn/Article/CJFDTOTAL-JDXK201401005.htmDing Y H, Liu Y J. Long term variation characteristics of fog and haze and their relationship with atmospheric humidity in China in recent 50 years. Sci China(Earth Sci), 2014, 44(1): 37-48. https://www.cnki.com.cn/Article/CJFDTOTAL-JDXK201401005.htm [7] 潘本锋, 汪巍, 李亮, 等. 我国大中型城市秋冬季节雾霾天气污染特征与成因分析. 环境与可持续发展, 2013(1): 33-36. doi: 10.3969/j.issn.1673-288X.2013.01.007Pan B F, Wang W, Li L, et al. Analysis of season of formation and characteristics of pollution about fog or haze at key cities in autumn and winter in China. Environment and Sustainable Development, 2013(1): 33-36. doi: 10.3969/j.issn.1673-288X.2013.01.007 [8] 宋洪军, 郜园园, 陈阳舟. 基于摄像机动态标定的交通能见度估计. 计算机学报, 2015, 38(6): 1172-1187. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201506007.htmSong H J, Gao Y Y, Chen Y Z. Traffic visibility estimation based on dynamic camera calibration. Chinese Journal of Computers, 2015, 38(6): 1172-1187. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201506007.htm [9] 李佳, 葛嘉琦, 陈启美. 路况视频能见度检测算法. 计算机工程, 2009, 35(16): 175-177. doi: 10.3969/j.issn.1000-3428.2009.16.063Li J, Ge J Q, Chen Q M. Visibility detection algorithm for road video. Computer Engineering, 2009, 35(16): 175-177. doi: 10.3969/j.issn.1000-3428.2009.16.063 [10] 吕伟涛, 陶善昌, 刘亦风, 等. 基于数字摄像技术测量气象能见度——双亮度差方法和试验研究. 大气科学, 2004, 28(4): 559-568. doi: 10.3878/j.issn.1006-9895.2004.04.08Lü W T, Tao S C, Liu Y F, et al. Measurement of meteorological visibility based on digital camera technology-double brightness difference method and experimental research. Chinese J Atmos Sci, 2004, 28(4): 559-568. doi: 10.3878/j.issn.1006-9895.2004.04.08 [11] 陈钊正, 陈启美. 基于摄像机自标定的视频对比度能见度检测算法与实现. 电子与信息学报, 2010, 32(12): 2907-2912. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX201012021.htmChen Z Z, Chen Q M. Video contrast visibility detection algorithm and its implementation based on camera self-calibration. Journal of Electronics & Information Technology, 2010, 32(12): 2907-2912. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX201012021.htm [12] 李勃, 董蓉, 陈启美. 无需人工标记的视频对比度道路能见度检测. 计算机辅助设计与图形学学报, 2009, 21(11): 1575-1582. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJF200911010.htmLi B, Dong R, Chen Q M. Visibility detection based on traffic video contrast analysis without artificial markers. Journal of Computer-aided Design & Computer Graphics, 2009, 21(11): 1575-1582. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJF200911010.htm [13] 郭尚书, 齐文新, 齐宇. 基于暗通道先验的视频能见度测量方法. 计算机与数字工程, 2014, 42(4): 694-697. doi: 10.3969/j.issn1672-9722.2014.04.036Guo S S, Qi W X, Qi Y. Video visibility measurement method based on dark channel prior. Computer & Digital Engineering, 2014, 42(4): 694-697. doi: 10.3969/j.issn1672-9722.2014.04.036 [14] 胡平, 杨旭东. 高速公路能见度快速检测算法. 公路交通科技, 2017, 34(4): 115-122. https://www.cnki.com.cn/Article/CJFDTOTAL-GLJK201704018.htmHu P, Yang X D. An algorithm for fast detecting expressway visibility. Journal of Highway and Transportation Research and Development, 2017, 34(4): 115-122. https://www.cnki.com.cn/Article/CJFDTOTAL-GLJK201704018.htm [15] 陆天舒, 杨俊, 邓敏, 等. 基于全视野数字图像的能见度估算方法. 应用气象学报, 2018, 29(6): 701-709. doi: 10.11898/1001-7313.20180606Lu T S, Yang J, Deng M, et al. A visibility estimation method based on digital total-sky images. J Appl Meteor Sci, 2018, 29(6): 701-709. doi: 10.11898/1001-7313.20180606 [16] 郜婧婧, 田华, 吴昊, 等. 基于亮度对比度和暗原色先验原理的白天道路图像能见度检测方法. 气象科技, 2019, 47(3): 386-396. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ201903003.htmGao J J, Tian H, Wu H, et al. A visibility detection method using daytime road images based on brightness contrast and dark channel prior principle. Meteor Sci Technol, 2019, 47(3): 386-396. https://www.cnki.com.cn/Article/CJFDTOTAL-QXKJ201903003.htm [17] 李颖, 陈怀亮. 机器学习技术在现代农业气象中的应用. 应用气象学报, 2020, 31(3): 257-266. doi: 10.11898/1001-7313.20200301Li Y, Chen H L. Review of machine learning approaches for modern agrometeorology. J Appl Meteor Sci, 2020, 31(3): 257-266. doi: 10.11898/1001-7313.20200301 [18] 金子琪, 王新敏, 鲍艳松, 等. 基于卷积神经网络的飑线识别算法. 应用气象学报, 2021, 32(5): 580-591. doi: 10.11898/1001-7313.20210506Jin Z Q, Wang X M, Bao Y S, et al. Squall line identification method based on convolution neural network. J Appl Meteor Sci, 2021, 32(5): 580-591. doi: 10.11898/1001-7313.20210506 [19] 杨璐, 南刚强, 陈明轩, 等. 基于三种机器学习方法的降水相态高分辨率格点预报模型的构建及对比分析. 气象学报, 2021, 79(6): 1022-1034. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202106008.htmYang L, Nan G Q, Chen M X, et al. The construction and comparison of high resolution precipitation type prediction models based on three machine learning methods. Acta Meteor Sinica, 2021, 79(6): 1022-1034. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202106008.htm [20] 米前川, 高西宁, 李玥, 等. 深度学习方法在干旱预测中的应用. 应用气象学报, 2022, 33(1): 104-114. doi: 10.11898/1001-7313.20220109Mi Q C, Gao X N, Li Y, et al. Application of deep learning method to drought prediction. J Appl Meteor Sci, 2022, 33(1): 104-114. doi: 10.11898/1001-7313.20220109 [21] 赵琳娜, 卢姝, 齐丹, 等. 基于全连接神经网络方法的日最高气温预报. 应用气象学报, 2022, 33(3): 257-269. doi: 10.11898/1001-7313.20220301Zhao L N, Lu S, Qi D, et al. Daily maximum air temperature forecast based on fully connected neural network. J Appl Meteor Sci, 2022, 33(3): 257-269. doi: 10.11898/1001-7313.20220301 [22] 张延彪, 陈明轩, 韩雷, 等. 数值天气预报多要素深度学习融合订正方法. 气象学报, 2022, 80(1): 153-167. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202201011.htmZhang Y B, Chen M X, Han L, et al. Multi-element deep learning fusion correction method for numerincal weather prediction. Acta Meteor Sinica, 2022, 80(1): 153-167. https://www.cnki.com.cn/Article/CJFDTOTAL-QXXB202201011.htm [23] 刘海知, 徐辉, 包红军, 等. 机器学习分类算法在降雨型滑坡预报中的应用. 应用气象学报, 2022, 33(3): 282-292. doi: 10.11898/1001-7313.20220303Liu H Z, Xu H, Bao H J, et al. Application of machine learning classification algorithm to precipitation-induced landslides forecasting. J Appl Meteor Sci, 2022, 33(3): 282-292. doi: 10.11898/1001-7313.20220303 [24] 石玉立, 王彬, 卜帆. 基于图像特征的大气能见度估算方法. 南京理工大学学报, 2018, 42(5): 532-559. https://www.cnki.com.cn/Article/CJFDTOTAL-NJLG201805007.htmShi Y L, Wang B, Bu F. Atmospheric visibility measurement based on image feature. Journal of Nanjing University of Science and Technology, 2018, 42(5): 532-559. https://www.cnki.com.cn/Article/CJFDTOTAL-NJLG201805007.htm [25] 苗开超, 王传辉, 张亚力, 等. 基于AlexNet算法的道路能见度估测方法. 计算机与现代化, 2019, 286(6): 87-91. doi: 10.3969/j.issn.1006-2475.2019.06.015Miao K C, Wang C H, Zhang Y L, et al. Road visibility estimation method based on AlexNet algorithm. Computer and Modernization, 2019, 286(6): 87-91. doi: 10.3969/j.issn.1006-2475.2019.06.015 [26] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 27964—2011雾的预报等级. 北京: 中国标准出版社, 2012.General Administration of Quality Supervision, Inspection and Quarantine of the People's Republic of China, Standardization Administration. GB/T 27964-2011. Grade of Fog Forecast. Beijing: Standards Press of China, 2012. [27] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述. 计算机学报, 2017, 191(11): 27-29. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201706001.htmZhou F Y, Jin L P, Dong J. Review of convolutional neural network. Chinese Journal of Computers, 2017, 191(11): 27-29. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201706001.htm [28] 赵红雨, 吴乐华, 史燕军, 等. 基于HSV颜色空间的运动目标检测方法. 现代电子技术, 2013, 36(12): 45-48. doi: 10.3969/j.issn.1004-373X.2013.12.014Zhao H Y, Wu L H, Shi Y J, et al. Moving target detection method based on HSV color space. Modern Electronics Technique, 2013, 36(12): 45-48. doi: 10.3969/j.issn.1004-373X.2013.12.014 [29] 范五东, 周尚波, 辛培宸. HSV颜色空间特征与Kalman滤波融合的目标跟踪. 计算机工程与应用, 2011, 47(13): 169-173. doi: 10.3778/j.issn.1002-8331.2011.13.048Fan W D, Zhou S B, Xin P C. Target tracking of HSV color space characteristic and Kalman filer. Computer Engineering and Applications, 2011, 47(13): 169-173. doi: 10.3778/j.issn.1002-8331.2011.13.048 [30] 袁春兰, 熊宗龙, 周雪花, 等. 基于Sobel算子的图像边缘检测研究. 激光与红外, 2009, 39(1): 85-87. doi: 10.3969/j.issn.1001-5078.2009.01.023Yuan C L, Xiong Z L, Zhou X H, et al. Study of infrared image edge detection based on Sobel operator. Laser & Nerared, 2009, 39(1): 85-87. doi: 10.3969/j.issn.1001-5078.2009.01.023 [31] Ketkar N. Introduction to Keras//Deep Learning with Python. Berkeley: Apress, 2017: 97-111. [32] 王晓娟, 郭躬德. 不平衡数据采样方法的对比学习. 微计算机信息, 2011, 27(12): 155-157. https://www.cnki.com.cn/Article/CJFDTOTAL-WJSJ201112064.htmWang X J, Guo G D. Comparative study of re-sampling methods for imbalanced data sets. Microcomputer Information, 2011, 27(12): 155-157. https://www.cnki.com.cn/Article/CJFDTOTAL-WJSJ201112064.htm [33] 周志华. 机器学习. 北京: 清华大学出版社, 2016.Zhou Z H. Machine Learning. Beijing: Tsinghua University Press, 2016. [34] 唐宁, 吕洋. 基于噪点检测的中值滤波图像去噪方法. 微型机与应用, 2015, 34(5): 35-38. doi: 10.3969/j.issn.1674-7720.2015.05.013Tang N, Lv Y. Median filering image denoising method based on noise pixel detection. Microcomputer and Application, 2015, 34(5): 35-38. doi: 10.3969/j.issn.1674-7720.2015.05.013 [35] Cressman G P. An operational objective analysis system. Mon Wea Rev, 1959, 87(10): 367-374. doi: 10.1175/1520-0493(1959)087<0367:AOOAS>2.0.CO;2 [36] Deng L. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 2012, 29(6): 141-142. doi: 10.1109/MSP.2012.2211477 [37] 刘娜, 熊安元, 张强, 等. 强对流天气人工智能应用训练基础数据集构建. 应用气象学报, 2021, 32(5): 530-541. doi: 10.11898/1001-7313.20210502Liu N, Xiong A Y, Zhang Q, et al. Development of basic dataset of severe convective weather for artificial intelligence training. J Appl Meteor Sci, 2021, 32(5): 530-541. doi: 10.11898/1001-7313.20210502 [38] Long Y, Xia G S, Li S, et al. On creating benchmark dataset for aerial image interpretation: Reviews, guidances and million-AID. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 4205-4230. doi: 10.1109/JSTARS.2021.3070368 -
图(6) / 表(4)
计量
- 摘要浏览量: 2108
- HTML全文浏览量: 255
- PDF下载量: 192
- 被引次数: 0
