Comprehensive Consistency Method of Data Quality Controlling with Its Application to Daily Temperature

Due to the historical daily temperature data playing an important role in climate analysis and climate change research, the data quality is attached more importance. At present the daily temperature data are checked for quality control using the traditional methods in China, lacking a systematic and comprehensive method to pick up the outliers data hidden in the historical temperature data. These error data in the daily temperature affect data application, therefore, it's necessary to carry out the research of new quality control method.Using linear regression model and historical daily temperature (average temperature, maximum temperature and minimum temperature) data of the neighbouring stations in the same period, a quality check algorithm based on linear regression estimation method is designed, which includes both time consistency check and spatial consistency check in quality control of meteorological observational data. To further enhance the detection performance of data quality check, a comprehensive consistency check method is developed based on this algorithm, which adds internal consistency check that refers the variation of related meteorological elements such as daily temperature (average temperature, maximum temperature and minimum temperature), precipitation and sunshine duration to check data quality.Using the data seeded errors check test and compared with spatial regression test, the method of linear regression data quality control algorithm has higher error data check performance. The algorithm can detect suspicious data that is about 3℃ difference from the correct value on the temperature.Through data quality control practices and analysis on historical data, the comprehensive consistency check method has the following advantages: The flagged rates of Type Ⅰ errors are lower, thus reducing false detection rate of that the correct data flagged as error data; the logical relationship are kept with time consistency, internal consistency, and spatial consistency in data quality control process, and these three methods of checking the consistency of data quality are as a whole at the same time; the weather factors are referred, thus reducing the impact on data quality of small-scale weather phenomena which can flag data incorrectly. Therefore, the method of comprehensive consistency data quality control, which compared to the traditional data quality control method, has higher error detection performance.The algorithm achieves good progress on the applications of daily temperature data from 251 weather stations from 1961 to 2009 in Hubei, Hunan and Henan provinces. Detection of outliers in the average temperature is 0.001%, that in the maximum temperature is 0.05%, and that in the minimum temperature is 0.04%.