ARQCS Starting Strategy and Its Relationship with Computing Resource Cost

AWS Observation Data Real-time Quality Control System (ARQCS) is an operational real-time meteorological data application system under IBM P570 high performance computing (HPC) Oracle 11g database platform. Functions including data decoding, database inserting, quality control (QC), storage management and share service are provided for more than 30000 AWS all over China. In 2009, when ARQCS is firstly built, QC methods including boundary value check, internal consistency check, time consistency check and spatial consistency check is applied to only 1 element of hourly precipitation. And the starting strategy is a static one, which start ARQCS at the 15th, 25th, 35th, 45th and 55th minute every hour. Later in 2010, QC methods of other important meteorological elements including air temperature, air pressure, humidity, wind direction and speed get to be applied in ARQCS. Meanwhile, the system computing logic is made more complex after 2 times of updating in 2011 and 2012. Now, it is planned to extend ARQCS to 158 elements in 11 classes totally, which need more calculating resources accordingly. To guarantee QC capability and service timeliness of ARQCS in a high level under limited computing resources, a series of schemes are designed and investigated. System log under IBM P570 HPC Oracle database environment from 1st April to 30th Sep in 2012 is used to analyze ARQCS performance. It is found that the database entry rate (ER) of AWS data exhibits a trapezoid shaped distribution, and variance of ER is large from the 5th to the 20th minute in one hour, which means accumulated ER at the 15th minute is unstable and a low accumulated ER may be got if ARQCS starts at this time. It also indicates that an accumulated ER of 95% is very possible (84.89%) to get before the 20th minute, and accumulated ER is increased by only 1.36% after the 20th minute in average. So a new dynamic starting strategy is employed, that ARQCS starts for the first time when accumulated ER gets more than 95% or until the 20th minute, and starts for the second time at the 55th minute. With this approach, the possibility for accumulated ER over 95% at the 1st QC starting is increased by 29% (from 66.38% to 95.83%). And the average 1st QC starting time is 20.6 seconds before the 15th minute in original static starting strategy. Also, less number of starts from 5 to 2 decrease the CPU time cost from 26.5 minutes to 10.2 minutes per hour, which means saving 391 minutes CPU time per day. It is concluded that the dynamic starting strategy is effective for ARQCS starting adaptively and ensures system robustness.