Design and Application of GISC Beijing Global Cache

The exponential growth of meteorological data poses significant challenges, rendering the existing WIS(World Meteorological Organization Information System)/GTS(Global Telecommunication System) framework increasingly inadequate to meet the future demands of real-time global meteorological data interaction. In response, World Meteorological Organization (WMO) has initiated the crucial process of replacing WIS/GTS with the advanced WMO Information System 2.0 (WIS2.0) in 2023. CMA(China Meteorological Administration) has committed to constructing 4 essential global service systems that support WIS2.0. The key to this transformation is the global data cache system, which has been meticulously developed by Beijing GISC (Global Information System Center). Its primary objective is to significantly enhance the global capacity for collecting meteorological data. This sophisticated system integrates essential functionalities, including proactive data discovery, streamlined and efficient downloading mechanisms, and intelligent cache processing. A comprehensive analysis of the construction background offers valuable insights into the system’s design philosophy. The global data cache’s overall architecture is introduced, including its core functionalities and the underlying technical implementation. The global cache leverages the message notification-driven global data active discovery technology inherent in MQTT (message queuing telemetry transport) protocol to establish a high-performance, low-latency, and highly available model for message collection and transmission. It analyzes the global meteorological data sharing mechanism, extracts core fields for feature value analysis to perform data cleansing, identifies and eliminates duplicates through sorting and comparison of similar elements, and employs file storage technology to ensure data integrity while providing a data access interface for the back-end system. Additionally, a standardized thematic hierarchy is established, handling notifications for the arrival of global data and metadata. It offers global users with high-throughput, high-concurrency, and high-timeliness data services. In October 2024, after extensive tests conducted by the expert group of WMO, the global data cache established by Beijing GISC successfully passed all functional and performance evaluations. Making China the first country among the six global data caches to fully meet the requirements of the global data cache assessments. Currently, the system actively ingests data from 82 WIS2 nodes distributed across the globe. It is a volume of approximately 2 TB of cached data daily, while supporting an average of over 2 million data download requests per day from 20 WIS2 nodes worldwide. In the future, the global data cache is expected to gradually take on the essential functions that were previously managed by GTS. By leveraging the powerful big data cloud platform Tianqing. The system will deliver enhanced services to the national meteorological department, providing robust, full-spectrum support for the CMA’s strategic vision of “global monitoring, global forecasting, global service” Concurrently, efforts will be intensified to actively expand global user services, further elevate the international influence of Beijing GISC, and provide dedicated assistance to countries within its area of responsibility, ensuring a smooth and successful migration to the WIS2.0 framework.