气象大数据云平台算法集约化环境设计与应用

Design and Application of Algorithm Intensive Environment for CMA Big Data and Cloud Platform

  • 摘要: 气象业务系统集约化发展和“云+端”业务技术体制改革是实现气象业务高质量发展的重要措施。2020年中国气象局提出构建以气象大数据云平台为云、气象业务系统为端的“云+端”业务技术体制, 明确气象大数据云平台作为关键基础技术平台的定位。加工流水线作为气象算法的集约化环境, 应用数算一体、高效任务调度、可视化流程编排、容器等技术, 实现气象算法的统一管理与高效集约调度运行。2021年加工流水线业务运行, 支撑全国202个业务系统的实时运行, 业务系统性能提升1~10倍, 集约化程度显著提高, 对提升业务系统的运行效率、增强业务系统的协同性、加速“云+端”业务技术体制改革进程和推进气象业务集约发展发挥了重要支撑作用。

     

    Abstract: With the advancement of meteorological services, various product processing systems and supporting data management systems have been developed for different business systems. However, this has led to the problem of system non-intensification. The lack of intensification in meteorological services can result in inconsistent data standards and make the operation and maintenance more challenging, which can lead to significant waste in investment due to duplicate data storage and inconsistent data caused by untimely synchronization. Moreover, the lack of information and technology hinders the integration of upstream and downstream businesses. The intensive development of meteorological business systems and the reform of "cloud+end" technology system are important measures to achieve high-quality development of meteorological business. China Meteorological Administration proposed to build a "cloud+end" technology system in 2020. CMA Big Data and Cloud Platform serves as the cloud, while the meteorological business system is the end, clarifying the positioning of CMA Big Data and Cloud Platform as a key foundational technology platform. The data processing line (DPL) is an intensive environment for meteorological algorithms. It addresses business needs such as efficient and stable processing of data products, data sharing and collaboration among business systems, and efficient and intensive business applications. To achieve this goal, algorithm libraries and task control functions are established by utilizing technologies such as integrated digital and computing, efficient task scheduling, visual process arrangement, and containers. The algorithm library facilitates the standardization, unified management and sharing of algorithms. Task control supports multiple scheduling strategies with high reliability and fault tolerance, enabling efficient and stable scheduling operations for algorithms. All functions mentioned above are in the form of interfaces for the application frontend. At the same time, based on the meteorological business comprehensive monitoring system (referred to as Tianjing) enables automatic collection of algorithm operation status and detection of abnormal alarms. Since its operation in 2021, the processing assembly line has facilitated the real-time operation of 202 business systems nationwide, resulting in a performance improvement of 1-10 times and a significant increase in efficiency. It plays an important supporting role in improving the operational efficiency of business systems, enhancing their collaboration, accelerating the process of "cloud+end" business technology system reform, and promoting the intensive development of meteorological business.

     

/

返回文章
返回