The Optimization of GRAPES Global Tangent Linear Model and Adjoint Model
-
摘要: 伴随技术是四维变分同化(4DVar)系统中计算代价函数梯度的最佳办法,切线性和伴随模式的效果和效率直接影响着4DVar系统的发展。基于GRAPES(Global and Regional Assimilation PrEdiction System)全球切线性和伴随模式1.0版本,利用GRAPES全球模式2.0版本在并行框架和性能等方面的改善,重新优化和设计了GRAPES全球切线性伴随模式2.0版本,提高了GRAPES全球切线性和伴随模式的效果和效率,优化了切线性模式程序结构,使其计算时间最优可控制在非线性模式的1.2倍以内;采用在切线性模式中保存基态的方法,重构了伴随模式的程序结构,使其计算时间最优控制在非线性模式的1.5倍以内;在GRAPES全球切线性物理过程的设计中,将线性物理过程的轨迹基态计算和切线性扰动计算解耦,提高了GRAPES全球切线性和伴随模式的计算效果和效率。Abstract: Adjoint models are widely applied in numerical weather prediction. For instance, in four-dimensional variational data assimilation (4DVar), they are the best method to efficiently determine optimal initial conditions. The minimization of the 4DVar cost function is solved with an iterative algorithm and is computationally demanding. Though the minimization is usually performed with a much lower resolution than in forecast model, obtaining the optimal model state requires dozens of iterations, and the model parallel efficiency must be fast enough. However, the parallel efficiency of GRAPES global tangent linear model and adjoint model version 1.0 based on GRAPES global non-linear model 1.0 is so low that it seriously impacts the development of GRAPES_4DVar. In order to reduce the computational cost, a new tangent linear model and adjoint model version 2.0 are re-designed and re-developed based on GRAPES global model version 2.0. By optimizing the program structure of tangent linear model, the calculating time of GRAPES tangent linear model can be best controlled within 1.2 times of GRAPES non-linear model's consumption with only dynamic framework. And by methods transferring the model base state and trajectory to the adjoint model, the calculating time of GRAPES adjoint model can be best controlled within 1.5 times of GRAPES non-linear model's consumption. Therefore, the new GRAPES tangent linear model and adjoint model version 2.0 are very successful in terms of computational efficiency to speed up the development of GRAPES_4DVar.In practical applications, the tangent linear model and adjoint model is run at a lower resolution than the non-linear model, since the dynamics is already simplified through the reduction in horizontal resolution, the linearized physics doesn't necessarily need to be exactly tangent to the full physics. In principle, physical parameterizations can already behave differently between non-linear and tangent-linear models due to the change in resolution. In order to reduce computational cost, it is often necessary to select different set of simplified linearized parameterizations with the full physical processes of GRAPES forecast model. By decoupling base states calculation in GRAPES and the perturbation calculation in the tangent linear and adjoint model, the computational cost of GRAPES tangent and adjoint model with simplified physical parameterizations increases only a little than no physical parameterizations versions, and the computational efficiency is higher than GRAPES forecast model with full physical parameterizations.
-
Key words:
- tangent linear model;
- adjoint model;
- 4DVar;
- GRAPES
-
图 3 位温扰动在模式第5层的6 h演化(a)无物理过程的非线性演变,(b)全物理过程的非线性演变,(c)无物理过程的切线性演变,(d)简化切线性物理过程的切线性演变
Fig. 3 6 h evolution of the potential temperature perturbation at the 5th level of model (a) non-linear evolution of no physical process, (b) non-linear evolution of all physical processes, (c) tangent evolution of no physical process, (d) tangent evolution of simple tangent physical process
表 1 Helmhots求解模块的切线性近似测试
Table 1 The tangent test of Helmhots subroutine
α F(α)(h(0)) F(α)(h(6)) 1.0 1.00396590233937211 0.998764138559583015 1.0-1 0.999634786924662788 0.999926134207449135 1.0-2 0.999951127242584059 0.999994765313335754 1.0-3 0.999994991727594429 0.999999500517701367 1.0-4 0.999999497605783549 0.999999950136184368 1.0-5 0.999999950154648043 0.999999995246000362 1.0-6 0.999999976297633264 1.00000000676183132 1.0-7 0.999999843023807067 1.00000000189906468 1.0-8 1.00000169720677556 1.00000119210668692 1.0-9 0.999984616091024181 0.999973064620637730 1.0-10 0.999860104875562095 1.00018956303390127 1.0-11 1.00204179083461287 1.00787797240779442 1.0-12 1.06901470470723736 1.13360506789842908 1.0-13 4.06883845840042380 7.47413649808233949 1.0-14 128.695572790297462 310.027208749653028 表 2 无物理过程下TLM和ADM的计算效率(单位:s)
Table 2 The parallel efficiency of TLM and ADM without physical processes (unit:s)
切线性和伴随模式 32核/节点 16核/节点 4节点 8节点 16节点 32节点 8节点 16节点 32节点 64节点 NLM 26.51 16.19 9.44 5.77 20.75 11.42 7.21 4.56 TLM 36.55 18.79 11.75 8.86 26.33 13.64 8.82 5.59 ADM 48.54 27.07 15.59 10.40 41.21 22.68 13.2 7.56 表 3 有物理过程的GRAPES全球模式的并行效率(单位:s)
Table 3 The parallel efficiency with physical processes (unit:s)
切线性和伴随模式 32核/节点 16核/节点 4节点 8节点 16节点 32节点 8节点 16节点 32节点 64节点 NLM 60.54 34.56 27.47 27.77 52.38 30.86 23.55 24.11 TLM 43.74 22.35 13.75 9.08 32.03 16.77 10.44 6.97 ADM 66.75 36.60 20.55 11.43 57.37 31.24 17.68 10.31 -
[1] Errico R M, VuKicevic T.Sensitivity analysis using an adjoint of the PSU-NCAR mesoscale model.Mon Wea Rev, 1992, 120:1644-1660. doi: 10.1175/1520-0493(1992)120<1644:SAUAAO>2.0.CO;2 [2] Errico R M.What is an adjoint model.Bull Amer Meteor Soc, 1997, 78:2577-2591. doi: 10.1175/1520-0477(1997)078<2577:WIAAM>2.0.CO;2 [3] Rabier F, Jarvinen H.The ECMWF operational implementation of four-dimensional variational assimilation.Ⅰ:Experimental results with simplified physics.Q J R Met Soc, 2000, 126(564):11-43. https://www.researchgate.net/publication/227624087_The_ECMWF_operational_implementation_of_four-dimensional_variational_assimilation_I_Experimental_results_with_simplified_physics [4] Molteni F R B, Palmer T N, Petroloagis T.The ECMWF ensemble prediction system:Methodlogy and validation.Q J R Met Soc, 1996, 119:269-298. https://www.researchgate.net/profile/R_Buizza/publication/236268963_The_ECMWF_Ensemble_Prediction_System_Methodology_and_validation/links/5589748108ae9076016f8ea6.pdf?inViewer=true&disableCoverPage=true&origin=publication_detail [5] Cardinal C.Forecast Sensitivity to Observation (FSO) as a Diagnostic Tool.ECMWF Tech Memo, 2009:26. http://www2.mmm.ucar.edu/wrf/users/wrfda/Tutorials/2010_Aug/docs/WRFDA_sensitivity.pdf [6] 陈德辉, 沈学顺.新一代数值预报系统GRAPES研究进展.应用气象学报, 2006, 17(6):773-777. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=200606125&flag=1 [7] 薛纪善, 陈德辉.数值预报系统GRAPES的科学设计与应用.北京:科学出版社, 2008. [8] 任迪生, 沈学顺, 薛纪善, 等.GRAPES伴随模式底层数据栈优化.应用气象学报, 2011, 22(3):362-366. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20110313&flag=1 [9] 蒋沁谷, 金之雁.GRAPES全球模式MPI与OpenMP混合并行方案.应用气象学报, 2014, 25(5):581-591. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20140507&flag=1 [10] 刘艳, 薛纪善, 张林, 等.GRAPES全球三维变分同化系统的检验与诊断.应用气象学报, 2016, 27(1):1-15. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20160101&flag=1 [11] 刘永柱, 沈学顺, 李晓莉.基于总能量模的GRAPES全球模式奇异向量扰动研究.气象学报, 2013, 71(3):517-526. http://www.cnki.com.cn/Article/CJFDTOTAL-QXXB201303014.htm [12] Janiskova M, Lopez P.Linearized Physics for Data Assimilation at ECMWF.ECMWF Tech Memo, 2012:26. https://www.researchgate.net/publication/289246750_Linearized_Physics_for_Data_Assimilation_at_ECMWF [13] Giering R, Kaminski T.Recipes for adjoint code construction.ACM Trans Math Software, 1998, 24(4):437-474. doi: 10.1145/293686.293695 [14] Laurent H.TAPENADE, Automatic Dierentiation by Program Transformation.http://www-sop.inria.fr/tropics.2007. [15] 宋君强, 伍湘君.GRAPES模式中Helmhothz方程两种求解方法的对比研究.计算机工程与科学, 2011, 33(11):65-70. http://www.cnki.com.cn/Article/CJFDTOTAL-JSJK201111017.htm [16] 徐国强, 陈德辉.GRAPES物理过程的优化试验及程序结构设计.科学通报, 2008(20):2428-2434. http://www.cnki.com.cn/Article/CJFDTOTAL-KXTB200820006.htm [17] 张林, 朱宗申.GRAPES模式切线性垂直扩散方案的误差分析和改进.应用气象学报, 2008, 19(2):194-200. http://qikan.camscma.cn/jams/ch/reader/view_abstract.aspx?file_no=20080235&flag=1 [18] 刘奇俊, 胡志晋, 周秀骥.HLAFS显式云降水方案及其对暴雨和云的模拟(I)云降水显式方案.应用气象学报, 2003, 14(增刊I):60-67. http://www.cnki.com.cn/Article/CJFDTotal-YYQX2003S1007.htm [19] Zou X.Tangent linear and adjoint of "on-off" processes and their feasibility for use in 4-dimensional variational data assimilation.Tells, 1997, 49:3-31. http://www.ingentaconnect.com/content/mksg/tea/1997/00000049/00000001/art00002