Abstract. Accurate specification of the error statistics required for data assimilation remains an ongoing challenge, partly because their estimation is an underdetermined problem that requires statistical assumptions. Even with the common assumption that background and observation errors are uncorrelated, the problem remains underdetermined. One natural question that could arise is as follows: can the increasing amount of overlapping observations or other datasets help to reduce the total number of statistical assumptions, or do they introduce more statistical unknowns? In order to answer this question, this paper provides a conceptual view on the statistical error estimation problem for multiple collocated datasets, including a generalized mathematical formulation, an illustrative demonstration with synthetic data, and guidelines for setting up and solving the problem. It is demonstrated that the required number of statistical assumptions increases linearly with the number of datasets. However, the number of error statistics that can be estimated increases quadratically, allowing for an estimation of an increasing number of error cross-statistics between datasets for more than three datasets. The presented generalized estimation of full error covariance and cross-covariance matrices between datasets does not necessarily accumulate the uncertainties of assumptions among error estimations of multiple datasets.
{"title":"How far can the statistical error estimation problem be closed by collocated data?","authors":"Annika Vogel, Richard Ménard","doi":"10.5194/npg-30-375-2023","DOIUrl":"https://doi.org/10.5194/npg-30-375-2023","url":null,"abstract":"Abstract. Accurate specification of the error statistics required for data assimilation remains an ongoing challenge, partly because their estimation is an underdetermined problem that requires statistical assumptions. Even with the common assumption that background and observation errors are uncorrelated, the problem remains underdetermined. One natural question that could arise is as follows: can the increasing amount of overlapping observations or other datasets help to reduce the total number of statistical assumptions, or do they introduce more statistical unknowns? In order to answer this question, this paper provides a conceptual view on the statistical error estimation problem for multiple collocated datasets, including a generalized mathematical formulation, an illustrative demonstration with synthetic data, and guidelines for setting up and solving the problem. It is demonstrated that the required number of statistical assumptions increases linearly with the number of datasets. However, the number of error statistics that can be estimated increases quadratically, allowing for an estimation of an increasing number of error cross-statistics between datasets for more than three datasets. The presented generalized estimation of full error covariance and cross-covariance matrices between datasets does not necessarily accumulate the uncertainties of assumptions among error estimations of multiple datasets.","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":"169 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135014181","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abstract. Until the 1980s, scaling notions were restricted to self-similar homogeneous special cases. I review developments over the last decades, especially in multifractals and generalized scale invariance (GSI). The former is necessary for characterizing and modelling strongly intermittent scaling processes, while the GSI formalism extends scaling to strongly anisotropic (especially stratified) systems. Both of these generalizations are necessary for atmospheric applications. The theory and some of the now burgeoning empirical evidence in its favour are reviewed. Scaling can now be understood as a very general symmetry principle. It is needed to clarify and quantify the notion of dynamical regimes. In addition to the weather and climate, there is an intermediate “macroweather regime”, and at timescales beyond the climate regime (up to Milankovitch scales), there is a macroclimate and megaclimate regime. By objectively distinguishing weather from macroweather, it answers the question “how long does weather last?”. Dealing with anisotropic scaling systems – notably atmospheric stratification – requires new (non-Euclidean) definitions of the notion of scale itself. These are needed to answer the question “how big is a cloud?”. In anisotropic scaling systems, morphologies of structures change systematically with scale even though there is no characteristic size. GSI shows that it is unwarranted to infer dynamical processes or mechanisms from morphology. Two “sticking points” preventing more widespread acceptance of the scaling paradigm are also discussed. The first is an often implicit phenomenological “scalebounded” thinking that postulates a priori the existence of new mechanisms, processes every factor of 2 or so in scale. The second obstacle is the reluctance to abandon isotropic theories of turbulence and accept that the atmosphere's scaling is anisotropic. Indeed, there currently appears to be no empirical evidence that the turbulence in any atmospheric field is isotropic. Most atmospheric scientists rely on general circulation models, and these are scaling – they inherited the symmetry from the (scaling) primitive equations upon which they are built. Therefore, the real consequence of ignoring wide-range scaling is that it blinds us to alternative scaling approaches to macroweather and climate – especially to new models for long-range forecasts and to new scaling approaches to climate projections. Such stochastic alternatives are increasingly needed, notably to reduce uncertainties in climate projections to the year 2100.
{"title":"Review article: Scaling, dynamical regimes, and stratification. How long does weather last? How big is a cloud?","authors":"S. Lovejoy","doi":"10.5194/npg-30-311-2023","DOIUrl":"https://doi.org/10.5194/npg-30-311-2023","url":null,"abstract":"Abstract. Until the 1980s, scaling notions were restricted to\u0000self-similar homogeneous special cases. I review developments over the last\u0000decades, especially in multifractals and generalized scale invariance (GSI). The former is necessary for characterizing and modelling strongly\u0000intermittent scaling processes, while the GSI formalism extends scaling to strongly anisotropic (especially stratified) systems. Both of these\u0000generalizations are necessary for atmospheric applications. The theory and\u0000some of the now burgeoning empirical evidence in its favour are reviewed. Scaling can now be understood as a very general symmetry principle. It is\u0000needed to clarify and quantify the notion of dynamical regimes. In addition\u0000to the weather and climate, there is an intermediate “macroweather regime”, and at timescales beyond the climate regime (up to Milankovitch scales), there is a macroclimate and megaclimate regime. By objectively\u0000distinguishing weather from macroweather, it answers the question “how long does weather last?”. Dealing with anisotropic scaling systems – notably\u0000atmospheric stratification – requires new (non-Euclidean) definitions of\u0000the notion of scale itself. These are needed to answer the question “how\u0000big is a cloud?”. In anisotropic scaling systems, morphologies of structures change systematically with scale even though there is no characteristic\u0000size. GSI shows that it is unwarranted to infer dynamical processes or\u0000mechanisms from morphology. Two “sticking points” preventing more widespread acceptance of the scaling paradigm are also discussed. The first is an often implicit\u0000phenomenological “scalebounded” thinking that postulates a priori the existence of\u0000new mechanisms, processes every factor of 2 or so in scale. The second obstacle is the reluctance to abandon isotropic theories of turbulence and\u0000accept that the atmosphere's scaling is anisotropic. Indeed, there currently appears to be no empirical evidence that the turbulence in any atmospheric\u0000field is isotropic. Most atmospheric scientists rely on general circulation models, and these are scaling – they inherited the symmetry from the (scaling) primitive\u0000equations upon which they are built. Therefore, the real consequence of\u0000ignoring wide-range scaling is that it blinds us to alternative scaling approaches to macroweather and climate – especially to new models for long-range forecasts and to new scaling approaches to climate projections. Such\u0000stochastic alternatives are increasingly needed, notably to reduce uncertainties in climate projections to the year 2100.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43072735","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abstract. The space spanned by the background ensemble provides a basis for correcting forecast errors in the ensemble Kalman filter. However, the ensemble space may not fully capture the forecast errors due to the limited ensemble size and systematic model errors, which affect the assimilation performance. This study proposes a new algorithm to generate pseudomembers to properly expand the ensemble space during the analysis step. The pseudomembers adopt vectors orthogonal to the original ensemble and are included in the ensemble using the centered spherical simplex ensemble method. The new algorithm is investigated with a six-member ensemble Kalman filter implemented in the 40-variable Lorenz model. Our results suggest that the ensemble singular vector, the ensemble mean vector, and their orthogonal components can serve as effective pseudomembers for improving the analysis accuracy, especially when the background has large errors.
{"title":"Using orthogonal vectors to improve the ensemble space of the ensemble Kalman filter and its effect on data assimilation and forecasting","authors":"Y. Cheng, Shu‐Chih Yang, Zhe Lin, Yung-An Lee","doi":"10.5194/npg-30-289-2023","DOIUrl":"https://doi.org/10.5194/npg-30-289-2023","url":null,"abstract":"Abstract. The space spanned by the background ensemble provides a basis for\u0000correcting forecast errors in the ensemble Kalman filter. However, the\u0000ensemble space may not fully capture the forecast errors due to the limited\u0000ensemble size and systematic model errors, which affect the assimilation\u0000performance. This study proposes a new algorithm to generate pseudomembers\u0000to properly expand the ensemble space during the analysis step. The\u0000pseudomembers adopt vectors orthogonal to the original ensemble and are\u0000included in the ensemble using the centered spherical simplex ensemble\u0000method. The new algorithm is investigated with a six-member ensemble Kalman\u0000filter implemented in the 40-variable Lorenz model. Our results suggest that\u0000the ensemble singular vector, the ensemble mean vector, and their orthogonal\u0000components can serve as effective pseudomembers for improving the analysis\u0000accuracy, especially when the background has large errors.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-07-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49551585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abstract. The pseudopotential method is used to derive electron hole structures in a suprathermal plasma with a regularized κ probability distribution function background. The regularized character allows the exploration of small κ values beyond the standard suprathermal case for which κ>3/2 is a necessary condition. We found the nonlinear dispersion relation yielding the amplitude of the electrostatic potential in terms of the remaining parameters, in particular the drift velocity, the wavenumber and the spectral index. Periodic, solitary wave, drifting and non-drifting solutions have been identified. In the linear limit, the dispersion relation yields generalized Langmuir and electron acoustic plasma modes. Standard electron hole structures are regained in the κ≫1 limit.
{"title":"Electron holes in a regularized kappa background","authors":"F. Haas, H. Fichtner, K. Scherer","doi":"10.5194/npg-30-277-2023","DOIUrl":"https://doi.org/10.5194/npg-30-277-2023","url":null,"abstract":"Abstract. The pseudopotential method is used to derive electron hole structures in a suprathermal plasma with a regularized κ probability\u0000distribution function background. The regularized character allows the exploration of small κ values beyond the standard suprathermal case for which κ>3/2 is a necessary condition. We found the nonlinear dispersion relation yielding the amplitude of the electrostatic\u0000potential in terms of the remaining parameters, in particular the drift velocity, the wavenumber and the spectral index. Periodic, solitary wave,\u0000drifting and non-drifting solutions have been identified. In the linear limit, the dispersion relation yields generalized Langmuir and electron\u0000acoustic plasma modes. Standard electron hole structures are regained in the κ≫1 limit.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44174478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-07-11DOI: 10.5194/egusphere-2023-1519
Amandine Kaiser, Nikki Vercauteren, Sebastian Krumscheid
Abstract. Numerical weather prediction and climate models encounter challenges in accurately representing flow regimes in the stably stratified atmospheric boundary layer and the transitions between them, leading to an inadequate depiction of regime occupation statistics. As a consequence, existing models exhibit significant biases in near-surface temperatures at high latitudes. To explore inherent uncertainties in modeling regime transitions, the response of the near-surface temperature inversion to transient small-scale phenomena is analyzed based on a stochastic modeling approach. A sensitivity analysis is conducted by augmenting a conceptual model for near-surface temperature inversions with randomizations that account for different types of model uncertainty. The stochastic conceptual model serves as a tool to systematically investigate what types of unsteady flow features, and in what contexts, may trigger abrupt transitions in the mean boundary layer state. The findings show that the incorporation of enhanced mixing, a common practice in numerical weather prediction models, blurs the two regime characteristic of the stably stratified atmospheric boundary layer. Simulating intermittent turbulence is shown to provide a potential workaround for this issue. Including key uncertainty in models could lead to a better statistical representation of the regimes in long-term climate simulation. This would help to improve our understanding and the forecasting of climate change especially in high-latitude regions.
{"title":"Sensitivity of the nocturnal and polar boundary layer to transient phenomena","authors":"Amandine Kaiser, Nikki Vercauteren, Sebastian Krumscheid","doi":"10.5194/egusphere-2023-1519","DOIUrl":"https://doi.org/10.5194/egusphere-2023-1519","url":null,"abstract":"<strong>Abstract.</strong> Numerical weather prediction and climate models encounter challenges in accurately representing flow regimes in the stably stratified atmospheric boundary layer and the transitions between them, leading to an inadequate depiction of regime occupation statistics. As a consequence, existing models exhibit significant biases in near-surface temperatures at high latitudes. To explore inherent uncertainties in modeling regime transitions, the response of the near-surface temperature inversion to transient small-scale phenomena is analyzed based on a stochastic modeling approach. A sensitivity analysis is conducted by augmenting a conceptual model for near-surface temperature inversions with randomizations that account for different types of model uncertainty. The stochastic conceptual model serves as a tool to systematically investigate what types of unsteady flow features, and in what contexts, may trigger abrupt transitions in the mean boundary layer state. The findings show that the incorporation of enhanced mixing, a common practice in numerical weather prediction models, blurs the two regime characteristic of the stably stratified atmospheric boundary layer. Simulating intermittent turbulence is shown to provide a potential workaround for this issue. Including key uncertainty in models could lead to a better statistical representation of the regimes in long-term climate simulation. This would help to improve our understanding and the forecasting of climate change especially in high-latitude regions.","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":"167 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abstract. In this paper, we propose a sampling algorithm based on state-of-the-art statistical machine learning techniques to obtain conditional nonlinear optimal perturbations (CNOPs), which is different from traditional (deterministic) optimization methods.1 Specifically, the traditional approach is unavailable in practice, which requires numerically computing the gradient (first-order information) such that the computation cost is expensive, since it needs a large number of times to run numerical models. However, the sampling approach directly reduces the gradient to the objective function value (zeroth-order information), which also avoids using the adjoint technique that is unusable for many atmosphere and ocean models and requires large amounts of storage. We show an intuitive analysis for the sampling algorithm from the law of large numbers and further present a Chernoff-type concentration inequality to rigorously characterize the degree to which the sample average probabilistically approximates the exact gradient. The experiments are implemented to obtain the CNOPs for two numerical models, the Burgers equation with small viscosity and the Lorenz-96 model. We demonstrate the CNOPs obtained with their spatial patterns, objective values, computation times, and nonlinear error growth. Compared with the performance of the three approaches, all the characters for quantifying the CNOPs are nearly consistent, while the computation time using the sampling approach with fewer samples is much shorter. In other words, the new sampling algorithm shortens the computation time to the utmost at the cost of losing little accuracy.
{"title":"An adjoint-free algorithm for conditional nonlinear optimal perturbations (CNOPs) via sampling","authors":"Bin Shi, Guodong Sun","doi":"10.5194/npg-30-263-2023","DOIUrl":"https://doi.org/10.5194/npg-30-263-2023","url":null,"abstract":"Abstract. In this paper, we propose a sampling algorithm based on state-of-the-art statistical machine learning techniques to obtain conditional nonlinear\u0000optimal perturbations (CNOPs), which is different from traditional (deterministic) optimization methods.1 Specifically, the traditional approach is unavailable in practice, which requires numerically computing the gradient (first-order\u0000information) such that the computation cost is expensive, since it needs a large number of times to run numerical models. However, the sampling\u0000approach directly reduces the gradient to the objective function value (zeroth-order information), which also avoids using the adjoint technique\u0000that is unusable for many atmosphere and ocean models and requires large amounts of storage. We show an intuitive analysis for the sampling\u0000algorithm from the law of large numbers and further present a Chernoff-type concentration inequality to rigorously characterize the degree to which\u0000the sample average probabilistically approximates the exact gradient. The experiments are implemented to obtain the CNOPs for two numerical models,\u0000the Burgers equation with small viscosity and the Lorenz-96 model. We demonstrate the CNOPs obtained with their spatial patterns, objective values,\u0000computation times, and nonlinear error growth. Compared with the performance of the three approaches, all the characters for quantifying the CNOPs\u0000are nearly consistent, while the computation time using the sampling approach with fewer samples is much shorter. In other words, the new\u0000sampling algorithm shortens the computation time to the utmost at the cost of losing little accuracy.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42296321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abstract. Non-equilibrium is dominant in geophysical and climate phenomena. However the study of non-equilibrium is much more difficult than equilibrium, and the relevance of probabilistic simplified models has been emphasized. Large deviation rates have been used recently in climate science. In this paper, after recalling progress during the last decades in understanding the role of large deviations in a class of non-equilibrium systems, we point out differences between equilibrium and non-equilibrium. For example, in non-equilibrium (a) large deviation rates may be extensive but not simply additive. (b) In non-equilibrium there are generically long-range space correlations, so large deviation rates are non-local. (c) Singularities in large deviation rates denote the existence of phase transitions often not possible in equilibrium. To exemplify, we shall refer to lattice gas models like the symmetric simple exclusion process and other models which are playing an important role in the understanding of non-equilibrium physics. The reasons why all this may be of interest in climate physics will be briefly indicated.
{"title":"Review article: Large fluctuations in non-equilibrium physics","authors":"G. Jona-Lasinio","doi":"10.5194/npg-30-253-2023","DOIUrl":"https://doi.org/10.5194/npg-30-253-2023","url":null,"abstract":"Abstract. Non-equilibrium is dominant in geophysical and climate\u0000phenomena. However the study of non-equilibrium is much more\u0000difficult than equilibrium, and the relevance of probabilistic\u0000simplified models has been emphasized. Large deviation rates have\u0000been used recently in climate science. In this paper, after recalling\u0000progress during the last decades in understanding the role of large\u0000deviations in a class of non-equilibrium systems, we point out\u0000differences between equilibrium and non-equilibrium. For example, in\u0000non-equilibrium (a) large deviation rates may be extensive but not\u0000simply additive. (b) In non-equilibrium there are generically long-range space correlations, so large deviation rates are non-local. (c) Singularities in large deviation rates denote the existence of phase\u0000transitions often not possible in equilibrium. To exemplify, we shall\u0000refer to lattice gas models like the symmetric simple exclusion\u0000process and other models which are playing an important role in the\u0000understanding of non-equilibrium physics. The reasons why all this may\u0000be of interest in climate physics will be briefly indicated.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42991285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Val'erian Jacques-Dumas, R. V. van Westen, F. Bouchet, H. Dijkstra
Abstract. In recent years, several climate subsystems have been identified that may undergo a relatively rapid transition compared to the changes in their forcing. Such transitions are rare events in general, and simulating long-enough trajectories in order to gather sufficient data to determine transition statistics would be too expensive. Conversely, rare events algorithms like TAMS (trajectory-adaptive multilevel sampling) encourage the transition while keeping track of the model statistics. However, this algorithm relies on a score function whose choice is crucial to ensure its efficiency. The optimal score function, called the committor function, is in practice very difficult to compute. In this paper, we compare different data-based methods (analog Markov chains, neural networks, reservoir computing, dynamical Galerkin approximation) to estimate the committor from trajectory data. We apply these methods on two models of the Atlantic Ocean circulation featuring very different dynamical behavior. We compare these methods in terms of two measures, evaluating how close the estimate is from the true committor and in terms of the computational time. We find that all methods are able to extract information from the data in order to provide a good estimate of the committor. Analog Markov Chains provide a very reliable estimate of the true committor in simple models but prove not so robust when applied to systems with a more complex phase space. Neural network methods clearly stand out by their relatively low testing time, and their training time scales more favorably with the complexity of the model than the other methods. In particular, feedforward neural networks consistently achieve the best performance when trained with enough data, making this method promising for committor estimation in sophisticated climate models.
{"title":"Data-driven methods to estimate the committor function in conceptual ocean models","authors":"Val'erian Jacques-Dumas, R. V. van Westen, F. Bouchet, H. Dijkstra","doi":"10.5194/npg-30-195-2023","DOIUrl":"https://doi.org/10.5194/npg-30-195-2023","url":null,"abstract":"Abstract. In recent years, several climate subsystems have been identified that may undergo a relatively rapid transition compared to the changes in their forcing. Such transitions are rare events in general, and simulating long-enough trajectories in order to gather sufficient data to determine transition statistics would be too expensive. Conversely, rare events algorithms like TAMS (trajectory-adaptive multilevel sampling) encourage the transition while keeping track of the model statistics. However, this algorithm relies on a score function whose choice is crucial to ensure its efficiency. The optimal score function, called the committor function, is in practice very difficult to compute. In this paper, we compare different data-based methods (analog Markov chains, neural networks, reservoir computing, dynamical Galerkin approximation) to estimate the committor from trajectory data. We apply these methods on two models of the Atlantic Ocean circulation featuring very different dynamical behavior. We compare these methods in terms of two measures, evaluating how close the estimate is from the true committor and in terms of the computational time. We find that all methods are able to extract information from the data in order to provide a good estimate of the committor. Analog Markov Chains provide a very reliable estimate of the true committor in simple models but prove not so robust when applied to systems with a more complex phase space. Neural network methods clearly stand out by their relatively low testing time, and their training time scales more favorably with the complexity of the model than the other methods. In particular, feedforward neural networks consistently achieve the best performance when trained with enough data, making this method promising for committor estimation in sophisticated climate models.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45664645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E. Kalnay, T. Sluka, Takuma Yoshida, Cheng Da, Safa Mote
Abstract. We assessed different coupled data assimilation strategies with a hierarchy of coupled models, ranging from a simple coupled Lorenz model to the state-of-the-art coupled general circulation model CFSv2 (Climate Forecast System version 2). With the coupled Lorenz model, we assessed the analysis accuracy by strongly coupled ensemble Kalman filter (EnKF) and 4D-Variational (4D-Var) methods with varying assimilation window lengths. The analysis accuracy of the strongly coupled EnKF with a short assimilation window is comparable to that of 4D-Var with a long assimilation window. For 4D-Var, the strongly coupled approach with the coupled model produces more accurate ocean analysis than the Estimating the Circulation and Climate of the Ocean (ECCO)-like approach using the uncoupled ocean model. Experiments with the coupled quasi-geostrophic model conclude that the strongly coupled approach outperforms the weakly coupled and uncoupled approaches for both the full-rank EnKF and 4D-Var, with the strongly coupled EnKF and 4D-Var showing a similar level of accuracy higher than other coupled data assimilation approaches such as outer-loop coupling. A strongly coupled EnKF software framework is developed and applied to the intermediate-complexity coupled model SPEEDY-NEMO and the state-of-the-art operational coupled model CFSv2. Experiments assimilating synthetic or real atmospheric observations into the ocean through strongly coupled EnKF show that the strongly coupled approach improves the analysis of the atmosphere and upper ocean but degrades observation fits in the deep ocean, probably due to the unreliable error correlation estimated by a small ensemble. The correlation-cutoff method is developed to reduce the unreliable error correlations between physically irrelevant model states and observations. Experiments with the coupled Lorenz model demonstrate that strongly coupled EnKF informed by the correlation-cutoff method produces more accurate coupled analyses than the weakly coupled and plain strongly coupled EnKF regardless of the ensemble size. To extend the correlation-cutoff method to operational coupled models, a neural network approach is proposed to systematically acquire the observation localization functions for all pairs between the model state and observation types. The following strongly coupled EnKF experiments with an intermediate-complexity coupled model show promising results with this method.
{"title":"Review article: Towards strongly coupled ensemble data assimilation with additional improvements from machine learning","authors":"E. Kalnay, T. Sluka, Takuma Yoshida, Cheng Da, Safa Mote","doi":"10.5194/npg-30-217-2023","DOIUrl":"https://doi.org/10.5194/npg-30-217-2023","url":null,"abstract":"Abstract. We assessed different coupled data assimilation\u0000strategies with a hierarchy of coupled models, ranging from a simple coupled\u0000Lorenz model to the state-of-the-art coupled general circulation model\u0000CFSv2 (Climate Forecast System version 2). With the coupled Lorenz model, we assessed the analysis accuracy by\u0000strongly coupled ensemble Kalman filter (EnKF) and 4D-Variational (4D-Var)\u0000methods with varying assimilation window lengths. The analysis accuracy of\u0000the strongly coupled EnKF with a short assimilation window is comparable to\u0000that of 4D-Var with a long assimilation window. For 4D-Var, the\u0000strongly coupled approach with the coupled model produces more accurate\u0000ocean analysis than the Estimating the Circulation and Climate of the\u0000Ocean (ECCO)-like approach using the uncoupled ocean model.\u0000Experiments with the coupled quasi-geostrophic model conclude that the\u0000strongly coupled approach outperforms the weakly coupled and uncoupled\u0000approaches for both the full-rank EnKF and 4D-Var, with the strongly coupled\u0000EnKF and 4D-Var showing a similar level of accuracy higher than other\u0000coupled data assimilation approaches such as outer-loop coupling. A\u0000strongly coupled EnKF software framework is developed and applied to the\u0000intermediate-complexity coupled model SPEEDY-NEMO and the state-of-the-art\u0000operational coupled model CFSv2. Experiments assimilating synthetic or real\u0000atmospheric observations into the ocean through strongly coupled EnKF show\u0000that the strongly coupled approach improves the analysis of the atmosphere\u0000and upper ocean but degrades observation fits in the deep ocean, probably\u0000due to the unreliable error correlation estimated by a small ensemble. The\u0000correlation-cutoff method is developed to reduce the unreliable error\u0000correlations between physically irrelevant model states and observations.\u0000Experiments with the coupled Lorenz model demonstrate that strongly coupled\u0000EnKF informed by the correlation-cutoff method produces more accurate\u0000coupled analyses than the weakly coupled and plain strongly coupled EnKF\u0000regardless of the ensemble size. To extend the correlation-cutoff method to\u0000operational coupled models, a neural network approach is proposed to\u0000systematically acquire the observation localization functions for all pairs\u0000between the model state and observation types. The following\u0000strongly coupled EnKF experiments with an intermediate-complexity coupled\u0000model show promising results with this method.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41707925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abstract. Controlling weather is an outstanding and pioneering challenge for researchers around the world, due to the chaotic features of the complex atmosphere. A control simulation experiment (CSE) on the Lorenz-63 model, which consists of positive and negative regimes represented by the states of variable x, demonstrated that the variables can be controlled to stay in the target regime by adding perturbations with a constant magnitude to an independent model run (Miyoshi and Sun, 2022). The current study tries to reduce the input manipulation of the CSE, including the total control times and magnitudes of perturbations, by investigating how controls affect the instability of systems. For that purpose, we first explored the instability properties of Lorenz-63 models without and under control. Experiments show that the maximum growth rate of the singular vector (SV) reduces when the variable x was controlled in the target regime. Subsequently, this research proposes to update the magnitude of perturbations adaptively based on the maximum growth rate of SV; consequently, the times to control will also change. The proposed method successfully reduces around 40 % of total control times and around 20 % of total magnitudes of perturbations compared to the case with a constant magnitude. Results of this research suggest that investigating the impacts of control on instability would be beneficial for designing methods to control the complex atmosphere with feasible manipulations.
{"title":"Reducing manipulations in a control simulation experiment based on instability vectors with the Lorenz-63 model","authors":"Ouyang Mao, Keita Tokuda, S. Kotsuki","doi":"10.5194/npg-30-183-2023","DOIUrl":"https://doi.org/10.5194/npg-30-183-2023","url":null,"abstract":"Abstract. Controlling weather is an outstanding and pioneering challenge for researchers around the world, due to the chaotic features of the complex\u0000atmosphere. A control simulation experiment (CSE) on the Lorenz-63 model, which consists of positive and negative regimes represented by the states\u0000of variable x, demonstrated that the variables can be controlled to stay in the target regime by adding perturbations with a constant magnitude to\u0000an independent model run (Miyoshi and Sun, 2022). The current study tries to reduce the input manipulation of the CSE, including the total control\u0000times and magnitudes of perturbations, by investigating how controls affect the instability of systems. For that purpose, we first explored the\u0000instability properties of Lorenz-63 models without and under control. Experiments show that the maximum growth rate of the singular vector (SV) reduces\u0000when the variable x was controlled in the target regime. Subsequently, this research proposes to update the magnitude of perturbations\u0000adaptively based on the maximum growth rate of SV; consequently, the times to control will also change. The proposed method successfully reduces\u0000around 40 % of total control times and around 20 % of total magnitudes of perturbations compared to the case with a constant magnitude.\u0000Results of this research suggest that investigating the impacts of control on instability would be beneficial for designing methods to control the\u0000complex atmosphere with feasible manipulations.\u0000","PeriodicalId":54714,"journal":{"name":"Nonlinear Processes in Geophysics","volume":" ","pages":""},"PeriodicalIF":2.2,"publicationDate":"2023-06-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45686815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：