Building Ensemble-Based Data Assimilation Systems with Coupled Models
0
Citation
0
Reference
20
Related Paper
Abstract:
Discussed is the construction of programs for efficient ensemble data assimilation systems based on a direct connection between a coupled simulation model and ensemble data assimilation software. The strategy allows us to set up a data assimilation program with high flexibility and parallel scalability with only small changes to the model. The direct connection is obtained by first extending the source code of the coupled model so that it is able to run an ensemble of model states. In addition, a filtering step is added using a combination of in-memory access and parallel communication to create an online-coupled ensemble assimilation program. The direct connection avoids the common need to stop and restart a whole coupled model system to perform the assimilation of observations in the analysis step of ensemble-based filter methods like ensemble Kalman or particle filters. Instead, the analysis step is performed in between time steps and is independent of the actual model coupler. This strategy allows us to perform both in-compartment (for weakly coupled assimilation) and cross-compartment (for strongly coupled assimilation) assimilation. The assimilation frequency can be kept flexible, so that assimilation of observations from different compartments can be performed at different time intervals. Using the parallel data assimilation framework (PDAF, http://pdaf.awi.de), the direct connection strategy will be exemplified for the ocean-atmosphere model ECHAM6-FESOM.Keywords:
Assimilation (phonology)
Ensemble Learning
Ensemble forecasting
Cite
In our work we implement the ensemble Kalman filter for assimilation of electron phase space density (PSD) data into the Versatile Electron Radiation Belt (VERB) model. In particular, the assimilation is performed locally along the direction of the dominant diffusion of electrons in the model, the local assimilation will enable the correct assimilation of data to be consistent with the flow of electrons throughout the model. A set of identical-twin assimilation experiments are presented, where the results show a significant improvement in phase space density estimation for VERB using ensemble Kalman filter.
Assimilation (phonology)
Cite
Citations (0)
Executable
Ensemble forecasting
Cite
Citations (246)
Ensemble filter algorithms can be implemented in a generic way
such that they can be applied with various models with only a minimum amount of recoding.
This is possible due to the fact that ensemble filters can operate
on abstract state vectors and require only limited information about
the numerical model and the observational data used for a data assimilation application.
To build an assimilation system, the analysis step of a filter
algorithm needs to be connected to the numerical model. Furthermore,
ensemble integrations have to be enabled. The Parallel Data Assimilation Framework PDAF has been developed to
provide these features: It is a generic framework that allows to
extend a numerical model with a filter to build an ensemble data
assimilation system with minimal changes to the model code. PDAF also provides a
selection of common ensemble
Kalman filter algorithms. As the computational cost of ensemble data
assimilation is a multiple of that of a pure forward model, the
framework and the filter algorithms are parallelized and support
parallelized models. Thus, data assimilation with high-dimensional
numerical models is feasible. PDAF is coded in Fortran and
available as free software (http://pdaf.awi.de). We discuss the
features of PDAF and the parallel computing performance of data
assimilation systems based on PDAF on the example of data assimilation
with the finite element ocean model FEOM.
Ensemble Learning
Ensemble forecasting
Fortran
Cite
Citations (1)
We discuss how to build an ensemble data assimilation system using a direct connection between a coupled Earth system model (ESM) and the ensemble data assimilation software PDAF (Parallel Data Assimilation Framework, http://pdaf.awi.de). The direct connection results in a data assimilation program with high flexibility, efficiency, and parallel scalability. For this we augment the source code of the coupled model by data assimilation routines and hence create an online-coupled assimilative model. This first modifies the coupled model to be able to simulate an ensemble. Using a combination of in-memory access and parallel communication with the Message Passing Interface (MPI) standard we can further add the analysis step of ensemble-based assimilation methods. Thus the assimilation of observations is computed without the need to stop and restart the whole coupled model system. Instead, the analysis step is performed in between time steps and is independent of the actual model coupler that couples the different model compartments. This strategy to build the assimilation system allows us to perform both weakly coupled (in-compartment) and strongly coupled (cross-compartment) assimilation. The assimilation frequency can be kept flexible, so that the assimilation of observations from different compartments of the ESM can be performed at different intervals. Further, the reading and writing of disk files is minimized. The resulting assimilative model can be run in the same way as the regular ESM, but with additional parameters controlling the assimilation and with a higher number of processors to simulate the ensemble. Using the example of the coupled climate model AWI-CM that contains the FESOM model for the ocean and sea ice and ECHAM6 for the atmosphere, both coupled through the OASIS-MCT coupler, we discuss the features of the online assimilation coupling strategy and the performance of the resulting assimilative model.
Assimilation (phonology)
Ensemble forecasting
Ensemble Learning
Cite
Citations (0)
Abstract. Data assimilation integrates information from observational measurements with numerical models. When used with coupled models of Earth system compartments, e.g. the atmosphere and the ocean, consistent joint states can be estimated. A common approach for data assimilation are ensemble-based methods which use an ensemble of state realizations to estimate the state and its uncertainty. These methods are far more costly to compute than a single coupled model because of the required integration of the ensemble. However, with uncoupled models, the methods also have been shown to exhibit a particularly good scaling behavior. This study discusses an approach to augment a coupled model with data assimilation functionality provided by the Parallel Data Assimilation Framework (PDAF). Using only minimal changes in the codes of the different compartment models, a particularly efficient data assimilation system is generated that utilizes parallelization and in-memory data transfers between the models and the data assimilation functions and hence avoids most of the filter reading and writing and also model restarts during the data assimilation process. The study explains the required modifications of the programs on the example of the coupled atmosphere-sea ice-ocean model AWI-CM. Using the case of the assimilation of oceanic observations shows that the data assimilation leads only small overheads in computing time of about 15 % compared to the model without data assimilation and a very good parallel scalability. The model-agnostic structure of the assimilation software ensures a separation of concerns in that the development of data assimilation methods and be separated from the model application.
Assimilation (phonology)
Cite
Citations (4)
Data assimilation applications with large-scale numerical models
exhibit extreme requirements on computational resources. Good
scalability of the assimilation system is necessary to make these
applications feasible. Sequential data assimilation methods based on
ensemble forecasts, like ensemble-based Kalman filters, provide such
good scalability, because the forecast of each ensemble member can be
performed independently. However, this parallelism has to be combined
with the parallelization of both the numerical model and the data
assimilation algorithm. In order to simplify the implementation of
scalable data assimilation systems based on existing numerical models,
the Parallel Data Assimilation Framework PDAF (http://pdaf.awi.de) has
been developed. PDAF provides support for implementing a data
assimilation system with parallel ensemble forecasts and parallel
numerical models. Further, it includes several optimized parallel
filter algorithms, like the Ensemble Transform Kalman Filter.
We will discuss the philosophy behind PDAF as well as features and
scalability of data assimilation systems based on PDAF on the example
of data assimilation with the finite element ocean model FEOM.
Ensemble forecasting
Ensemble Learning
Assimilation (phonology)
Cite
Citations (0)
We discuss how to build an ensemble data assimilation system using a direct connection between a coupled model system and the ensemble data assimilation software PDAF (Parallel Data Assimilation Framework, http://pdaf.awi.de). The direct connection results in a data assimilation program with high flexibility, efficiency, and parallel scalability. For this we augment the source code of the coupled model by data assimilation routines and hence create an online-coupled assimilative model. This first modifies the coupled model to be able to simulate an ensemble. Using a combination of in-memory access and parallel communication with the Message Passing Interface (MPI) standard we can further add the analysis step of ensemble-based filter methods, which compute the assimilation of observations, without the need to stop and restart the whole coupled model system. Instead, the analysis step is performed in between time steps and is independent of the actual model coupler that couples the different model compartments. This strategy to build the assimilation system allows us to perform both weakly coupled (in-compartment) and strongly coupled (cross-compartment) assimilation. The assimilation frequency can be kept flexible, so that the assimilation of observations from different compartments can be performed at different intervals. Further, the reading and writing of disk files is minimized. The resulting assimilative model can be run in the same way as the regular coupled model, but with additional parameters controlling the assimilation and with a higher number of processors to simulate the ensemble. Using the example of the coupled climate model AWI-CM that contains the FESOM model for the ocean and sea ice and ECHAM6 for the atmosphere, both coupled through the OASIS-MCT coupler, we discuss the features of the online assimilation coupling strategy and the performance of the resulting assimilative model.
Assimilation (phonology)
Ensemble forecasting
Ensemble Learning
Message Passing Interface
Cite
Citations (0)
Abstract Ensemble‐based data assimilation is rapidly proving itself as a computationally efficient and skilful assimilation method for numerical weather prediction, which can provide a viable alternative to more established variational assimilation techniques. However, a fundamental shortcoming of ensemble techniques is that the resulting analysis increments can only span a limited subspace of the state space, whose dimension is less than the ensemble size. This limits the amount of observational information that can effectively constrain the analysis. In this paper, a data selection strategy that aims to assimilate only the observational components that matter most and that can be used with both stochastic and deterministic ensemble filters is presented. This avoids unnecessary computations, reduces round‐off errors and minimizes the risk of importing observation bias in the analysis. When an ensemble‐based assimilation technique is used to assimilate high‐density observations, the data selection procedure allows the use of larger localization domains that may lead to a more balanced analysis. Results from the use of this data selection technique with a two‐dimensional linear and a nonlinear advection model using both in situ and remote sounding observations are discussed.
Ensemble Learning
Cite
Citations (13)
The Parallel Data Assimilation Framework (PDAF) is a unified framework for ensemble data assimilation. PDAF has been developed to simplify the implementation of scalable ensemble data assimilation systems with existing high-dimensional numerical models. It provides support for the parallelization of the ensemble integration and fully implemented and parallelized ensemble Kalman and nonlinear filters. PDAF encapsulates the filter algorithms so that model and data assimilation developments can be conducted separately. I will review the structure and features of PDAF and discuss its use in different applications of ocean-biogeochemical and coupled atmosphere-ocean models.
Ensemble Learning
Assimilation (phonology)
Cite
Citations (0)
A direct connection between a coupled model system and ensemble data assimilation software allows to set up a data assimilation program with high flexibility, efficiency, and parallel scalability. The direct connection can be obtained by extending the source code of the coupled model to create an online-coupled assimilation program. Using a combination of in-memory access and parallel communication with the Message Passing Interface (MPI) standard, the direct connection avoids the need to stop and restart a whole coupled model system to perform the assimilation of observations in the analysis step of ensemble-based filter methods. Instead, the analysis step is performed in between time steps and is independent of the actual model coupler. This strategy allows us to perform both in-compartment (for weakly coupled assimilation) and cross-compartment (for strongly coupled assimilation) assimilation. The assimilation frequency can be kept flexible, so that assimilation of observations from different compartments can be performed at different intervals. Using the parallel data assimilation framework (PDAF, http://pdaf.awi.de), the online connection strategy will be exemplified for coupled models using a single executable and such that use multiple executables for different compartments and a model coupler as in the case of the OASIS-MCT coupled climate model ECHAM6-FESOM.
Executable
Assimilation (phonology)
Message Passing Interface
Cite
Citations (0)