Leveraging data-driven weather models for improving numerical weather prediction skill through large-scale spectral nudging (2407.06100v3)

Published 8 Jul 2024 in physics.ao-ph and cs.LG

Abstract: Operational meteorological forecasting has long relied on physics-based numerical weather prediction (NWP) models. Recently, this landscape has faced disruption by the advent of data-driven AI-based weather models, which offer tremendous computational performance and competitive forecasting accuracy. However, data-driven models for medium-range forecasting generally suffer from major limitations, including low effective resolution and a narrow range of predicted variables. This study illustrates the relative strengths and weaknesses of these competing paradigms using the physics-based GEM (Global Environmental Multiscale) and the AI-based GraphCast models. Analyses of their respective global predictions in physical and spectral space reveal that GraphCast-predicted large scales outperform GEM, particularly for longer lead times, even though fine scales predicted by GraphCast suffer from excessive smoothing. Building on this insight, a hybrid NWP-AI system is proposed, wherein temperature and horizontal wind components predicted by GEM are spectrally nudged toward GraphCast predictions at large scales, while GEM itself freely generates the fine-scale details critical for local predictability and weather extremes. This hybrid approach is capable of leveraging the strengths of GraphCast to enhance the prediction skill of the GEM model while generating a full suite of physically consistent forecast fields with a full power spectrum. Additionally, trajectories of tropical cyclones are predicted with enhanced accuracy without significant changes in intensity. Work is in progress for operationalization of this hybrid system at the Canadian Meteorological Centre.

Citations (3)

View on Semantic Scholar

Summary

The paper demonstrates a hybrid system that integrates data-driven models with traditional NWP through large-scale spectral nudging.
The methodology uses discrete cosine transform for spectral filtering to nudge large-scale atmospheric features, enhancing forecast detail and accuracy.
Results indicate reduced RMSE, improved transient-eddy anomaly prediction, and better tropical cyclone trajectory forecasts compared to standalone models.

Leveraging Data-Driven Weather Models for Improving Numerical Weather Prediction Skill Through Large-Scale Spectral Nudging

Introduction

The paper presents a research paper focusing on enhancing numerical weather prediction (NWP) by integrating data-driven models with traditional physics-based approaches, specifically by incorporating large-scale spectral nudging. The paper utilizes the Global Environmental Multiscale (GEM) model and the GraphCast model from Google-DeepMind to assess the merits of a hybrid approach, combining the strengths of both methodologies.

Background

Numerical Weather Prediction and AI Models

Traditional NWP models utilize a dynamic core to solve nonlinear equations related to atmospheric processes. However, these physics-based models are increasingly being supplemented or even challenged by data-driven models, which promise higher computational efficiency. Models like GraphCast can surpass traditional models in accuracy over extended lead times but often suffer from limitations like fine-scale smoothing and lower effective resolution due to reliance on training data such as the ERA5 reanalysis.

Global Environmental Multiscale (GEM) Model

GEM, used for operational forecasting by Environment and Climate Change Canada, applies a physics-based methodology to weather forecasting. It solves for multiple atmospheric variables and incorporates comprehensive parameterizations for various physical processes. The paper employs GEM as a representative NWP model to be enhanced through AI-driven forecasting.

GraphCast

The GraphCast model uses graph neural networks trained on ERA5 reanalyses to emulate atmospheric conditions. Despite their computational efficiency and accuracy at large scales, AI models like GraphCast face challenges regarding their effective resolution and versatility in variable prediction.

Methodology

Spectral Nudging Implementation

The paper implements spectral nudging by adjusting large-scale atmospheric features predicted by the GEM model toward those predicted by GraphCast. This technique allows the incorporation of the accuracy of GraphCast predictions into GEM, particularly valuable for improving forecasts of large-scale atmospheric patterns.

Spectral Decomposition Approach:

Utilizes discrete cosine transform (DCT) for spectral filtering.
Targets specific scales for nudging while retaining GEM's fine-scale numerical features.
Retained scales are weighted averages between GEM and GraphCast predictions.

Optimal Configuration:

Variables for nudging include horizontal wind components and virtual temperature.
A vertical nudging profile omitting boundary layer and stratosphere, aligning with areas where GraphCast shows weaker performance.
Length scales and relaxation times are optimized to leverage GraphCast’s strengths while preserving GEM's detailed forecasting.

Results and Analysis

Verification with Power Spectra

The power spectra analysis demonstrates that the hybrid system, referred to as GDPS-SN, matches the traditional NWP approach in preserving fine-scale details and avoids excessive smoothing (Figure 1). Spectral coherence with analysis datasets improves, indicating enhanced prediction of transient-eddy anomalies, especially beyond the 5-day forecast horizon (Figure 2).

Figure 1: Global kinetic energy power and variance spectra demonstrating the resolution capabilities of different systems.

Figure 2: Spectral amplitude ratio and coherence underscore the improved alignment achieved through spectral nudging.

Radiosonde Observations

When verified against radiosonde data, GDPS-SN forecasts are consistently closer to observations for most variables and levels, achieving significant reductions in RMSE (Root Mean Square Error) compared against both standalone GEM and GraphCast predictions. Improvements are evident both in mid-level layers targeted by nudging and extend, albeit less prominently, to other layers.

Region-Specific Impacts: Across global regions and successive layers, GDPS-SN demonstrates improved accuracy, most notably in scenarios with significant synoptic-scale influence.

Figure 3: Radiosonde verification highlighting GDPS-SN improvements over standalone GEM and GraphCast predictions.

ECMWF Analyses and Surface Observations

Further validation against ECMWF and surface observations consolidates the improved performance offered by GDPS-SN. Anomaly correlation coefficients and quantitative precipitation forecasts indicate substantial gains in predictability and reduced false alarms, particularly for winter precipitation forecasts (Figure 4).

Figure 4: Performance analysis against ECMWF and surface data illustrating enhanced forecast reliability.

Tropical Cyclone and Computational Cost

The hybrid system also enhances trajectory predictions for tropical cyclones without degrading intensity forecasts. The computational cost of implementing spectral nudging is currently high, but potential reductions are feasible through optimization.

Conclusion

The paper demonstrates the feasibility and benefits of integrating data-driven models with traditional NWP methods. By implementing large-scale spectral nudging, the hybrid GDPS-SN system effectively capitalizes on the predictive strengths of AI while maintaining the intricate resolution of physics-based models. The results suggest a pathway forward where both AI and NWP methodologies are viewed as complementary, providing a potent combination for future operational implementations. Further work is recommended to optimize the computational cost and explore expanded nudging in the boundary layer with improved AI model configurations.