Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control (2111.10992v1)

Abstract: Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from subsets of limited and noisy data. The aggregate model statistics are then used to produce inclusion probabilities of the candidate functions, which enables uncertainty quantification and probabilistic forecasts. We apply this ensemble-SINDy (E-SINDy) algorithm to several synthetic and real-world data sets and demonstrate substantial improvements to the accuracy and robustness of model discovery from extremely noisy and limited data. For example, E-SINDy uncovers partial differential equations models from data with more than twice as much measurement noise as has been previously reported. Similarly, E-SINDy learns the Lotka Volterra dynamics from remarkably limited data of yearly lynx and hare pelts collected from 1900-1920. E-SINDy is computationally efficient, with similar scaling as standard SINDy. Finally, we show that ensemble statistics from E-SINDy can be exploited for active learning and improved model predictive control.

• The paper presents Ensemble-SINDy, which significantly improves sparse model discovery even in low-data, high-noise conditions.
• It leverages bootstrapping and statistical aggregation to reduce overfitting and enhance noise tolerance in identifying differential equations.
• By integrating active learning and model predictive control, Ensemble-SINDy demonstrates improved efficiency and reliability for dynamic system modeling.

Ensemble-SINDy: Advancements in Sparse Model Discovery

The paper "Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control" focuses on enhancing the capabilities of the Sparse Identification of Nonlinear Dynamics (SINDy) framework. This advancement leverages ensemble methods to improve model discovery in conditions that typically challenge data-driven techniques, specifically the low-data and high-noise scenarios.

Context and Objectives

SINDy has established itself as a powerful tool for identifying sparse, interpretable models from data, especially when deriving the system's equations from first principles is difficult. However, traditional SINDy algorithms may underperform in scenarios with sparse data or significant noise challenges. Addressing these issues, the authors propose Ensemble-SINDy (E-SINDy), which incorporates bootstrap aggregating (bagging) and statistical inference techniques to enhance robustness and accuracy.

Methodology

E-SINDy extends classical SINDy by incorporating ensemble methods, traditionally used in machine learning to improve predictive performance and robustness. The approach involves:

1. Data Bootstrapping: Instead of using the entire dataset, multiple subsets are sampled with replacement, and each subset is used to generate a SINDy model. This technique reduces overfitting and enhances noise tolerance.
2. Statistical Aggregation: The ensemble of models derived from various data subsets is statistically aggregated to provide inclusion probabilities for candidate functions. This aggregation not only quantifies uncertainty but also aids in probabilistic forecasting.
3. Active Learning and Control: E-SINDy exploits ensemble statistics for active learning, which guides data sampling towards regions of high uncertainty to maximize information gain. Additionally, the paper explores the use of E-SINDy for model predictive control (MPC) in dynamic systems, such as the forced Lorenz system.

Results and Contributions

The ensemble approach significantly enhances SINDy's abilities to discover models accurately, even in the presence of higher noise levels and limited data:

• E-SINDy demonstrated strong performance improvements over traditional SINDy methods in identifying ordinary and partial differential equations (PDEs) from synthetic and real-world datasets.
• For PDEs, E-SINDy reduced model coefficient errors and increased success rates in accurate model structure identification. Noise robustness improved by more than two-fold compared to traditional SINDy.
• The utility of E-SINDy in model predictive control was illustrated by comparing its performance to traditional SINDy under various data constraints.
• The paper emphasizes E-SINDy's computational efficiency and its integration into the PySINDy package, enhancing accessibility for researchers and practitioners.

Implications and Future Directions

The implications of this research are significant for fields relying on dynamical models derived from data, such as fluid dynamics, neuroscience, and ecological modeling. The probabilistic nature of E-SINDy's model outputs could further facilitate the exploration of real-world systems under uncertainty. Moreover, the integration of active learning strategies into model discovery processes reflects a promising direction toward more autonomous and intelligent data collection methodologies.

Going forward, the authors suggest exploring E-SINDy's capabilities in broader contexts, such as task-agnostic modeling crucial for reinforcement learning in complex environments. The potential for combining E-SINDy with other probabilistic and deep learning frameworks also presents an intriguing avenue for further research and application.

In conclusion, E-SINDy offers a considerable advancement in sparse model discovery, particularly enhancing the practicality and reliability of data-driven approaches in challenging environments characterized by noise and data scarcity.