A feature-based information-theoretic approach for detecting interpretable, long-timescale pairwise interactions from time series (2404.05929v1)

Abstract: Quantifying relationships between components of a complex system is critical to understanding the rich network of interactions that characterize the behavior of the system. Traditional methods for detecting pairwise dependence of time series, such as Pearson correlation, Granger causality, and mutual information, are computed directly in the space of measured time-series values. But for systems in which interactions are mediated by statistical properties of the time series (`time-series features') over longer timescales, this approach can fail to capture the underlying dependence from limited and noisy time-series data, and can be challenging to interpret. Addressing these issues, here we introduce an information-theoretic method for detecting dependence between time series mediated by time-series features that provides interpretable insights into the nature of the interactions. Our method extracts a candidate set of time-series features from sliding windows of the source time series and assesses their role in mediating a relationship to values of the target process. Across simulations of three different generative processes, we demonstrate that our feature-based approach can outperform a traditional inference approach based on raw time-series values, especially in challenging scenarios characterized by short time-series lengths, high noise levels, and long interaction timescales. Our work introduces a new tool for inferring and interpreting feature-mediated interactions from time-series data, contributing to the broader landscape of quantitative analysis in complex systems research, with potential applications in various domains including but not limited to neuroscience, finance, climate science, and engineering.

• The paper introduces a novel feature-based mutual information method to capture long-timescale dependencies between time series.
• It demonstrates that extracting statistical features from sliding windows enhances detection accuracy compared to traditional methods like Pearson correlation and Granger causality.
• The approach improves interpretability and resilience to noise, offering significant benefits for analyzing complex systems in fields such as neuroscience, finance, and climate science.

Feature-Based Information-Theoretic Approach for Inferring Long-Timescale Dependence

The paper introduces a novel information-theoretic methodology to detect long-timescale, interpretable pairwise interactions between time series, mediated by statistical properties referred to as time-series features. This feature-based mutual information approach addresses the limitations of traditional methods, like Pearson correlation and Granger causality, which operate on direct time-series values and often fail to capture relationships stemming from long-timescale interactions.

In the proposed method, the authors suggest a shift from conventional signal-space mutual information (MI) to a feature-space perspective, MI$_F$. This involves assessing the dependence between a target time series and statistical properties—features—extracted from sliding windows of a source time series. The dependency, quantified using MI$_F$, potentially outperforms traditional approaches in scenarios characterized by short time series, high noise, and extended interaction timescales.

The paper validates this methodology using three simulated processes: (1) a stationary random noise process, (2) a non-stationary third-order autoregressive (AR(3)) process, and (3) a bimodal spiking process. Each process simulates a target time series responsive to specific statistical properties of the source. In all scenarios, MI$_F$ is juxtaposed with MI$_s$, revealing superior detection of dependencies particularly when the capturing feature set includes the genuine driving feature.

Key findings indicate that MI$_F$ is notably robust to noise, achieving nearly perfect capture rates for lower noise levels and still outperforming MI$_s$ at higher noise strengths. When applied to non-stationary processes, MI$_F$ excels in detecting dependencies even when the capturing timescale does not precisely match the true interaction timescale. This flexibility is facilitated by the ability to use an ensemble of potential features in MI$_F$, increasing the likelihood of capturing relevant dependencies despite individual feature noise or timescale inaccuracies.

Intriguingly, MI$_F$ also provides insights into the nature of interactions through interpretability of features, enabling comprehension of underlying dynamics when selecting appropriate feature sets. The ensemble approach of multiple features introduces redundancy and synergy, enhancing resilience against noise, which can be particularly beneficial in severely noisy data environments.

Practically, the proposed approach finds potential in fields requiring analysis of complex systems with long memory, such as neuroscience, finance, and climate science. The ability to efficiently infer dependencies from high-dimensional data and interpret the interactions provides a notable advantage in these domains.

The theoretical implications are significant, suggesting directions for extending the framework to multi-feature and multi-target scenarios, incorporating more complex transfer entropy metrics, and exploring feature selection strategies for optimizing detection and interpretability. Further, the connection between feature complexity and capture efficacy highlights an area ripe for exploration, aiming to establish guiding principles for feature selection based on the inherent properties of the data and the expected interaction characteristics.

In summary, the feature-based information-theoretic approach advances the capability to infer and interpret long-timescale dependencies, with promising implications for both theoretical developments and practical applications in the analysis of complex systems.