Online Data Augmentation for Forecasting with Deep Learning (2404.16918v2)

Abstract: Deep learning approaches are increasingly used to tackle forecasting tasks involving datasets with multiple univariate time series. A key factor in the successful application of these methods is a large enough training sample size, which is not always available. Synthetic data generation techniques can be applied in these scenarios to augment the dataset. Data augmentation is typically applied offline before training a model. However, when training with mini-batches, some batches may contain a disproportionate number of synthetic samples that do not align well with the original data characteristics. This work introduces an online data augmentation framework that generates synthetic samples during the training of neural networks. By creating synthetic samples for each batch alongside their original counterparts, we maintain a balanced representation between real and synthetic data throughout the training process. This approach fits naturally with the iterative nature of neural network training and eliminates the need to store large augmented datasets. We validated the proposed framework using 3797 time series from 6 benchmark datasets, three neural architectures, and seven synthetic data generation techniques. The experiments suggest that online data augmentation leads to better forecasting performance compared to offline data augmentation or no augmentation approaches. The framework and experiments are publicly available.

• The paper introduces OnDAT, a dynamic augmentation method that integrates data synthesis into model training to enhance forecast accuracy.
• It employs rolling seasonality decomposition and moving block bootstrapping to dynamically generate realistic time series variations.
• Experimental results on 8 benchmark datasets reveal lower SMAPE scores and improved model robustness compared to static augmentation methods.

Enhancing Forecasting Models with On-the-fly Data Augmentation: Facilitating Deep Learning with OnDAT

Introduction

Recent advances in time series forecasting have highlighted the potential of deep learning models, such as NHITS and N-BEATS, which outperform traditional methods like ARIMA in benchmark tasks. A critical factor in harnessing the full power of these models is the availability of substantial training data. In scenarios with data scarcity, data augmentation can supplement limited data and improve model performance. Conventional augmentation approaches prepare a static augmented dataset prior to training, which may not sufficiently capture the variability of the data generating process. This paper introduces On-the-fly Data Augmentation for Time series (OnDAT), a dynamic augmentation approach that integrates data augmentation directly into the training and validation processes of deep learning models.

Motivation and Background

Deep learning models' success in time series forecasting often hinges on large training datasets, which are not always available. Traditional data augmentation techniques address this limitation by generating synthetic data before model training. However, this static approach might fail to comprehensively explore the data space, potentially resulting in models that are less robust and more prone to overfitting. The proposed method, OnDAT, aims to mitigate these risks by implementing data augmentation dynamically during model training, ensuring a continuous exposure to varied data iterations and fostering a more thorough exploration of the data space.

OnDAT Methodology

OnDAT leverages a unique combination of rolling seasonality decompositions and moving block bootstrapping (MBB) to produce augmented data dynamically during the training process. This method involves:

• Decomposing each time series in a mini-batch into trend, seasonality, and remainder components.
• Applying MBB to the remainder to create a synthetic series, maintaining the temporal dependencies.
• Reconstructing the time series from the augmented remainder and the original trend and seasonal components.
• Using this freshly augmented batch for model training or validation.

This process is repeated for each training iteration, providing the model with rich and varied data which enhances generalization and reduces overfitting.

Experimental Design and Results

The efficacy of OnDAT was assessed across 8 benchmark datasets totaling over 75,000 time series. Key findings include:

• OnDAT significantly outperforms traditional static data augmentation methods, showcasing lower average SMAPE scores across multiple datasets.
• The dynamic augmentation process enhances not only the robustness but the general predictive capabilities of the forecasting models, as evidenced by consistent improvements over baseline methods.
• Both the training and validation phases benefit from OnDAT, producing models that not only perform well on unseen data but also offer reliable performance estimates during validation.
• Computational overheads, while present, are not prohibitive, suggesting that OnDAT's benefits can be realized in practical scenarios without excessive costs.

Implications and Future Directions

The introduction of OnDAT marks a significant step forward in the application of data augmentation in time series forecasting. It successfully addresses the limitations of static augmentation methods and paves the way for more accurate and robust forecasting models. Future research could explore the application of OnDAT with different neural network architectures and in conjunction with other data augmentation techniques to further enhance its effectiveness and applicability.

Moreover, there is potential to integrate adaptive mechanisms that fine-tune the augmentation process based on real-time feedback during training, optimizing the quality of synthetic data generated and potentially reducing computational demands.

Conclusion

OnDAT introduces a flexible, dynamic framework for data augmentation that is directly integrated into the training process of forecasting models, enhancing their performance and reliability. Its ability to continuously adapt and generate new data iterations represents a substantial improvement over static augmentation methods, making it a valuable tool for improving forecasts in scenarios with limited data.