Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking

Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking

This paper investigates the development and evaluation of respiratory acoustic foundation models. The authors recognize the significant potential of leveraging respiratory sounds—such as coughing and breathing—for health monitoring and disease detection. This potential spans a gamut of applications, including respiratory rate estimation, lung function analysis, detection of sleep apnea, assessment of smoking effects, and diagnosis of respiratory diseases like influenza and asthma.

Challenges in Existing Approaches

Traditional methods mainly rely on supervised deep learning models that necessitate extensive volumes of labeled data, which is often labor-intensive and costly to produce. Conventional signal processing techniques, despite their utility, have inherent limitations in their performance and usually demand domain expertise. Currently, open-source acoustic models like AudioMAE and CLAP utilize general audio event datasets, with a mere 0.3% representation of respiratory sounds, limiting their applicability to intricate respiratory sound variations. Moreover, despite the presentation of a non-open-source model pretrained on respiratory sounds recently, the closed nature of this endeavor restricts replication, analysis, and further development.

OPERA: Open Respiratory Acoustic Foundation Model

To counter these limitations, the authors propose OPERA—an open respiratory acoustic foundation model pretraining and benchmarking system. OPERA is a comprehensive system that amalgamates the curation of datasets, the pretraining of acoustic models, and a rigorous benchmarking process.

Data Curation

The authors curate a diverse and large-scale dataset comprising approximately 136K samples, equating to around 440 hours of respiratory audio data from five sources. These datasets cover various respiratory sound modalities, including breathing, coughing, and lung sounds, and are orders of magnitude larger compared to existing datasets used for training open acoustic models.

Model Pretraining

The paper introduces three pretrained models using different self-supervised learning (SSL) techniques:

1. OPERA-CT: A contrastive learning-based transformer model.
2. OPERA-CE: A contrastive learning-based CNN model.
3. OPERA-GT: A generative pretrained transformer model.

These approaches were chosen to leverage large-scale unlabeled data for learning meaningful representations, aiming to enhance the models' transferability and applicability to supervised fine-tuning tasks.

Benchmarking

The paper extensively benchmarks these pretrained models across 19 respiratory health tasks, categorized into health condition inference and lung function estimation. These tasks utilize ten labeled respiratory audio datasets, of which six were unseen during pretraining, ensuring fair and robust evaluation of the models' generalizability.

Key Results and Findings

The findings demonstrate that the pretrained respiratory acoustic foundation models outperform traditional feature extraction methods and existing general audio pretrained models on 16 out of 19 tasks. Specific results include:

• Health Condition Inference: The models surpass the 0.7 AUROC threshold on multiple tasks, indicating high utility in discriminating health conditions.
• Lung Function Estimation: The generative pretrained models exhibit lower MAE, particularly for tasks requiring global feature extraction.

Among the three models, OPERA-CT excels in classification-based tasks, while OPERA-GT performs robustly in regression tasks. The superior performance of transformer-based models compared to CNN models underscores the efficacy of these architectures in handling respiratory sound variations, albeit with higher computational demands.

Implications and Future Directions

The paper introduces a paradigm shift in employing foundation models for respiratory health applications, showcasing their potential to streamline diagnostic processes and personalized health monitoring. The following future directions are highlighted:

1. Fine-Tuning: Exploring data-efficient fine-tuning methods tailored for audio models could bridge the gap between limited labeled data and extensive model capabilities.
2. Scaling Laws: Investigating the scaling of model size and pretraining data volume to further enhance performance, particularly as more respiratory audio datasets become available.
3. Novel Pretraining Strategies: Advancing SSL techniques specifically adapted to the unique challenges of respiratory audio, including heterogeneous sound types and complex temporal-frequency correlations.

Conclusion

The introduction of OPERA marks a significant step toward the development of open-source, generalizable respiratory acoustic foundation models. The system not only provides a comprehensive dataset and a robust benchmarking framework but also elucidates the strengths and limitations of various pretraining approaches. This foundational work paves the way for future exploration and application of machine learning in respiratory health monitoring, potentially transforming the landscape of personalized healthcare.