Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement (2405.15973v4)

Abstract: Large vision-LLMs (LVLMs) have achieved impressive results in visual question-answering and reasoning tasks through vision instruction tuning on specific datasets. However, there remains significant room for improvement in aligning visual and language modalities. Existing methods often depend on external models or data, leading to uncontrollable and unstable alignment results. In this paper, we propose SIMA, a self-improvement framework that enhances visual and language modality alignment without external dependencies. SIMA leverages existing vision instruction tuning datasets to self-generate responses, incorporating an in-context self-critic mechanism that constructs preference pairs for tuning. Crucially, our approach allows LVLMs to act as critics by designing effective critic prompts, eliminating the need for additional fine-tuning with external instruction data. We introduce three novel visual metrics within the self-critic process to guide judgment, significantly improving the accuracy of self-critic. Through extensive experiments across 14 hallucination and comprehensive benchmarks, we demonstrate that SIMA significantly improves LVLM's performance and outperforms previous approaches, achieving superior modality alignment.

• The paper introduces SIMA, a self-improvement framework that leverages intrinsic model capabilities to generate self-critiques and enhance visual-language modality alignment.
• It employs a novel in-context self-critic mechanism using visual critic metrics to improve accuracy in object description, relationships, and attributes.
• Experimental results on 14 benchmarks show significant gains, including up to 16.1% reduction in hallucinations and a 7.5% overall performance boost.

Enhancing Visual-Language Modality Alignment in Large Vision LLMs via Self-Improvement

The paper introduces a novel framework, Self-Improvement Modality Alignment (SIMA), aimed at improving the alignment between visual and language modalities in Large Vision LLMs (LVLMs) without the need for external AI models or data. The authors propose an innovative approach leveraging the LVLM’s intrinsic capabilities to generate responses and implement a self-critique mechanism to iteratively enhance its own performance.

Core Contributions

1. Self-Generating and In-Context Self-Critic Mechanism: SIMA employs a self-generating mechanism where the model uses prompts from existing vision instruction tuning datasets to generate responses. These responses are then evaluated using an in-context self-critic mechanism, where the LVLM itself assesses the quality of its responses based on predefined visual critic metrics.
2. Visual Critic Metrics: The paper introduces three key metrics used during the self-critique stage:
   • Accuracy in Object Description: Evaluates how accurately the objects in the image are described.
   • Accuracy in Depicting Relationships: Assesses the correctness in describing relationships between objects.
   • Accuracy in Describing Attributes: Measures the precision in depicting specific attributes of objects.
3. Performance and Benchmarking: The proposed framework is tested on LLAVA-1.5-7B across 14 different benchmarks. Results indicate significant improvements in both hallucination mitigation and comprehensive understanding, with an average performance increase of 7.5%.

Experimental Results

The experiments conducted demonstrate the efficacy of SIMA in enhancing LVLM alignment and performance:

• Hallucination Reduction: Using benchmarks like CHAIR, MM-Hal, and Mementos, SIMA demonstrates substantial reductions in object and behavior hallucination rates. Notably, SIMA achieves an average performance improvement of 16.1% on object hallucination benchmarks.
• Comprehensive Benchmark Performance: On nine comprehensive benchmarks, including LLaVA in the Wild, ScienceQA, TextVQA, and others, SIMA shows an average improvement of 3.5%, outperforming other preference tuning methods and several other open-source LVLMs.

The paper employs various preference tuning methods, drawing comparisons with baselines like LLaVA-RLHF, HA-DPO, and POVID. For instance, in hallucination benchmarks, SIMA outperforms LLaVA-1.5-7B, reducing CHAIRs from 50.8 to 40.9 and improving Mementos-Object F1 scores from 39.29% to 46.08%.

Critical Analysis

The use of self-generated responses and an in-context self-critic in SIMA marks a significant shift from traditional methods that rely on external models and datasets. This approach not only enhances performance but also ensures scalability and cost-effectiveness. By utilizing the model’s own capabilities, SIMA mitigates distribution shift issues commonly introduced by external datasets.

Future Directions

The implications of this research are noteworthy for both practical applications and theoretical advancements. Practically, the reduction in hallucination and improved understanding can enhance the reliability of LVLMs in applications requiring visual comprehension, such as autonomous vehicles, medical imaging analysis, and human-computer interaction.

Theoretically, the success of the self-improvement framework raises questions about the limits of LVLM self-evaluation and improvement. Future research could explore more sophisticated self-critique mechanisms, potentially incorporating unsupervised or semi-supervised learning strategies to further enhance model performance.

Additionally, while SIMA addresses immediate performance improvements, it does not tackle potential biases inherent in self-generated data. Future studies might examine methodologies to detect and correct for such biases, ensuring fairer and more accurate model outputs.

Conclusion

SIMA represents a significant advancement in the design of LVLMs, shifting towards self-reliant improvement mechanisms that do not depend on external models or data. This innovative framework enhances both the alignment between visual and language modalities and the overall performance of LVLMs across various benchmarks. The paper sets a new direction for future research in vision-LLMs, advocating for approaches that leverage intrinsic model capabilities for continuous self-improvement.