ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning (2404.15449v1)

Abstract: The rapid development of diffusion models has triggered diverse applications. Identity-preserving text-to-image generation (ID-T2I) particularly has received significant attention due to its wide range of application scenarios like AI portrait and advertising. While existing ID-T2I methods have demonstrated impressive results, several key challenges remain: (1) It is hard to maintain the identity characteristics of reference portraits accurately, (2) The generated images lack aesthetic appeal especially while enforcing identity retention, and (3) There is a limitation that cannot be compatible with LoRA-based and Adapter-based methods simultaneously. To address these issues, we present \textbf{ID-Aligner}, a general feedback learning framework to enhance ID-T2I performance. To resolve identity features lost, we introduce identity consistency reward fine-tuning to utilize the feedback from face detection and recognition models to improve generated identity preservation. Furthermore, we propose identity aesthetic reward fine-tuning leveraging rewards from human-annotated preference data and automatically constructed feedback on character structure generation to provide aesthetic tuning signals. Thanks to its universal feedback fine-tuning framework, our method can be readily applied to both LoRA and Adapter models, achieving consistent performance gains. Extensive experiments on SD1.5 and SDXL diffusion models validate the effectiveness of our approach. \textbf{Project Page: \url{https://idaligner.github.io/}}

• The paper introduces a feedback learning framework using identity consistency and aesthetic rewards to improve identity preservation in text-to-image generation.
• The methodology integrates face recognition and human preference data, demonstrating compatibility with both LoRA and Adapter models while reducing training time.
• Quantitative results show improved Face Sim, DINO, CLIP-I, and LAION-Aes scores compared to competing methods across various diffusion model architectures.

Enhancing Identity-Preserving Text-to-Image Generation

Introduction

The paper "ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning" (2404.15449) addresses the challenges of identity-preserving text-to-image (ID-T2I) generation, focusing on maintaining identity characteristics, enhancing aesthetic appeal, and ensuring compatibility with both LoRA and Adapter methods. ID-T2I has vast applications, from AI portraits to advertising, necessitating accurate identity retention and pleasing aesthetics in generated images. Although existing methods like LoRA and IP-Adapter provide satisfactory personalization, they struggle with identity preservation and aesthetic quality, and lack a unified approach compatible with multiple frameworks.

Figure 1: An overview of the ID-Aligner method incorporating feedback learning for identity preservation and aesthetic enhancement.

ID-Aligner proposes a feedback learning framework using identity consistency rewards and identity aesthetic rewards. These mechanisms prioritize accurate identity preservation by employing face recognition models and human preference data while ensuring visual appeal through structured feedback. The method's compatibility across different model architectures promises a comprehensive solution to improve identity retention and aesthetic presentation simultaneously, demonstrating performance improvements across various experiments using SD1.5 and SDXL models.

Methodology

ID-Aligner utilizes feedback learning in diffusion models to address identity features and aesthetics concurrently.

Text-to-Image Diffusion Model

Diffusion models generate images by transforming Gaussian noise into structured data through iterative denoising. This process utilizes pre-trained VAE encoders to obtain latent representations, which are progressively refined using UNet models conditioned on text prompts. The denoising objective minimizes deviation from real-world data, forming the basis of ID-T2I frameworks.

Identity Reward

The identity reward system involves two components:

• Identity Consistency Reward: Implements face detection and recognition models to calculate cosine similarity between embeddings from reference and generated faces, providing direct feedback for identity alignment during generation (Equation \ref{eq:idsim}).
• Identity Aesthetic Reward: Combines appeal scoring via human-annotated data and structure validation using ControlNet-generated negative samples (Figure 2), guiding the model to produce aesthetically pleasing images (Equation \ref{eq:reward_training}).

  Figure 2: Construction of aesthetic feedback data using manual and automatic methods to improve image structures and appeal.

The feedback guiding the UNet model ensures adherence to identity fidelity and enhances visual aesthetics through systematic preference scoring.

Experimental Results

Extensive experiments validate ID-Aligner's effectiveness in both identity preservation and aesthetic enhancement.

Adapter and LoRA Models

In both SD1.5 and SDXL architectures, ID-Aligner exhibits superior performance over existing methods (Figure 3). Notably, when integrated with LoRA models, ID-Aligner accelerates identity adaptation, reducing training time and increasing identity similarity and aesthetic quality, demonstrating adaptability across model variations (Figure 4).

Figure 3: Comparison of identity conditional generation results showcasing enhanced identity retention and aesthetics with ID-Aligner.

Figure 4: Visual results from LoRA models highlighting accelerated training and improved identity preservation.

Quantitative Analysis

ID-Aligner achieves higher Face Sim., DINO, CLIP-I, and LAION-Aes scores than competing methods, indicating improved identity alignment and aesthetic appeal. The model's adaptability to diverse prompts further showcases its robustness (Table \ref{tab:quantitative_results}).

Conclusion

The ID-Aligner framework effectively enhances identity-preserving text-to-image generation by integrating feedback learning models. By addressing critical challenges in identity retention and aesthetics, it sets new standards for future research in adaptive generative models, promoting broader applications across domains like portrait generation, virtual environments, and personalized advertising.