Confidence Regularized Self-Training (1908.09822v3)

Abstract: Recent advances in domain adaptation show that deep self-training presents a powerful means for unsupervised domain adaptation. These methods often involve an iterative process of predicting on target domain and then taking the confident predictions as pseudo-labels for retraining. However, since pseudo-labels can be noisy, self-training can put overconfident label belief on wrong classes, leading to deviated solutions with propagated errors. To address the problem, we propose a confidence regularized self-training (CRST) framework, formulated as regularized self-training. Our method treats pseudo-labels as continuous latent variables jointly optimized via alternating optimization. We propose two types of confidence regularization: label regularization (LR) and model regularization (MR). CRST-LR generates soft pseudo-labels while CRST-MR encourages the smoothness on network output. Extensive experiments on image classification and semantic segmentation show that CRSTs outperform their non-regularized counterpart with state-of-the-art performance. The code and models of this work are available at https://github.com/yzou2/CRST.

• The paper introduces a CRST framework that incorporates confidence regularization to reduce error propagation from noisy pseudo-labels in self-training.
• It reformulates self-training as a regularized optimization problem using soft pseudo-labels, linking label smoothing with entropy minimization.
• Experimental results show significant performance gains in image classification and semantic segmentation, validating the framework’s effectiveness.

Confidence Regularized Self-Training (CRST)

In recent advancements of unsupervised domain adaptation (UDA), self-training methods have shown considerable promise due to their iterative mechanism of pseudo-labeling and retraining. However, these methods are susceptible to the propagation of errors due to noisy pseudo-labels, which leads to performance degradation due to overconfident wrong class predictions. The paper Confidence Regularized Self-Training (CRST) proposes a novel framework to mitigate this issue by incorporating confidence regularization into the self-training process.

Key Contributions

1. Framework Introduction: The CRST framework introduces confidence regularization to address the robustness problem in self-training. The pseudo-labels are treated as continuous latent variables, optimized via alternating optimization. Two regularization methods are proposed:
   • Label Regularization (CRST-LR): It generates soft pseudo-labels to diffuse the label distribution.
   • Model Regularization (CRST-MR): It encourages output smoothness and discourages overly confident predictions.
2. Mathematical Formulation: The paper reformulates the self-training approach into a regularized optimization problem. CRST's core is an entropy minimization objective augmented with a regularizer. The pseudo-labels are optimized within a continuous space, which contrasts with conventional discrete one-hot encoding.
3. Theoretical Foundation: It connects CRST to classification expectation maximization (CEM) and proves its convergence under certain conditions. Theoretical insights include:
   • Label regularized pseudo-labels as an instance of softmax with temperature.
   • Equivalent transformations between KLD regularization and label smoothing techniques.
4. Experimental Validation: Extensive experiments demonstrate CRST's applicability across different domain adaptation tasks including image classification and semantic segmentation:
   • Image Classification: CRST outperforms baseline methods on benchmarks like VisDA17 and Office-31.
   • Semantic Segmentation: Remarkable improvement is shown in synthetic to real adaptation tasks (e.g., GTA5 to Cityscapes).

Practical Insights and Future Directions

1. Numerical Results: The numerical experiments present compelling results:
   • On VisDA17, CRST variants (MRKLD, LRENT) achieve a mean accuracy improvement over non-regularized self-training, exemplifying the efficacy of confidence regularization.
   • In Office-31, despite the smaller dataset, CRST maintains a clear performance edge.
   • In semantic segmentation tasks, CRST consistently provides superior feature alignment and less error propagation across class boundaries.
2. Comparative Analysis: The paper finds MRKLD to be particularly robust, largely due to its effective mitigation of false positives while maintaining true positive rates. It also reveals that combining MR and LR types can synergize their strengths, albeit at a computational cost.
3. Practical Recommendations: The paper emphasizes practical scenarios where MRKLD is preferred due to its simplicity and effectiveness. It warns about potential additional overheads using label regularization due to the need for dataset-level storage of soft labels.

Implications and Future Directions

The CRST framework extends the boundary of self-training methods by addressing their fundamental challenge of noisy pseudo-labels. The theoretical underpinnings and experimental validation make a strong case for incorporating confidence regularization in future UDA methods. This has significant implications in fields relying on domain adaptation, such as autonomous driving and robotic vision, where cross-domain shifts are prevalent.

Future research could investigate:

• Adaptive regularization techniques that dynamically adjust the regularizer weight based on model confidence.
• Scalability of CRST in more complex and large-scale domains, including video tasks or multi-modal data.
• Extending the framework to semi-supervised learning scenarios where partial labeling information is available, potentially enhancing the general applicability of the approach.

This paper delivers a nuanced understanding of domain adaptation through self-training, proposing a well-founded and experimentally validated framework that sets a concrete ground for future explorations in the domain.