Training from a Better Start Point: Active Self-Semi-Supervised Learning for Few Labeled Samples

Training with fewer annotations is a key issue for applying deep models to various practical domains. To date, semi-supervised learning has achieved great success in training with few annotations. However, confirmation bias increases dramatically as the number of annotations decreases making it difficult to continue reducing the number of annotations. Based on the observation that the quality of pseudo-labels early in semi-supervised training plays an important role in mitigating confirmation bias, in this paper we propose an active self-semi-supervised learning (AS3L) framework. AS3L bootstraps semi-supervised models with prior pseudo-labels (PPL), where PPL is obtained by label propagation over self-supervised features. We illustrate that the accuracy of PPL is not only affected by the quality of features, but also by the selection of the labeled samples. We develop active learning and label propagation strategies to obtain better PPL. Consequently, our framework can significantly improve the performance of models in the case of few annotations while reducing the training time. Experiments on four semi-supervised learning benchmarks demonstrate the effectiveness of the proposed methods. Our method outperforms the baseline method by an average of 7\% on the four datasets and outperforms the baseline method in accuracy while taking about 1/3 of the training time.