AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations (1905.00292v2)

Abstract: The cosine-based softmax losses and their variants achieve great success in deep learning based face recognition. However, hyperparameter settings in these losses have significant influences on the optimization path as well as the final recognition performance. Manually tuning those hyperparameters heavily relies on user experience and requires many training tricks. In this paper, we investigate in depth the effects of two important hyperparameters of cosine-based softmax losses, the scale parameter and angular margin parameter, by analyzing how they modulate the predicted classification probability. Based on these analysis, we propose a novel cosine-based softmax loss, AdaCos, which is hyperparameter-free and leverages an adaptive scale parameter to automatically strengthen the training supervisions during the training process. We apply the proposed AdaCos loss to large-scale face verification and identification datasets, including LFW, MegaFace, and IJB-C 1:1 Verification. Our results show that training deep neural networks with the AdaCos loss is stable and able to achieve high face recognition accuracy. Our method outperforms state-of-the-art softmax losses on all the three datasets.

• The paper presents an adaptive cosine scaling mechanism that dynamically adjusts the scale parameter to optimize deep face recognition performance.
• It demonstrates superior accuracy and stable convergence on benchmarks like LFW, MegaFace, and IJB-C compared to CosFace and ArcFace.
• The approach simplifies model training by eliminating manual hyperparameter tuning while enhancing recognition robustness.

Overview of AdaCos: Adaptively Scaling Cosine Logits for Learning Deep Face Representations

This paper introduces AdaCos, a novel adaptive cosine-based softmax loss function designed to enhance deep face representation learning. The authors propose a method that eliminates the need for manually tuning hyperparameters, specifically the scale and angular margin parameters, which are critical in determining the effectiveness of cosine-based softmax losses used in deep learning face recognition tasks. By automatically adjusting the scale parameter during training, AdaCos ensures stable and high-performance recognition across several benchmarks, outperforming established methods like CosFace and ArcFace.

The paper begins by acknowledging the contributions of deep face representations developed by convolutional neural networks (CNNs) and the role of effective loss functions in model training. It highlights the challenges associated with open-set face recognition tasks, where the training and testing identities often differ.

The authors delve into the mechanisms of existing cosine-based softmax losses, noting the importance of scale and margin parameters in shaping classification probabilities. Specifically, they explore the impact of these parameters on predicted classification probabilities. Their analysis reveals that improper settings can lead to insufficient supervision during training, negatively affecting model convergence and recognition accuracy.

AdaCos sets itself apart by introducing an adaptive scaling mechanism that dynamically modulates the cosine similarities during training. This approach aims to align the predicted class probabilities with their semantic significance, thereby facilitating more effective training supervision without additional computational complexity.

The paper conducts extensive experiments on datasets like LFW, MegaFace, and IJB-C, demonstrating superior performance over state-of-the-art methods. These experiments underline AdaCos's ability to maintain stable convergence and achieve higher recognition accuracies by adaptively tuning the scale parameter, thus eliminating the laborious process of manual hyperparameter optimization.

The implications of this research are significant for the development of more efficient and robust face recognition systems. By addressing the sensitivity of hyperparameters, AdaCos can potentially simplify the deployment of face recognition models while ensuring high accuracy and stability. Future developments in this area could further refine adaptive mechanisms, possibly extending beyond cosine-based functions to broader applications in deep learning and AI.

In summary, this work provides a practical solution to the hyperparameter tuning dilemma in deep face recognition, contributing to the continual advancement of face recognition technologies in both research and industry.