Perturbation Analysis of Neural Collapse (2210.16658v2)

Abstract: Training deep neural networks for classification often includes minimizing the training loss beyond the zero training error point. In this phase of training, a "neural collapse" behavior has been observed: the variability of features (outputs of the penultimate layer) of within-class samples decreases and the mean features of different classes approach a certain tight frame structure. Recent works analyze this behavior via idealized unconstrained features models where all the minimizers exhibit exact collapse. However, with practical networks and datasets, the features typically do not reach exact collapse, e.g., because deep layers cannot arbitrarily modify intermediate features that are far from being collapsed. In this paper, we propose a richer model that can capture this phenomenon by forcing the features to stay in the vicinity of a predefined features matrix (e.g., intermediate features). We explore the model in the small vicinity case via perturbation analysis and establish results that cannot be obtained by the previously studied models. For example, we prove reduction in the within-class variability of the optimized features compared to the predefined input features (via analyzing gradient flow on the "central-path" with minimal assumptions), analyze the minimizers in the near-collapse regime, and provide insights on the effect of regularization hyperparameters on the closeness to collapse. We support our theory with experiments in practical deep learning settings.

• The paper proves that optimized features exhibit reduced within-class variability compared to input features, aligning with empirical observations in deep networks.
• The paper provides a rigorous gradient flow analysis demonstrating a monotonic decrease in the NC1 metric, with exponential convergence driven by feature regularization.
• The paper’s perturbation analysis establishes a linear relationship between feature perturbations, offering insights into the distinct roles of intra-class and inter-class variations.

The paper "Perturbation Analysis of Neural Collapse" (2210.16658) addresses a significant gap between theoretical studies of Neural Collapse (NC) and empirical observations in practical deep learning. While idealized "unconstrained features models" (UFMs) predict exact collapse of features (zero within-class variability and perfect simplex equiangular tight frame structure for class means), real-world neural networks trained on complex datasets exhibit a reduction in NC metrics, but rarely reach exact zero collapse. Furthermore, hyperparameters like regularization are empirically known to affect the degree of collapse, a phenomenon not explained by standard UFMs where all minimizers exhibit exact collapse regardless of non-zero regularization levels.

To bridge this gap, the authors propose a novel model that extends the UFM framework. They introduce an objective function for optimizing the final layer weights $W$ and the deepest features $H$, which includes a regularization term $\frac{\beta}{2Kn}\|H - H_0\|_F^2$. Here, $H_0$ represents a fixed "input" features matrix (e.g., from an intermediate layer) and $\beta$ is a positive hyperparameter. This term forces the optimized features $H$ to stay close to $H_0$. The authors interpret this model as capturing the relationship between features of successive layers in a deep neural network, where $H_0$ are features from a shallower layer and $H$ are features from a deeper layer, and the magnitude of $\beta$ reflects the complexity (or simplicity) of the subnetwork connecting these layers. The focus is on the large $\beta$ regime, where $H$ is constrained to be in the vicinity of $H_0$.

The paper makes several key contributions:

1. Proof of Depthwise Within-Class Variability Reduction: A major finding is the theoretical proof that the optimized features $H$ exhibit lower within-class variability compared to the input features $H_0$, provided $H_0$ is not already fully collapsed. This aligns with empirical observations that deeper layers often show reduced within-class scatter. The proof is derived by analyzing the gradient flow of an associated UFM objective (where $W$ is optimally determined by $H$). The analysis shows that the $\widetilde{NC}_1$ metric (ratio of trace of within-class covariance to trace of between-class covariance) strictly decreases along this flow.
   • Practical Implication: This provides a theoretical justification for the progressive reduction of within-class variability observed across layers during deep network training. It suggests that even simple mappings from a preceding layer, when optimized appropriately, can reduce the scatter of samples belonging to the same class in the feature space. This supports the idea that feature extractors iteratively refine class representations by reducing intra-class variance.
2. Detailed Gradient Flow Analysis: The paper provides a rigorous analysis of the gradient flow for the associated UFM, notably with minimal assumptions compared to prior work (e.g., not assuming zero global mean or invertibility of covariance matrices, and avoiding artificial renormalization steps). This analysis shows a monotonic decrease in $\widetilde{NC}_1$. Crucially, it establishes a separation in the behavior of the within-class covariance trace $\Tr(_W(H_t))$ and the between-class covariance trace $\Tr(_B(H_t))$. $\Tr(_W)$ decreases, while $\Tr(_B)$ increases. Furthermore, if $\lambda_H > 0$, $\Tr(_W(H_t))$ decays exponentially, implying an exponential rate of convergence for $\widetilde{NC}_1$ towards zero.
   • Practical Implication: This analysis illuminates the dynamics of within-class variance reduction during training. The finding that $\lambda_H$ (regularization on features) drives exponential decay of $\Tr(_W)$ suggests that appropriate regularization on the feature extractor layers is critical for effectively collapsing within-class features during the terminal phase of training.
3. Analysis in the Near-Collapse Regime: By performing a perturbation analysis around an exactly collapsed state, the authors derive a closed-form approximation for the change in optimized features ($\delta H$) resulting from a small perturbation in the input features ($\delta H_0$). The relationship is approximately linear: $$\mathrm{vec}(\delta H) \approx F \, \mathrm{vec}(\delta H_0)$$, where $F$ is a matrix derived from second-order derivatives of the objective function. The analysis of the singular values of $F$, particularly its block structure corresponding to intra-class ($F_{k,k}$) and inter-class ($F_{k,\tilde{k}}$) perturbations, reveals insights into how deviations from collapse in $H_0$ propagate to $H$.
   • Practical Implication: This analysis provides a framework for understanding how factors influencing intermediate features (like data variations, noise, or architectural constraints) affect the final features' distance from exact collapse.
     • The finding that intra-class blocks ($F_{k,k}$) are significantly more dominant (full rank) than inter-class blocks ($F_{k,\tilde{k}}$ are rank-1) suggests that deviations from collapse are primarily driven by within-class variations in the preceding layer, rather than by interference from other classes. This implies a degree of class-wise refinement occurs relatively independently near the collapsed state.
     • The analysis of the effect of $\lambda_H$ and $\lambda_W$ on the singular values of these blocks shows that $\lambda_H$ primarily attenuates the dominant intra-class perturbations, while $\lambda_W$ helps reduce the influence of inter-class perturbations. This provides a theoretical explanation for the empirical observation that feature regularization ($\lambda_H$, analogous to weight decay on feature layers) often has a stronger impact on reducing NC metrics than classifier regularization ($\lambda_W$, analogous to weight decay on the classification layer).
     • The analysis also offers a potential explanation for why NC1 metrics often plateau at lower values than other NC components (like NC2). Certain singular vectors corresponding to NC1-irrelevant perturbations are not attenuated by regularization.
4. Empirical Validation: The theoretical findings are supported by experiments on CIFAR-10 and MNIST datasets using practical network architectures like ResNet18 and MLP.
   • Experiments with layer-wise training of an MLP demonstrate the predicted depthwise decrease in NC1 metrics.
   • Experiments varying weight decay only in the feature mapping layers versus only in the classification layer confirm that modifying feature mapping regularization has a larger impact on the final NC metrics (NC1 and NC2), aligning with the theoretical prediction about the dominance of $\lambda_H$ in attenuating key perturbations.
   • Further experiments using a dataset with blurred classes show that $\lambda_W$ helps mitigate the negative impact of these "hard" classes on the feature structure of other classes, supporting the role of $\lambda_W$ in handling inter-class interference.

In summary, this paper offers a more realistic model for studying Neural Collapse in deep networks by incorporating the influence of preceding layers via a regularization term. Through perturbation analysis in the small vicinity of input features and gradient flow analysis, it provides theoretical justifications for empirically observed phenomena like depthwise NC1 reduction, the distinct roles of feature and classifier regularization in approaching collapse, and the relative impact of these regularization types on different components of the NC structure. The work provides valuable insights into the mechanisms driving feature refinement and collapse in practical deep learning settings and highlights the importance of regularization strategies aligned with theoretical understanding. The perturbation analysis approach developed here is potentially applicable to studying NC in other complex settings, such as different loss functions, multi-layer models, or imbalanced data.