Training Dynamics of Neural Software Defect Predictors under Coupled Data-Quality Issues

Software defect prediction (SDP) models are essential for managing software quality, yet their reliability is often undermined by poor data quality. Two pervasive issues are class imbalance (where defective instances are rare) and class overlap (where defective and non-defective instances share similar feature-space regions). While previous research has evaluated these issues using endpoint performance metrics like F1-score or AUC, these metrics are often insufficient for debugging. They reveal that a model is failing, but they do not explain whether the failure stems from skewed class representation, ambiguous boundaries, or an interaction between the two.

This study shifts the focus from endpoint performance to neural training dynamics—such as gradient magnitudes, weight updates, and error trajectories—to identify diagnostic signals of these data-quality issues. The primary objective is to develop an interaction-aware protocol that accounts for the fact that imbalance and overlap are often coupled in practice.

The authors conduct a controlled intervention study using class-level datasets from the Unified Bug Dataset (UBD). They train a fixed multilayer perceptron (MLP) under three conditions: imbalance-only, overlap-only, and joint conditions. To ensure methodological rigor, the study employs:

• Fidelity Monitoring: The researchers recompute realized imbalance (C2) and overlap (N1) metrics after every intervention to track non-target drift.
• Controlled Interventions: Imbalance is manipulated via progressive undersampling, while overlap is introduced by moving boundary-near instances toward opposite-class neighbors.
• Pattern Characterization: The study uses effect sizes, trajectory analysis, and rule-based classification to categorize training-dynamics patterns as issue-associated, shared, or joint-condition specific.

By cataloging how specific data-quality issues manifest in the internal learning process of neural networks, this research provides a foundation for more effective model debugging. Instead of relying on trial-and-error, practitioners can use these training-dynamics patterns to determine whether they should rebalance their data, refine feature boundaries, or reconsider the model's suitability for a specific maintenance task. This approach moves the field toward more transparent and trustworthy defect-prediction pipelines.