Efficient Neural Network Model Selection for Few-Class Application Datasets

Most neural network research focuses on large-scale datasets with hundreds or thousands of classes. However, many real-world applications—such as mobile robotics, drone navigation, and IoT monitoring—rely on datasets with very few classes (typically fewer than ten). The authors investigate whether standard model selection practices, which often rely on extrapolating performance from large datasets, are inefficient for these specific, resource-constrained use cases.

The authors propose a data-side approach to model selection by defining a quantitative metric for "classification difficulty." This metric is derived from the geometric properties of the data, specifically measuring intra-class similarity (how similar items within the same class are) and inter-class similarity (how similar items are across different classes). By calculating these similarities using cosine distance in the feature space, the authors create a lightweight proxy for model performance. This allows practitioners to compare different models and datasets 6 to 29 times faster than by performing full training and testing cycles.

The study identifies a phenomenon termed "few-class distinctiveness," where classification accuracy follows a distinct pattern in few-class settings that differs from large-scale benchmarks. By leveraging their difficulty metric, the authors demonstrate that they can identify optimal model architectures more efficiently. They successfully applied this to scale down model families, producing versions smaller than existing published models (e.g., up to 42% smaller than YOLOv5-nano) while maintaining comparable accuracy for specific tasks. This confirms that focusing on dataset-specific properties allows for the selection of leaner, more efficient models tailored to the actual complexity of the target application.