Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data. We further curate DICS-6M, a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date; remarkably, DICS reaches 94.52\% of the official InternVL3-8B-Instruct performance using less than 25\% of its reported training data. Code can be seen at https://github.com/cqu-student/DICS
Alex: Welcome to another episode of ResearchPod. Today we're looking at DICS — a framework for selecting high-quality training subsets for vision-language models, and a direct challenge to the assumption that bigger datasets are always better.
Sam: So the paper is attacking the bottleneck of training on massive, noisy multimodal datasets. What's the central claim?
Alex: That you can drop to less than 25% of your original training volume and actually improve downstream performance — if you select the right 25%. The key is what the authors call "intrinsic consistency," which is a fundamentally different selection criterion from the gradient similarity or global distribution matching that most existing methods rely on.
Sam: What does intrinsic consistency actually mean in practice?
Alex: The core idea is that each training sample should be evaluated on its own internal coherence, not just its relationship to the rest of the dataset. They operationalize this with two metrics. The first is Visual Information Consistency — essentially asking whether the image is actually necessary to answer the question. The second is Response Information Consistency — checking whether the model's response is genuinely grounded in both the image and the prompt, rather than just plausible-sounding language.
Sam: So it's a kind of blindfold test? If the model can answer just as well without seeing the image, the sample gets flagged as redundant?
Alex: Exactly. The mechanism is a loss differential — you compare the model's uncertainty on a sample when it has full visual access versus when the image is masked. A large gap means the image is doing real inferential work. A small gap means the model is coasting on language priors, and that sample is contributing very little to multimodal learning — or worse, it's reinforcing hallucination.
Sam: That's a clean diagnostic. But scoring every sample with forward passes sounds expensive. How do they scale it?
Alex: They use an adaptive selection strategy that shifts based on the target subset size. For smaller subsets, they prioritize samples with the highest DIC scores — pure quality filtering. For larger subsets, they layer in Farthest Point Sampling to maintain distributional coverage while still pruning what they call "lazy" samples — the ones the model can answer without engaging vision at all. So it's not just a quality filter; it's a quality-plus-diversity filter, with the balance tuned to the operating regime.
Sam: Does this actually generalize across architectures, or is it tuned to whatever backbone they developed it on?
Alex: That's where the cross-architecture results are worth paying attention to. They demonstrate what they call Weak-to-Strong generalization — data selected by a smaller 7B-parameter model transfers meaningfully to a larger model. The implication is that the DIC signal is capturing something about the data itself, not just the idiosyncrasies of the scoring model. That said, I'd want to see this tested more broadly before treating it as a general principle. The paper's architecture comparisons are suggestive, but they're not an exhaustive ablation across model families.
Sam: And the headline performance number — maintaining over 94% of full-dataset performance with less than a quarter of the data — how load-bearing is that result?
Alex: It's the central empirical claim, and it holds across the benchmarks they report. But the important qualifier is that "94% of full-dataset performance" is a relative measure. The absolute gains depend heavily on how noisy the original dataset is. If you start with a relatively clean corpus, the filtering dividend will be smaller. The method is most compelling in the high-noise regime, which is where most large-scale multimodal datasets actually live — web-scraped image-caption pairs with weak alignment between modalities.
Sam: So the noisier the dataset, the more the DIC filter earns its keep.
Alex: Right. And the practical upside of the Weak-to-Strong result is significant — if a small model can curate data for a much larger one, you're front-loading compute into a relatively cheap scoring phase rather than burning it on training runs with degraded data. That's a meaningful shift in where the compute budget goes.
Sam: What are the honest limitations here?
Alex: Two main ones. First, the scoring cost scales linearly with dataset size — every sample needs inference-time forward passes, which isn't trivial at the scale of modern pretraining corpora. It's a favorable trade relative to wasted training compute, but it's not free. Second, the framework currently handles static image-text pairs only. Video and audio are the obvious next frontiers for multimodal learning, and it's not clear how the DIC signal would translate to temporal or sequential modalities where the grounding relationship is more complex.
Sam: There's also an interesting direction they gesture at — rather than discarding low-DIC samples, using the signal to trigger rewriting. If a sample is almost useful but the caption is misaligned, you could repair it rather than throw it away.
Alex: That's a natural extension, and it would change the framework from a filter into something more like an active curation pipeline. Whether that's practical at scale is an open question, but it's the right direction to think about — especially as the field moves toward synthetic data generation, where you have more control over what gets produced in the first place.
Sam: So the core contribution is a principled diagnostic for multimodal data quality — one that's grounded in whether visual information is actually doing inferential work, rather than just whether a sample looks diverse relative to the rest of the corpus.
Alex: That's the crux of it. The shift from "collect more data" to "verify that each sample is earning its place" is a meaningful reframing — and if the cross-architecture generalization holds up under broader scrutiny, it suggests the DIC signal is capturing something real about multimodal learning, not just overfitting to one experimental setup. Thanks for listening to ResearchPod.