Hina Otake, Shinei Arakawa, Keitaro Tanaka, Yoshihiro Fukuhara, Hirokatsu Kataoka, Shigeo Morishima
8 min
Abstract
Formula-Driven Supervised Learning (FDSL) enables vision pre-training without real images, but existing methods cast pre-training as classification on discretized labels, discarding continuity in the generative process. We propose Formula-Driven Supervised Metric Learning (FDSML), which derives relational supervision from image-generation parameters beyond discrete class labels. FDSML combines FineFractalDB, our refined fractal dataset with higher intra-class variation and more stable structures, with a Parameter-Aware Triplet (PA-Triplet) Loss that assigns positives and negatives according to distances in the underlying Iterated Function System parameter space. This objective trains embeddings to respect parameter-derived relative similarity orderings. Experiments show that FDSML improves transferability to image retrieval while remaining competitive on downstream classification and improving robustness under several corruption settings. On ResNet-50, Recall@1 improves from 65.6% to 74.0% on Stanford Online Products and from 56.2% to 78.4% on Stanford Cars over FDSL pre-training. These results suggest that parameter-derived relational supervision provides a stronger synthetic pre-training signal than label-based classification for transfer beyond recognition.
Alex: Then the network should place that positive nearer the reference than the other candidate. It doesn't have to reproduce the actual parameter distance.
Sam: That distinction matters. Their Parameter-Aware Triplet Loss uses the ordering, not the distance as a calibrated measure of visual resemblance. Training penalizes the network when its image representations fail to keep the positive closer than the negative by a specified margin.
Alex: Calling something a parameter distance doesn't establish that it matches appearance. Could different recipes produce similar summaries but very different pictures?
Sam: They can, and the paper tests that problem directly. First, though, there's a sampling decision. Class labels still restrict which candidates enter a comparison, but they don't make the final positive and negative assignments.
Alex: So the labels haven't disappeared. What does restricting the candidates buy them?
Sam: Consider the strategy called “same.” Both candidates belong to the reference image's category. Their shared label cannot distinguish them, but their parameters can rank subtle differences. Another strategy, called “each,” chooses a within-category candidate and an outside-category candidate.
Alex: Those test different things. The within-category comparison asks for information the label cannot supply. The mixed comparison might mostly recover the label distinction.
Sam: That is what their validation finds. Under the mixed strategy, parameter-based assignments almost always match the class-label rule. Under the within-category strategy, the descriptor provides a distinction that labels alone cannot express.
Alex: What reference do they use to check whether that distinction corresponds to the rendered shape?
Sam: They use Chamfer distance, which compares foreground pixels with their nearest counterparts in the other rendering. Within-category parameter rankings agree with these shape rankings roughly eighty percent of the time. Chance agreement would be fifty percent.
Alex: That's useful, but not perfect. Does the signal survive when candidates come from unrelated categories?
Sam: No. With unrestricted candidates, or candidates exclusively outside the reference category, agreement falls below chance. The descriptor compresses a whole collection of transformations into a small summary. Different collections can therefore receive nearby summaries without looking structurally similar.
Alex: So the defensible claim is local: parameter comparisons can supply useful supervision within these categories. It isn't evidence that the descriptor measures visual similarity everywhere.
Sam: That's the key boundary. The authors also refine the images themselves, creating FineFractalDB with a million generated images. They increase independent parameter variation within categories and enforce contracting transformations, which help prevent unstable or collapsed fractal structures.
Alex: Now both the dataset and training objective have changed. How do they know which change helps?
Sam: Their classification ablations hold the original FractalDB dataset fixed and replace classification training with the triplet objective. Performance improves on each tested benchmark in that comparison. The refined dataset adds further gains under metric learning, but does not improve conventional classification pre-training.
Alex: That supports a contribution from the objective itself, at least for those classification tests. What happens on retrieval, where the quality of the comparisons should matter most?
Sam: They pre-train a ResNet-fifty model, then fine-tune it for retrieval using the same downstream objective across pre-training strategies. On Stanford Online Products, recall at one rises from about sixty-six percent with conventional fractal pre-training to seventy-four percent with their method. That's how often the top-ranked retrieved image is relevant.
Alex: That gives the gain a concrete meaning: the first retrieved result is correct more often. Does another dataset show the same direction?
Sam: On Stanford Cars, the corresponding result rises from about fifty-six percent to seventy-eight percent. Their method also improves clothing retrieval. Across these retrieval datasets, it beats both conventional fractal pre-training and training from scratch, although supervised ImageNet pre-training remains ahead.
Alex: And “without real images” describes pre-training, not the entire experimental pipeline. They still use natural-image datasets for downstream fine-tuning.
Sam: Yes. These results show transfer from synthetic pre-training after downstream fine-tuning. They do not show a retrieval system trained entirely without real images. Also, the headline retrieval comparison changes both the pre-training data and objective; the fixed-data ablations concern classification.
Alex: How broad are the other benefits? Does the method consistently help classification and resistance to corrupted images?
Sam: Classification is competitive, with gains on several benchmarks, rather than uniformly better. Corruption results depend on both architecture and benchmark. On ImageNet-C, the convolutional model improves across corruption groups, while the small vision transformer worsens on noise and digital corruptions; on CIFAR-ten-C, that transformer improves across groups.
Alex: Then this isn't a blanket robustness claim. What else limits how far someone should take the findings?
Sam: Validation is confined to these fractal generators and this dataset scale. Other formula-driven generators would need their own parameter similarity definitions and tests. The method also adds descriptor computation and triplet construction, and the paper doesn't analyze that overhead in detail.
Alex: Who should read the full document, and where should they begin before deciding to use the method?
Sam: Researchers working on synthetic pre-training or retrieval should read “Validating the Parameter Descriptor” first, then “Image Retrieval.” Follow with the fixed-dataset comparison in “Ablation Studies.” Together, those sections show where the supervision is meaningful, what transfers, and what can be attributed to the training objective.
Alex: And for someone who doesn't work on fractals, what's the one idea worth carrying into their own research?
Sam: A generator can provide more than category labels: it can provide comparisons—but those comparisons need validation. This study makes that useful within fractal categories, not universally across their parameter space.
Alex: Thanks for listening.