MatBind: A Shared Embedding Space for Multimodal Materials Characterization

Materials science data is inherently fragmented, with critical information stored in disparate formats—atomic structures, diffraction patterns, electronic properties, and textual literature. This isolation makes it difficult for researchers to query across modalities, such as using a measured diffraction pattern to find a material's electronic properties. MatBind addresses this by creating a shared, unified embedding space that treats these diverse data types as complementary projections of the same physical reality.

Inspired by the ImageBind framework, the authors use crystal structure as a central physical "anchor." They train four separate encoders—a graph neural network for crystal structures, a ResNet for pXRD patterns, a transformer for density of states (DOS), and a domain-specific BERT model for text—using pairwise contrastive objectives. By aligning each modality to the crystal structure anchor, the model implicitly learns to align all other pairs, even those that never appear together during training. This creates a shared representation where materials with similar physical properties cluster together regardless of the input modality.

The researchers demonstrate that MatBind successfully organizes materials according to physically meaningful properties, such as band gap and crystal system, without explicit supervision. The framework achieves high performance in cross-modal retrieval, with crystal structure-to-text and crystal structure-to-DOS pairs showing near-perfect recall. Notably, the model exhibits emergent zero-shot capabilities; for instance, the link between DOS and text performs better than some directly trained pairs. Furthermore, the authors show that combining modalities at query time—such as using a pXRD pattern alongside a partial textual description—significantly resolves identification ambiguities, proving that these modalities provide complementary constraints on the underlying physical system.