Multi-Modal Contrastive Learning for Implicit Earth Embeddings via Location Tying

Spatial prediction tasks often suffer from a scarcity of labeled ground-truth data. While unlabelled geospatial data—such as satellite imagery, geotagged photos, and text—is abundant, traditional supervised models cannot effectively leverage it. This paper explores whether incorporating multiple, unpaired modalities into a contrastive learning framework can improve the quality of implicit Earth embeddings (vector representations of geographic coordinates).

The authors propose two architectures to address this:

• MELT (Multimodal Embedding via Location Tying): A joint training approach that processes multiple modalities simultaneously in each batch, using co-location as the bridge to align them in a shared embedding space.
• SALT (Sequential Alternating Location Training): An epoch-level scheduling approach that alternates which non-location modality is active during training, keeping the location encoder permanently active.

Both methods use geographic coordinates as a shared anchor, allowing the model to learn from unpaired data sources without requiring synchronized multi-modal observations.

The empirical results show that both MELT and SALT are technically viable and perform competitively with the strongest two-modality baselines (like SATCLIP) across four global downstream tasks: elevation, population density, country classification, and biome classification.

However, the study reveals a critical bottleneck: increasing the number of modalities does not consistently yield performance improvements. The authors conclude that the location encoder architecture itself, rather than the diversity of the input modalities, acts as the primary performance ceiling. The contrastive objective appears to reach its peak effectiveness early, regardless of how much additional modality diversity or pre-training volume is introduced.

This research provides a clear empirical assessment of multimodal scaling for geospatial foundation models. By demonstrating that adding more data sources does not automatically improve downstream performance, the authors shift the focus of future research toward improving the location encoder architecture rather than simply aggregating more diverse data modalities. MELT, in particular, is highlighted as a more stable foundation for future scaling efforts compared to the alternating approach of SALT.