Accurate ocean mapping is essential for applications such as bathymetry estimation, seabed characterization, marine litter detection, and ecosystem monitoring. However, ocean remote sensing (RS) remains constrained by limited labeled data and by the reduced transferability of models pre-trained mainly on land-dominated Earth observation imagery. In this paper, we propose OceanMAE, an ocean-specific masked autoencoder that extends standard MAE pre-training by integrating multispectral Sentinel-2 observations with physically meaningful ocean descriptors during self-supervised learning. By incorporating these auxiliary ocean features, OceanMAE is designed to learn more informative and ocean-aware latent representations from large- scale unlabeled data. To transfer these representations to downstream applications, we further employ a modified UNet-based framework for marine segmentation and bathymetry estimation. Pre-trained on the Hydro dataset, OceanMAE is evaluated on MADOS and MARIDA for marine pollutant and debris segmentation, and on MagicBathyNet for bathymetry regression. The experiments show that OceanMAE yields the strongest gains on marine segmentation, while bathymetry benefits are competitive and task-dependent. In addition, an ablation against a standard MAE on MARIDA indicates that incorporating auxiliary ocean descriptors during pre-training improves downstream segmentation quality. These findings highlight the value of physically informed and domain-aligned self-supervised pre- training for ocean RS. Code and weights are publicly available at https://git.tu-berlin.de/joanna.stamer/SSLORS2.
Alex: Welcome to another episode of ResearchPod. Sam, I've been thinking about how satellites watch the oceans for things like oil spills or plastic trash, but the AI tools often struggle there. What's this paper got to say about fixing that?
Sam: This paper, "OceanMAE: A Foundation Model for Ocean Remote Sensing," shows that computers trained mostly on land pictures struggle with ocean images. The authors built OceanMAE, which learns straight from unlabeled satellite photos of water. It uses a huge set from around the world to understand ocean patterns.
Alex: So land-trained AI just sees the sea as blurry blue patches with glare from waves and sun? OceanMAE fixes that by getting ocean-smart?
Sam: Yes—standard models are tuned for solid ground, not shifting sparkles and colors on water. The paper shows adding physical details, like water depth or plant levels, helps the AI spot real features amid the noise. That builds better tools for mapping trash or spills.
Alex: Labeling every ocean photo by hand is tough and expensive. How does OceanMAE learn without those labels?
Sam: It starts like a puzzle game: the computer sees a satellite image but with random sections hidden. It guesses what's under those covers from the visible parts, forcing it to learn ocean patterns. This is called a masked autoencoder, or MAE. To make it ocean-specific, they add maps of water depth—bathymetry—and chlorophyll levels, which track tiny ocean plants.
Alex: Like giving the AI a cheat sheet on depth and clarity to fill in blanks better? That helps it tell a deep blue patch from a shallow one that looks the same.
Sam: Precisely. They mask most of the image—about 90 percent—and turn the visible bits into a compact code. They adjust the ocean details to match, combine them, then rebuild the hidden parts. The system improves by getting closer to the real hidden bits, capturing water tricks like sun glint on waves.
Alex: So it's learning what makes ocean water different from land, not just memorizing pixels. Why does blending those physical facts matter in rebuilding?
Sam: Picture drawing a seascape blindfolded; notes like "shallow here with algae" make it accurate. The physical info gets reshaped to match the image code and combined with a summary from visible patches. This visual-plus-physical mix goes to the decoder, helping spot waves that look like debris. The paper suggests it creates richer codes that work well on real tasks.
Alex: That's why land models fail on glare or ripples—they miss water context. How do they turn these codes into spotting plastic?
Sam: They feed the code into a UNet network, which squeezes an image to key features then expands it back, keeping details sharp. They fuse the code at the squeeze point by stacking it with the image data. This mixes local pixels with global ocean knowledge. They test on datasets for marine debris and oil spills.
Alex: Fusion at the bottleneck balances details with big-picture smarts?
Sam: Yes. On oil spills, their best setup beat the prior top model by a clear margin in boundary accuracy.
Alex: And it works much better than land-biased models on balanced measures. Does it hold for all ocean tasks?
Sam: Gains are strongest for picking out debris or pollutants, where sorting water from objects matters. For depth estimation, it stays competitive, but results vary by water type and data amount. Benefits depend on the task and labels available.
Alex: Makes sense with scarce labels. Proof those physical add-ons really drive the gains?
Sam: On debris data, a plain MAE got lower accuracy separating trash from sea clutter, while OceanMAE improved notably. The boost is clear but metric-specific.
Alex: Physically informed training lets it read water like a marine scientist. No more seeing ocean as noisy land.
Sam: Exactly. Trained on 100,000 unlabeled patches worldwide, it builds ocean-specific smarts to handle label shortages. Freezing the codes works well for segmentation. This sets up reliable monitoring without constant relabeling.
Alex: A solid step for actionable ocean insights from satellites. Thanks, Sam—always clears the waves.
Sam: My pleasure, Alex.