Online Earth monitoring with multi-modal Satellite Image Time Series

Iris Dumeur1,2, Aitor Aitola2, Jérémy Anger1,2, Gabriele Facciolo2,3
1 Safran.AI Paris France
2 Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, 91190, Gif-sur-Yvette, France
3 Institut Universitaire de France
📄 Paper (preprint) 💻 Model weights (available in October) 📊 Dataset Gold-panning 📊 Dataset Building Density (published in October) 📊 Dataset Land Cover (published in October)

Abstract

While online Earth monitoring with Satellite Image Time Series (SITS) is essential for tracking rapid anthropogenic changes, existing remote sensing foundation models remain unsuited for streaming SITS. We introduce IRMA, the first foundation model tailored for online land monitoring. IRMA leverages a novel dual-form framework that unifies parallelized multi-modal pre-training (Sentinel-1/2) with an efficient recurrent inference mechanism. Our self-supervised objective produces latent representations that simultaneously maintain temporal stability against seasonal variations and sensitivity to permanent land modifications. To evaluate our method, we present MELBA, a multi-temporal benchmark spanning land-cover, building density, and gold-panning tasks. Experimental results show that IRMA achieves competitive performance with state-of-the-art baselines in the mono-modal setting while using fewer parameters. Furthermore, qualitative analyses suggest that IRMA effectively fuses multi-modal observations, thus providing representations relevant for online monitoring.

Key Contributions

  • IRMA: A novel foundation model with a dual framework (supporting both parallel pre-training and recurrent inference) necessary for online monitoring. IRMA is pre-trained on S1 and S2 SITS via a self-supervised objective specifically designed to produce latent representations that are invariant to seasonal, predictable environmental variations.
  • MELBA: A Multi-Temporal Evaluation BenchmArk. Three different datasets spanning distinct real-world downstream tasks: gold-panning detection, urban building density estimation, and land-cover segmentation. These datasets will be publicly released upon publication.

Method

IRMA multi-modal architecture

IRMA's multi-modal architecture: modality-specific Sentinel-1 and Sentinel-2 embeddings are fused through a Multi-modal Temporal Fusion Layer.

IRMA self-supervised learning objective

Self-supervised pre-training objective combining SITS forecasting and latent (JEPA-style) predictions across two branches.

IRMA's temporal fusion layer uses a hybrid dual-form attention: half of the heads apply standard Linear Attention for cross-modal fusion, while the other half apply a temporal cosine-reweighted attention (TimeCosAttention) that encodes relative acquisition dates. This design supports both parallelized pre-training and O(1) recurrent inference per new acquisition, compared to O(T) for conventional attention-based SITS foundation models.

MELBA benchmark

Map of the MELBA benchmark locations

Geographic distribution of the MELBA benchmark sites: Land-Cover, Building Regression, and Gold-panning.

Name Source Task Nsite Years Location
Gold Panning (GP) OAM Binary segmentation 364 2018–2019 French Guyana
Building Density (BD) SpaceNet-7 Regression 156 2017–2020 World cities
Land-Cover (LC) DynamicEarthNet Multi-class segmentation 135 2018–2019 World

Description of the three MELBA online monitoring tasks.

Results

On mono-modal (Sentinel-2 only) downstream tasks, IRMA matches or exceeds larger foundation model baselines while using up to 7× fewer parameters and offering O(1) recurrent inference.

IRMA building density predictions over time

IRMA's pre-trained, frozen backbone on the building density task: predictions stay stable over time and adapt as soon as new construction appears in the imagery — including early detection from Sentinel-1 before it is visible in Sentinel-2.

Citation

@inproceedings{
dumeur2026dualform,
title={Dual-Form Foundation Model for Online Earth Monitoring},
author={Iris Dumeur and Aitor Artola and J{\'e}r{\'e}my Anger and Gabriele Facciolo},
booktitle={ECCV 2026 Workshop TerraBytes II},
year={2026},
url={https://openreview.net/forum?id=2fBXXMyr53}
}
}

Acknowledgments

This work was financed by the Agence Innovation Défense (AID), within the framework of the Dual Innovation Support Scheme (RAPID - Régime d'APpui à l'Innovation Duale), for the project 'DETEVENT' (Agreement No. 2024 29 0970).

This work was granted access to the HPC resources of IDRIS under the allocations 2025-AD011016513 and 2025-AD011012453R4 made by GENCI.

Sentinel-1 and Sentinel-2 data is from Copernicus. This paper contains modified Copernicus Sentinel data.