GeoCR: Learning a Generalist Cloud Removal Prior from Heterogeneous Observations

GeoCR. Learning a Generalist Cloud Removal Prior from Heterogeneous Observations. By Jeonghyeok Do and one other, from KAIST. Published on the twenty-sixth of September. Its code repository is worked out: about eighty source files, with a readme and a licence.

Cloud removal methods are typically specialized: built for one dataset and one input configuration. That limits reuse across sensors, across spectral bands, and across observation settings. A model shaped around one combination of inputs has little to offer when the next dataset arrives with a different one, and so each setting tends to get a model of its own.

GeoCR is a generalist model that brings these configurations into a single network. It handles cloud removal from RGB alone and from multispectral input, from a single cloudy observation or several taken over time, and with SAR guidance as an option. The aim is one learned cloud removal prior that serves all of these cases.

The key is a shared latent interface. A pretrained RGB autoencoder is extended with compact input and output stems that accommodate different spectral and sensing domains, while its encoder and decoder trunks stay frozen. On top of that interface, a single flow transformer jointly models clean RGB latents and non-RGB latents. It is conditioned on separate streams for the cloudy observations, and on SAR tokens when those are available.

Pretraining is joint, over the training splits of ten datasets that together hold roughly eight hundred eighty-three thousand cloud-free target images. From that heterogeneous mix, GeoCR learns a shared cloud removal prior. The same pretrained checkpoint can then be used directly, with no dataset-specific fine-tuning, or adapted efficiently through low-rank adaptation, known as LoRA.

Evaluation uses the test splits of the contributing datasets, in full-band and in RGB-only settings, against general image restoration methods as well as dedicated cloud removal methods. GeoCR achieves the best FID and DISTS on full-band SEN12MS-CR and Sen2_MTC_New, and on RGB-only CUHK-CR2, outperforming the existing models it is compared with.

In place of a new specialist for each sensor combination, the result is one reusable generative model: a frozen backbone, light stems at its edges, and a single transformer in the middle that learns from every dataset at once. The authors present the results as evidence that such a model is effective across diverse settings, from a single checkpoint that can be used as it is or lightly adapted.