Removing Clouds and Recovering Ground Observations in Satellite Image Sequences via Temporally Contiguous Robust Matrix Completion

Jialei Wang, Peder A. Olsen, Andrew R. Conn, Aurelie C. Lozano; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2754-2763

We consider the problem of removing and replacing clouds in satellite image sequences, which has a wide range of applications in remote sensing. Our approach first detects and removes the cloud-contaminated part of the image sequences, then recovers the missing scenes from the clean parts by the proposed "TECROMAC" (TEmporally Contiguous RObust MAtrix Completion) objective. The objective function balances temporal smoothness with a low rank solution while staying close to the original observations. The matrix where the rows are pixels and columns are the days of the image has low-rank because the pixels reflect land-types such as vegetation, roads and lakes and there are relatively few of these. We provide efficient optimization algorithms for TECROMAC, so we can run on images containing millions of pixels. Empirical results on real satellite image sequences as well as simulated data demonstrate that our approach is able to recover underlying images from heavily cloud-contaminated observations.