Robust Recursive Fusion of Multi-Resolution Multispectral Images with Location-Aware Neural Network

Multi-resolution image fusion has been studied for years to solve the trade-off between temporal and spatial resolution in remote sensing instruments and has been widely applied to detect and monitor natural phenomena like floods. Despite the considerable research on this topic, the mitigation of the influence of outliers, such as cloud and shadow miscorrections, on satellite image fusion has not been fully developed. Moreover, strategies that integrate robustness, recursive operation and learned models are missing. In this paper, we design a robust recursive image fusion framework leveraging a location-aware neural network (NN) to model image dynamics. Outliers are modeled by representing the probability of contamination of a given pixel and band. An NN model trained on a small dataset provides accurate predictions of stochastic image time evolution, which improves both the accuracy and robustness of the method. A recursive solution is proposed to estimate high-resolution images by using a Bayesian variational inference framework. Experiments fusing images from the Landsat 8 and MODIS instruments show that the proposed method generally reduces the root mean square error (RMSE) and misclassification percentage of the estimated image by over (Formula presented.) without cloud cover and over (Formula presented.) with cloud cover compared with the benchmark KF algorithm. This indicates that the proposed approach is significantly more robust against cloud cover, without losing performance when no clouds are present.