Eliminating the Observer Effect: Shadow Removal in Orthomosaics of the Road Network

Supannee Tanathong, William A. P. Smith, Stephen Remde; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 262-269

High resolution images of the road surface can be obtained using a vehicle equipped with a camera oriented towards the road surface. These images can be stitched into an orthomosaic (i.e. a mosaiced image approximating an orthographic view) providing a virtual top down view of the road network. However, the vehicle casts a shadow onto the road surface that is sometimes visible in the captured images. This causes large artefacts in the stitched orthomosaic. In this paper, we propose a model-based solution to this problem. We capture a 3D model of the vehicle, transform it to a canonical pose and use it to predict shadow masks by ray casting from the sun direction. Shadow masks are precomputed, stored in a look up table and used to generate per-pixel weights for stitching. We integrate this approach into a pipeline for pose estimation and gradient domain stitching that we show is capable of producing shadow-free, high quality orthomosaics from uncontrolled, real world datasets.