A Unified Model for Near and Remote Sensing

Scott Workman, Menghua Zhai, David J. Crandall, Nathan Jacobs; The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2688-2697


We propose a novel convolutional neural network architecture for estimating geospatial functions such as population density, land cover, or land use. In our approach, we combine overhead and ground-level images in an end-to-end trainable neural network, which uses kernel regression and density estimation to convert features extracted from the ground-level images into a dense feature map. The output of this network is a dense estimate of the geospatial function in the form of a pixel-level labeling of the overhead image. To evaluate our approach, we created a large dataset of overhead and ground-level images from a major urban area with three sets of labels: land use, building function, and building age. We find that our approach is more accurate for all tasks, in some cases dramatically so.

Related Material

[pdf] [Supp] [arXiv]
author = {Workman, Scott and Zhai, Menghua and Crandall, David J. and Jacobs, Nathan},
title = {A Unified Model for Near and Remote Sensing},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}