Building Usage Classification in Indian Cities: Utilizing Street View Images and Object Detection Models

Yamini Sahu, Vasu Dhull, Satyajeet Shashwat, Vaibhav Kumar; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2024, pp. 664-678

Urban land use maps at the building instance level are crucial geo-information for many applications, yet they are challenging to obtain. Land-use classification based on spaceborne or aerial remote sensing images has been extensively studied over the last few decades. Such classification is usually a patch-wise or pixel-wise labeling over the whole image. However, for many applications, such as urban population density estimation or urban utility mapping, a classification map based on individual buildings (residential, commercial, mixed-type, and religious) is much more informative. Nonetheless, this type of semantic classification still poses fundamental challenges, such as retrieving fine boundaries of individual buildings. Street view images (SVI) are highly suited for predicting building functions because building facades provide clear hints. Although SVIs are used in many studies, their application in generating building usage maps is limited. Furthermore, their application to Indian cities remains void. In this paper, we propose a comprehensive framework for classifying the functionality of individual buildings. Our method leverages the YOLOs model and utilizes SVIs, including those from Google Street View and Open- StreetMap. Geographic information is employed to mask individual buildings and associate them with the corresponding SVIs. We created our own dataset in Indian cities for training and evaluating our model.