Reconstructing Creative Lego Models

George Tattersall, Dizhong Zhu, William A. P. Smith, Sebastian Deterding, Patrik Huber; Proceedings of the Asian Conference on Computer Vision (ACCV), 2020

Lego is one of the most successful toys in the world. Being able to scan, analyse and reconstruct Lego models has many applications, for example studying creativity. In this paper, from a set of 2D input images, we create a monolithic mesh, representing a physical 3D Lego model as input, and split it in to its known components such that the output of the program can be used to completely reconstruct the input model, brick for brick. We present a novel, fully automatic pipeline to reconstruct Lego models in 3D from 2D images; A-DBSCAN, an angular variant of DBSCAN, useful for grouping both parallel and anti-parallel vectors; and a method for reducing the problem of non-Manhattan reconstruction to that of Manhattan reconstruction. We evaluate the presented approach both qualitatively and quantitatively on a set of Lego duck models from a public data set, and show that the algorithm is able to identify and reconstruct the Lego models successfully.