MRGAN: Multi-Rooted 3D Shape Representation Learning With Unsupervised Part Disentanglement

Rinon Gal, Amit Bermano, Hao Zhang, Daniel Cohen-Or; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 2039-2048

Abstract


We introduce MRGAN, or multi-rooted GAN, the first generative adversarial network to learn a part-disentangled 3D shape representation without any part supervision. The network fuses multiple branches of tree-structured graph convolution layers which produce point clouds in a controllable manner. Specifically, each branch learns to grow a different shape part, offering control over the shape generation at the part level. Our network encourages disentangled generation of semantic parts via two key ingredients: a root-mixing training strategy which helps decorrelate the different branches to facilitate disentanglement, and a set of loss terms designed with part disentanglement and shape semantics in mind. Of these, a novel convexity loss incentivizes the generation of parts that are more convex, as semantic parts tend to be. In addition, a root-dropping loss further ensures that each root seeds a single part, preventing the degeneration or over-growth of the point-producing branches. We evaluate the performance of our network on a number of 3D shape classes, and offer qualitative and quantitative comparisons to previous works and baseline approaches. We demonstrate the controllability offered by our part-disentangled representation through two applications for shape modeling: part mixing and individual part variation, without receiving segmented shapes as input.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Gal_2021_ICCV, author = {Gal, Rinon and Bermano, Amit and Zhang, Hao and Cohen-Or, Daniel}, title = {MRGAN: Multi-Rooted 3D Shape Representation Learning With Unsupervised Part Disentanglement}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops}, month = {October}, year = {2021}, pages = {2039-2048} }