Inverting Audio-Visual Simulation for Shape and Material Perception

Zhoutong Zhang, Jiajun Wu, Qiujia Li, Zhengjia Huang, Joshua B. Tenenbaum, William T. Freeman; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 2536-2538

Abstract


Humans perceive objects through both their visual appearance and the sounds they make. Given a short audio clip of objects interacting, humans can recover rich information about the materials, surface smoothness, and the quantity of objects involved. Although visual information provides cues for some of these questions, others can only be assessed with sound. For example, objects with different masses and Young's moduli may have almost identical appearance, but they make different sounds when impacted, and vice versa.

Related Material


[pdf]
[bibtex]
@InProceedings{Zhang_2018_CVPR_Workshops,
author = {Zhang, Zhoutong and Wu, Jiajun and Li, Qiujia and Huang, Zhengjia and Tenenbaum, Joshua B. and Freeman, William T.},
title = {Inverting Audio-Visual Simulation for Shape and Material Perception},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2018}
}