Robust Portrait Image Matting and Depth-of-Field Synthesis via Multiplane Images

Zhefan Rao, Tianjia Zhang, Yuen Fui Lau, Qifeng Chen; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 9589-9599

High-quality portrait photography has become an essential function in our daily lives. However due to the limited aperture and focal length of a smartphone camera images captured by a smartphone cannot match the same level of bokeh effect by a digital single-lens reflex camera. A typical solution on a smartphone is to simulate out-of-focus effects from an all-in-focus image where the key is robust depth estimation and portrait matting. To achieve this we propose a multi-stage multi-branch matting network to estimate a strand-level portrait alpha mask which is then used to refine the coarse depth map from the pre-trained model. Combining the input portrait image with the estimated depth map and alpha mask we propose a learning-free optimization mechanism to construct a multi-plane image (MPI) representation for depth-of-field synthesis. The MPI consists of multiple layers of disk-blurred images with kernel size proportional to the absolute depth distance to the focus layer. Then a depth-aware blurring process is applied to enforce the bokeh effect. Besides each MPI layer has an alpha channel controlling the visibility according to the corresponding depth. Finally an image with bokeh is rendered by compositing all MPI layers. We conduct comprehensive experiments to evaluate our method which demonstrates that our method can generate more accurate alpha masks and more realistic images with bokeh compared to prior work.