How Much You Ate? Food Portion Estimation on Spoons

Aaryam Sharma, Chris Czarnecki, Yuhao Chen, Pengcheng Xi, Linlin Xu, Alexander Wong; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3761-3770

Monitoring dietary intake is a crucial aspect of promoting healthy living. In recent years advances in computer vision technology have facilitated dietary intake monitoring through the use of images and depth cameras. However the current state-of-the-art image-based food portion estimation algorithms assume that users take images of their meals one or two times which can be inconvenient and fail to capture food items that are not visible from a top-down perspective such as ingredients submerged in a stew. To address these limitations we introduce an innovative solution that utilizes stationary user-facing cameras to track food items on utensils not requiring any change of camera perspective after installation. The shallow depth of utensils provides a more favorable angle for capturing food items and tracking them on the utensil's surface offers a significantly more accurate estimation of dietary intake without the need for post-meal image capture. The system is reliable for estimation of nutritional content of liquid-solid heterogeneous mixtures such as soups and stews. Through a series of experiments we demonstrate the exceptional potential of our method as a non-invasive user-friendly and highly accurate dietary intake monitoring tool.