Enhancing Human-Robot Collaborative Object Search Through Human Behavior Observation and Dialog

Takahiro Ishii, Jun Miura, Kotaro Hayashi; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 1849-1856

Human-robot collaborative object search entails joint efforts between a human and a robot operating in the same environment to locate a target object. Achieving efficient collaboration requires to avoiding duplicated search areas and sharing the space appropriately. This paper introduces a method for determining the robot's search strategy through the observation of human search behavior and engaging in dialog with the human. The behavior is determined by comparing estimated travel times for different behaviors with the actual elapsed time. When faced with multiple potential behaviors, the robot selectively generates informative queries to resolve ambiguities and obtain valuable responses. This occasional dialog activation serves as a crucial factor in achieving an efficient collaborative object search. Through collaborative experiments with real human subjects conducted in a virtual environment, we validate the effectiveness of our proposed method in reducing overlapped search areas and minimizing the time required to locate target objects.