Remote assistance using visual prompts for demented elderly in cooking

Abstract

This paper presents a smart kitchen system for supporting demented elderly cooking in a kitchen. In this system, a remote supporter provides a user with verbal and visual prompts manually. As a prompting method for this system projection should be chosen as a priority because it enables direct pointing of targets in a real scene. Even if projection is used as a priority, this system should be designed to solve three practical problems: 1) there are many areas where sharp images cannot be projected in a kitchen, 2) kitchen is a too complex environment to display visual prompts using only a single method, 3) remote supporters do not know the kitchen environment well enough. In this paper we propose a novel system overcoming the above problems as follows, respectively.1) The system first tries indirect projection of circular markers as visual prompts around target objects.2) Proper method is automatically selected from using projector, embedded monitor and illumination devices based on environmental information.3) The system automatically acquires flat surfaces to project on and their reflectance properties as the environmental information. Based on an observation of two elderly suffering from dementia, we discuss the feasibility and concrete problems in displaying visual prompt and how to solve them technically.