MemoriEase: An Interactive Lifelog Retrieval System for LSC’23

Abstract

Lifelogging is an activity of recording all events that happen in the daily life of an individual. The events can contain images, audio, health index, etc which are collected through various devices such as wearable cameras, smartwatches, and other digital services. Exploiting lifelog data can bring significant benefits for lifeloggers from creating personalized healthcare plans to retrieving events in the past. In recent years, there has been a growing development of interactive lifelog retrieval systems, such as competitors at the annual Lifelog Search Challenge (LSC), to assist lifeloggers in finding events from the past. This paper introduces an interactive lifelog image retrieval called MemoriEase for the LSC’23 challenge. This system combines concept-based and embedding-based retrieval approaches to answer accurate images for LSC’23 queries. This system uses BLIP for the embedding-based retrieval approach to reduce the semantic gap between images and text queries. The concept-based retrieval approach uses full-text search in Elasticsearch to retrieve images having visual concepts similar to keywords in the query. Regarding the user interface, we make it as simple as possible to make novices users can use it with only a small effort. This is the first version of MemoriEase and we expect this can help users perform well in the LSC’23 competition.