FIRST - Flexible Interactive Retrieval SysTem for Visual Lifelog Exploration at LSC 2020

Proceedings of the Third Annual Workshop on Lifelog Search Challenge Pub Date : 2020-06-09 DOI:10.1145/3379172.3391726

M. Tran, Thanh-An Nguyen, Quoc-Cuong Tran, Mai-Khiem Tran, Khanh-Duy Nguyen, Van-Tu Ninh, Tu-Khiem Le, Hoang-Phuc Trang-Trung, Hoang-Anh Le, Hai-Dang Nguyen, Trong-Le Do, Viet-Khoa Vo-Ho, C. Gurrin

{"title":"FIRST - Flexible Interactive Retrieval SysTem for Visual Lifelog Exploration at LSC 2020","authors":"M. Tran, Thanh-An Nguyen, Quoc-Cuong Tran, Mai-Khiem Tran, Khanh-Duy Nguyen, Van-Tu Ninh, Tu-Khiem Le, Hoang-Phuc Trang-Trung, Hoang-Anh Le, Hai-Dang Nguyen, Trong-Le Do, Viet-Khoa Vo-Ho, C. Gurrin","doi":"10.1145/3379172.3391726","DOIUrl":null,"url":null,"abstract":"Lifelog can provide useful insights of our daily activities. It is essential to provide a flexible way for users to retrieve certain events or moments of interest, corresponding to a wide variation of query types. This motivates us to develop FIRST, a Flexible Interactive Retrieval SysTem, to help users to combine or integrate various query components in a flexible manner to handle different query scenarios, such as visual clustering data based on color histogram, visual similarity, GPS location, or scene attributes. We also employ personalized concept detection and image captioning to enhance image understanding from visual lifelog data, and develop an autoencoder-like approach for query text and image feature mapping. Furthermore, we refine the user interface of the retrieval system to better assist users in query expansion and verifying sequential events in a flexible temporal resolution to control the navigation speed through sequences of images.","PeriodicalId":340585,"journal":{"name":"Proceedings of the Third Annual Workshop on Lifelog Search Challenge","volume":"289 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2020-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"17","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the Third Annual Workshop on Lifelog Search Challenge","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3379172.3391726","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 17

Abstract

Lifelog can provide useful insights of our daily activities. It is essential to provide a flexible way for users to retrieve certain events or moments of interest, corresponding to a wide variation of query types. This motivates us to develop FIRST, a Flexible Interactive Retrieval SysTem, to help users to combine or integrate various query components in a flexible manner to handle different query scenarios, such as visual clustering data based on color histogram, visual similarity, GPS location, or scene attributes. We also employ personalized concept detection and image captioning to enhance image understanding from visual lifelog data, and develop an autoencoder-like approach for query text and image feature mapping. Furthermore, we refine the user interface of the retrieval system to better assist users in query expansion and verifying sequential events in a flexible temporal resolution to control the navigation speed through sequences of images.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

LSC 2020可视化生活日志探索的柔性交互检索系统

生活日志可以为我们的日常活动提供有用的见解。必须为用户提供一种灵活的方式来检索特定的事件或感兴趣的时刻，以适应各种查询类型。这促使我们开发FIRST，一个灵活的交互检索系统，帮助用户以灵活的方式组合或集成各种查询组件，以处理不同的查询场景，如基于颜色直方图、视觉相似性、GPS位置或场景属性的视觉聚类数据。我们还采用个性化的概念检测和图像字幕来增强对视觉生活日志数据的图像理解，并开发了一种类似于自动编码器的方法来查询文本和图像特征映射。此外，我们改进了检索系统的用户界面，以更好地帮助用户在灵活的时间分辨率下扩展查询和验证顺序事件，从而通过图像序列控制导航速度。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Proceedings of the Third Annual Workshop on Lifelog Search Challenge

自引率

0.00%

发文量