When diary meets lifelog video

As the increasing quantities of personal data is collected by individuals, the number of lifelog video is increasing. People make microblogging in the form of the text, later, in form of the text with pictures or videos. In this paper, a cross-media lifelog video retrieval approach is proposed to automatically match the corresponding lifelog video clip from a long lifelog video according to diary description(Fig.2). This model consists of a video captioning model and a text retrieval model. We train an encoder-decoder architecture to effectively learn video captioning by MSVD and MSR-VTT datasets. We use the similarity judgment to achieve the retrieval of the text. The similarity is measured by measuring the cosine distance between the two vectors. We experiment on some participants' lifelog videos and diaries. This approach is evaluated by investigating participants' satisfaction with results of lifelog video selected, the results show most of the testers were satisfied with the results.