The recipe flow graph is a directed acyclic graph for representing the procedure of the cooking recipe, and it is expected to contribute to the research to understand the whole meaning of the recipe text precisely. The recent rapid increase in health awareness has generated a large amount of user-generated recipe texts and corresponding recipe flow graphs in social networking services. In this research, to alleviate the problem of huge memory consumption and long computation time for handling a large number of recipe flow graphs, we propose to utilize succinct data structures to store the databases of recipe flow graphs. The effectiveness of our implementations of flow graph databases in two types of succinct data structures for trees and graphs was confirmed by the preliminary experiments using real recipe flow graphs.
{"title":"Representation of Recipe Flow Graphs in Succinct Data Structures","authors":"Takuya Namiki, Tomonobu Ozaki","doi":"10.1145/3326458.3326930","DOIUrl":"https://doi.org/10.1145/3326458.3326930","url":null,"abstract":"The recipe flow graph is a directed acyclic graph for representing the procedure of the cooking recipe, and it is expected to contribute to the research to understand the whole meaning of the recipe text precisely. The recent rapid increase in health awareness has generated a large amount of user-generated recipe texts and corresponding recipe flow graphs in social networking services. In this research, to alleviate the problem of huge memory consumption and long computation time for handling a large number of recipe flow graphs, we propose to utilize succinct data structures to store the databases of recipe flow graphs. The effectiveness of our implementations of flow graph databases in two types of succinct data structures for trees and graphs was confirmed by the preliminary experiments using real recipe flow graphs.","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125422078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper conducts the cooking sound analysis for understanding cooking activities toward cooking support systems. Although there have been attempts to use images, accelerations or temperature sensors to understand cooking behavior, only limited studies have been conducted using acoustic signals. In this study, a data set was newly constructed by recording sounds generated from actual cooking processes and cooking state estimation was carried out based on the constructed data set. Two types of features, which are derived from mel-frequency cepstral coefficients (MFCC) analysis and non-negative matrix factorization (NMF), are examined, and the performance of classification based on Gaussian mixture models (GMM) incorporating these features is investigated.
{"title":"Cooking State Recognition based on Acoustic Event Detection","authors":"Yusaku Korematsu, D. Saito, N. Minematsu","doi":"10.1145/3326458.3326932","DOIUrl":"https://doi.org/10.1145/3326458.3326932","url":null,"abstract":"This paper conducts the cooking sound analysis for understanding cooking activities toward cooking support systems. Although there have been attempts to use images, accelerations or temperature sensors to understand cooking behavior, only limited studies have been conducted using acoustic signals. In this study, a data set was newly constructed by recording sounds generated from actual cooking processes and cooking state estimation was carried out based on the constructed data set. Two types of features, which are derived from mel-frequency cepstral coefficients (MFCC) analysis and non-negative matrix factorization (NMF), are examined, and the performance of classification based on Gaussian mixture models (GMM) incorporating these features is investigated.","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129591621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yunan Wang, Jingjing Chen, C. Ngo, Tat-Seng Chua, Wanli Zuo, Zhaoyan Ming
Mix dish recognition, whose goal is to identify each of the dish type presented on one plate, is generally regarded as a difficult problem. The major challenge of this problem is that different dishes presented in one plate may overlap with each other and there may be no clear boundaries among them. Therefore, labeling the bounding box of each dish type is difficult and not necessarily leading to good results. This paper studies the problem from the perspective of multi-label learning. Specially, we propose to perform dish recognition on region level with multiple granularities. For experimental purpose, we collect two mix dish datasets: mixed economic rice and economic beehoon. The experimental results on these two datasets demonstrate the effectiveness of the proposed region-level multi-label learning methods.
{"title":"Mixed Dish Recognition through Multi-Label Learning","authors":"Yunan Wang, Jingjing Chen, C. Ngo, Tat-Seng Chua, Wanli Zuo, Zhaoyan Ming","doi":"10.1145/3326458.3326929","DOIUrl":"https://doi.org/10.1145/3326458.3326929","url":null,"abstract":"Mix dish recognition, whose goal is to identify each of the dish type presented on one plate, is generally regarded as a difficult problem. The major challenge of this problem is that different dishes presented in one plate may overlap with each other and there may be no clear boundaries among them. Therefore, labeling the bounding box of each dish type is difficult and not necessarily leading to good results. This paper studies the problem from the perspective of multi-label learning. Specially, we propose to perform dish recognition on region level with multiple granularities. For experimental purpose, we collect two mix dish datasets: mixed economic rice and economic beehoon. The experimental results on these two datasets demonstrate the effectiveness of the proposed region-level multi-label learning methods.","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125970127","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recent rapid increase in health awareness is producing a large amount of user generated cooking recipes in online community sites. For the effective use of such cooking recipes, it is necessary not only to understand their meaning but also to extract certain structures among them, by paying attention to cooking steps in detail. One of the most precise representations of cooking procedure is the recipe flow graph that is a directed acyclic graph having recipe terms in vertices and their relations in edges. In this paper, as a preliminary attempt for acquiring a new vector representation reflecting various aspects of cooking procedures, we propose a simple method to learn a distributed representation of recipe flow graphs using frequent fragments of cooking procedures. Experiments using real world dataset are conducted to compare the distributed representation of recipe flow graphs and that of recipe texts. As a result, we confirm that the proposed representation can capture the difference among recipes well, and it is suitable for the classification tasks.
{"title":"Learning Distributed Representation of Recipe Flow Graphs via Frequent Subgraphs","authors":"Akari Ninomiya, Tomonobu Ozaki","doi":"10.1145/3326458.3326931","DOIUrl":"https://doi.org/10.1145/3326458.3326931","url":null,"abstract":"Recent rapid increase in health awareness is producing a large amount of user generated cooking recipes in online community sites. For the effective use of such cooking recipes, it is necessary not only to understand their meaning but also to extract certain structures among them, by paying attention to cooking steps in detail. One of the most precise representations of cooking procedure is the recipe flow graph that is a directed acyclic graph having recipe terms in vertices and their relations in edges. In this paper, as a preliminary attempt for acquiring a new vector representation reflecting various aspects of cooking procedures, we propose a simple method to learn a distributed representation of recipe flow graphs using frequent fragments of cooking procedures. Experiments using real world dataset are conducted to compare the distributed representation of recipe flow graphs and that of recipe texts. As a result, we confirm that the proposed representation can capture the difference among recipes well, and it is suitable for the classification tasks.","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125682513","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Taichi Nishimura, Atsushi Hashimoto, Yoko Yamakata, Shinsuke Mori
In cooking procedure instruction, text format plays an important role in conveying quantitative information accurately, such as time and quantity. On the other hand, image format can smoothly convey qualitative information (e.g., the target food state of a procedure) at a glance. Our goal is to produce multimedia recipes, which have texts and corresponding pictures, for chefs to better understand the procedures. The system takes a procedural text and its unedited execution video as the input and outputs selected frames for instructions in the text. We assume that a frame suits to an instruction when they share key objects. Under this assumption, we extract the information of key objects using named entity recognizer from the text and object detection from the frame, and we convert them into feature vectors and calculate their cosine similarity. To enhance the measurement, we also calculate the scene importance based on the latest changes in object appearance, and aggregate it to the cosine similarity. Finally we align the instruction sequence and the frame sequence using the Viterbi algorithm referring to this suitability and get the frame selection for each instruction. We implemented our method and tested it on a dataset consisting of text recipes and their execution videos. In the experiments we compared the automatic alignment results with those by human annotators. The precision, recall, and F-measure showed that the proposed approach made a steady improvement in this challenging problem of selecting pictures from an unedited video.
{"title":"Frame Selection for Producing Recipe with Pictures from an Execution Video of a Recipe","authors":"Taichi Nishimura, Atsushi Hashimoto, Yoko Yamakata, Shinsuke Mori","doi":"10.1145/3326458.3326928","DOIUrl":"https://doi.org/10.1145/3326458.3326928","url":null,"abstract":"In cooking procedure instruction, text format plays an important role in conveying quantitative information accurately, such as time and quantity. On the other hand, image format can smoothly convey qualitative information (e.g., the target food state of a procedure) at a glance. Our goal is to produce multimedia recipes, which have texts and corresponding pictures, for chefs to better understand the procedures. The system takes a procedural text and its unedited execution video as the input and outputs selected frames for instructions in the text. We assume that a frame suits to an instruction when they share key objects. Under this assumption, we extract the information of key objects using named entity recognizer from the text and object detection from the frame, and we convert them into feature vectors and calculate their cosine similarity. To enhance the measurement, we also calculate the scene importance based on the latest changes in object appearance, and aggregate it to the cosine similarity. Finally we align the instruction sequence and the frame sequence using the Viterbi algorithm referring to this suitability and get the frame selection for each instruction. We implemented our method and tested it on a dataset consisting of text recipes and their execution videos. In the experiments we compared the automatic alignment results with those by human annotators. The precision, recall, and F-measure showed that the proposed approach made a steady improvement in this challenging problem of selecting pictures from an unedited video.","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"109 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116375467","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Session details: Session 1: Long Oral Session","authors":"Keisuke Doman","doi":"10.1145/3340225","DOIUrl":"https://doi.org/10.1145/3340225","url":null,"abstract":"","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128713401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Having foods in a fine nutritional balance is important for both physical and mental health. Due to the change in lifestyle, people often have lunch and dinner at restaurants and buy pre-cooked foods at supermarkets. Having only those foods might cause off-balance of nutrition. This paper proposes a food recommendation system for meals-out to support the nutrient balance. The users may input the foods that they have had. The proposed system calculates the intake of nutrients and energies, and then recommends foods for meals-out that support to adjust the nutrient balance. We conducted evaluation experiments with the proposed system. It was confirmed that the proposed system could support users to improve the balance of nutrient intake.
{"title":"Foods Recommendation System for Meals-out in Nutrient Balance","authors":"Takumi Ohata, Yoko Nishihara, Ryosuke Yamanishi","doi":"10.1145/3326458.3326926","DOIUrl":"https://doi.org/10.1145/3326458.3326926","url":null,"abstract":"Having foods in a fine nutritional balance is important for both physical and mental health. Due to the change in lifestyle, people often have lunch and dinner at restaurants and buy pre-cooked foods at supermarkets. Having only those foods might cause off-balance of nutrition. This paper proposes a food recommendation system for meals-out to support the nutrient balance. The users may input the foods that they have had. The proposed system calculates the intake of nutrients and energies, and then recommends foods for meals-out that support to adjust the nutrient balance. We conducted evaluation experiments with the proposed system. It was confirmed that the proposed system could support users to improve the balance of nutrient intake.","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130128240","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Session details: Session 2: Short Oral Session","authors":"Jing-Jing Chen","doi":"10.1145/3340226","DOIUrl":"https://doi.org/10.1145/3340226","url":null,"abstract":"","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127580538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This research aims to develop a data augmentation framework in order to improve the attractiveness estimation accuracy for food photography. Machine learning-based methods require numerous food images accompanied with their attractiveness for learning the relationship between image features and the attractiveness. To efficiently obtain such food images, we apply data augmentation; the proposed method applies four kinds of image transformations: rotation, scaling, shifting, and random noise addition to the original images accompanied with their attractiveness. The key idea here is to apply the image transformations within a parameter space in which the attractiveness of the transformed image can be regarded as the same as that of the original one. By this way, we can obtain a large number of images accompanied with their attractiveness without any additional subjective experiments. Experimental results showed the effectiveness of the proposed method framework.
{"title":"Application of Data Augmentation for Accurate Attractiveness Estimation for Food Photography","authors":"Tatsumi Hattori, Keisuke Doman, I. Ide, Y. Mekada","doi":"10.1145/3326458.3326927","DOIUrl":"https://doi.org/10.1145/3326458.3326927","url":null,"abstract":"This research aims to develop a data augmentation framework in order to improve the attractiveness estimation accuracy for food photography. Machine learning-based methods require numerous food images accompanied with their attractiveness for learning the relationship between image features and the attractiveness. To efficiently obtain such food images, we apply data augmentation; the proposed method applies four kinds of image transformations: rotation, scaling, shifting, and random noise addition to the original images accompanied with their attractiveness. The key idea here is to apply the image transformations within a parameter space in which the attractiveness of the transformed image can be regarded as the same as that of the original one. By this way, we can obtain a large number of images accompanied with their attractiveness without any additional subjective experiments. Experimental results showed the effectiveness of the proposed method framework.","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128008947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","authors":"","doi":"10.1145/3326458","DOIUrl":"https://doi.org/10.1145/3326458","url":null,"abstract":"","PeriodicalId":184771,"journal":{"name":"Proceedings of the 11th Workshop on Multimedia for Cooking and Eating Activities","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116917010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：