Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711540
Nicolas Pinto, Youssef Barhomi, David D. Cox, J. DiCarlo
Tolerance (“invariance”) to identity-preserving image variation (e.g. variation in position, scale, pose, illumination) is a fundamental problem that any visual object recognition system, biological or engineered, must solve. While standard natural image database benchmarks are useful for guiding progress in computer vision, they can fail to probe the ability of a recognition system to solve the invariance problem [23, 24, 25]. Thus, to understand which computational approaches are making progress on solving the invariance problem, we compared and contrasted a variety of state-of-the-art visual representations using synthetic recognition tasks designed to systematically probe invari-ance. We successfully re-implemented a variety of state-of-the-art visual representations and confirmed their published performance on a natural image benchmark. We here report that most of these representations perform poorly on invariant recognition, but that one representation [21] shows significant performance gains over two baseline representations. We also show how this approach can more deeply illuminate the strengths and weaknesses of different visual representations and thus guide progress on invariant object recognition.
{"title":"Comparing state-of-the-art visual features on invariant object recognition tasks","authors":"Nicolas Pinto, Youssef Barhomi, David D. Cox, J. DiCarlo","doi":"10.1109/WACV.2011.5711540","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711540","url":null,"abstract":"Tolerance (“invariance”) to identity-preserving image variation (e.g. variation in position, scale, pose, illumination) is a fundamental problem that any visual object recognition system, biological or engineered, must solve. While standard natural image database benchmarks are useful for guiding progress in computer vision, they can fail to probe the ability of a recognition system to solve the invariance problem [23, 24, 25]. Thus, to understand which computational approaches are making progress on solving the invariance problem, we compared and contrasted a variety of state-of-the-art visual representations using synthetic recognition tasks designed to systematically probe invari-ance. We successfully re-implemented a variety of state-of-the-art visual representations and confirmed their published performance on a natural image benchmark. We here report that most of these representations perform poorly on invariant recognition, but that one representation [21] shows significant performance gains over two baseline representations. We also show how this approach can more deeply illuminate the strengths and weaknesses of different visual representations and thus guide progress on invariant object recognition.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"98 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122657451","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711568
L. Schwarz, D. Mateus, Joé Lallemand, Nassir Navab
In this paper, we propose a method for detection and tracking of multiple planes in sequences of Time of Flight (ToF) depth images. Our approach extends the recent J-linkage algorithm for estimation of multiple model instances in noisy data to tracking. Instead of randomly selecting plane hypotheses in every image, we propagate plane hypotheses through the sequence of images, resulting in a significant reduction of computational load in every frame. We also introduce a multi-pass scheme that allows detecting and tracking planes of varying spatial extent along with their boundaries. Our qualitative and quantitative evaluation shows that the proposed method can robustly detect planes and consistently track the hypotheses through sequences of ToF images.
{"title":"Tracking planes with Time of Flight cameras and J-linkage","authors":"L. Schwarz, D. Mateus, Joé Lallemand, Nassir Navab","doi":"10.1109/WACV.2011.5711568","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711568","url":null,"abstract":"In this paper, we propose a method for detection and tracking of multiple planes in sequences of Time of Flight (ToF) depth images. Our approach extends the recent J-linkage algorithm for estimation of multiple model instances in noisy data to tracking. Instead of randomly selecting plane hypotheses in every image, we propagate plane hypotheses through the sequence of images, resulting in a significant reduction of computational load in every frame. We also introduce a multi-pass scheme that allows detecting and tracking planes of varying spatial extent along with their boundaries. Our qualitative and quantitative evaluation shows that the proposed method can robustly detect planes and consistently track the hypotheses through sequences of ToF images.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114463324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711555
Mary Fletcher, A. Dornhaus, M. Shin
Motion and behavior analysis of social insects such as ants requires tracking many ants over time. This process is highly labor-intensive and tedious. Automatic tracking is challenging as ants often interact with one another, resulting in frequent occlusions that cause drifts in tracking. In addition, tracking many objects is computationally expensive. In this paper, we present a robust and efficient method for tracking multiple ants. We first prevent drifts by maximizing the coverage of foreground pixels at at global scale. Secondly, we improve speed by reducing markov chain length through dynamically changing the target proposal distribution for perturbed ant selection. Using a real dataset with ground truth, we demonstrate that our algorithm was able to improve the accuracy by 15% (resulting in 98% tracking accuracy) and the speed by 76%.
{"title":"Multiple ant tracking with global foreground maximization and variable target proposal distribution","authors":"Mary Fletcher, A. Dornhaus, M. Shin","doi":"10.1109/WACV.2011.5711555","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711555","url":null,"abstract":"Motion and behavior analysis of social insects such as ants requires tracking many ants over time. This process is highly labor-intensive and tedious. Automatic tracking is challenging as ants often interact with one another, resulting in frequent occlusions that cause drifts in tracking. In addition, tracking many objects is computationally expensive. In this paper, we present a robust and efficient method for tracking multiple ants. We first prevent drifts by maximizing the coverage of foreground pixels at at global scale. Secondly, we improve speed by reducing markov chain length through dynamically changing the target proposal distribution for perturbed ant selection. Using a real dataset with ground truth, we demonstrate that our algorithm was able to improve the accuracy by 15% (resulting in 98% tracking accuracy) and the speed by 76%.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121083883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711517
Hakan Bilen, Vinay P. Namboodiri, L. Gool
We address the problem of recognizing actions in reallife videos. Space-time interest point-based approaches have been widely prevalent towards solving this problem. In contrast, more spatially extended features such as regions have not been so popular. The reason is, any local region based approach requires the motion flow information for a specific region to be collated temporally. This is challenging as the local regions are deformable and not well delineated from the surroundings. In this paper we address this issue by using robust tracking of regions and we show that it is possible to obtain region descriptors for classification of actions. This paper lays the groundwork for further investigation into region based approaches. Through this paper we make the following contributions a) We advocate identification of salient regions based on motion segmentation b) We adopt a state-of-the art tracker for robust tracking of the identified regions rather than using isolated space-time blocks c) We propose optical flow based region descriptors to encode the extracted trajectories in piece-wise blocks. We demonstrate the performance of our system on real-world data sets.
{"title":"Action recognition: A region based approach","authors":"Hakan Bilen, Vinay P. Namboodiri, L. Gool","doi":"10.1109/WACV.2011.5711517","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711517","url":null,"abstract":"We address the problem of recognizing actions in reallife videos. Space-time interest point-based approaches have been widely prevalent towards solving this problem. In contrast, more spatially extended features such as regions have not been so popular. The reason is, any local region based approach requires the motion flow information for a specific region to be collated temporally. This is challenging as the local regions are deformable and not well delineated from the surroundings. In this paper we address this issue by using robust tracking of regions and we show that it is possible to obtain region descriptors for classification of actions. This paper lays the groundwork for further investigation into region based approaches. Through this paper we make the following contributions a) We advocate identification of salient regions based on motion segmentation b) We adopt a state-of-the art tracker for robust tracking of the identified regions rather than using isolated space-time blocks c) We propose optical flow based region descriptors to encode the extracted trajectories in piece-wise blocks. We demonstrate the performance of our system on real-world data sets.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116278405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711535
Zhang Tao, T. Boult
Stereo reconstruction is an important research and application area, both for general 3D reconstruction and for operations like robotic navigation and remote sensing. This paper addresses the determination of parameters for a stereo system to optimize/minimize 3D reconstruction errors. Previous work on error analysis in stereo reconstruction optimized error in disparity space which led to the erroneous conclusion that, ignoring matching errors, errors decrease when the baseline goes to infinity. In this paper, we derive the first formal error model based on the more realistic “point-of-closest-approach” ray model used in modern stereo systems. We then show this results in finite optimal baseline that minimizes reconstruction errors in all three world directions. We also show why previous oversimplified error analysis results in infinite baselines. We derive the mathematical relationship between the error variances and the stereo system parameters. In our analysis, we consider the situations where errors exist in only one camera as well as errors in both cameras. We have derived the results for both parallel and verged systems, though only the simpler models are presented algebraically herein. The paper includes simulations to highlight the results and validate the approximations in the error propagation. The results should allow stereo system designers, or those using motion-stereo, to improve their system.
{"title":"Realistic stereo error models and finite optimal stereo baselines","authors":"Zhang Tao, T. Boult","doi":"10.1109/WACV.2011.5711535","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711535","url":null,"abstract":"Stereo reconstruction is an important research and application area, both for general 3D reconstruction and for operations like robotic navigation and remote sensing. This paper addresses the determination of parameters for a stereo system to optimize/minimize 3D reconstruction errors. Previous work on error analysis in stereo reconstruction optimized error in disparity space which led to the erroneous conclusion that, ignoring matching errors, errors decrease when the baseline goes to infinity. In this paper, we derive the first formal error model based on the more realistic “point-of-closest-approach” ray model used in modern stereo systems. We then show this results in finite optimal baseline that minimizes reconstruction errors in all three world directions. We also show why previous oversimplified error analysis results in infinite baselines. We derive the mathematical relationship between the error variances and the stereo system parameters. In our analysis, we consider the situations where errors exist in only one camera as well as errors in both cameras. We have derived the results for both parallel and verged systems, though only the simpler models are presented algebraically herein. The paper includes simulations to highlight the results and validate the approximations in the error propagation. The results should allow stereo system designers, or those using motion-stereo, to improve their system.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"2015 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127774303","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711526
S. Oldridge, S. Fels, G. Miller
This paper introduces a system that automatically classifies image pairs based on the type of registration required to align them. The system uses support vector machines to classify between panoramas, high-dynamic-range images, focal stacks, super-resolution, and unrelated image pairs. A feature vector was developed to describe the images, and 1100 pairs were used to train and test the system with 5-fold cross validation. The system is able to classify the desired registration application using a 1: Many classifier with an accuracy of 91.18%. Similarly 1:1 classifiers were developed for each class with classification rates as follows: Panorama image pairs are classified at 93.15%, high-dynamic-range pairs at 97.56%, focal stack pairs at 95.68%, super-resolution pairs at 99.25%, and finally unrelated image pairs at 95.79%. An investigation into feature importance outlines the utility of each feature individually. In addition, the invariance of the classification system towards the size of the image used to calculate the feature vector was explored. The classification of our system remains level at ∼91% until the image size is scaled to 10% (150 × 100 pixels), suggesting that our feature vector is image size invariant within this range.
{"title":"Classification of image registration problems using support vector machines","authors":"S. Oldridge, S. Fels, G. Miller","doi":"10.1109/WACV.2011.5711526","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711526","url":null,"abstract":"This paper introduces a system that automatically classifies image pairs based on the type of registration required to align them. The system uses support vector machines to classify between panoramas, high-dynamic-range images, focal stacks, super-resolution, and unrelated image pairs. A feature vector was developed to describe the images, and 1100 pairs were used to train and test the system with 5-fold cross validation. The system is able to classify the desired registration application using a 1: Many classifier with an accuracy of 91.18%. Similarly 1:1 classifiers were developed for each class with classification rates as follows: Panorama image pairs are classified at 93.15%, high-dynamic-range pairs at 97.56%, focal stack pairs at 95.68%, super-resolution pairs at 99.25%, and finally unrelated image pairs at 95.79%. An investigation into feature importance outlines the utility of each feature individually. In addition, the invariance of the classification system towards the size of the image used to calculate the feature vector was explored. The classification of our system remains level at ∼91% until the image size is scaled to 10% (150 × 100 pixels), suggesting that our feature vector is image size invariant within this range.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130851847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711512
Raghavender R. Jillela, A. Ross, P. Flynn
The focus of this work is on improving the recognition performance of low-resolution iris video frames acquired under varying illumination. To facilitate this, an image-level fusion scheme with modest computational requirements is proposed. The proposed algorithm uses the evidence of multiple image frames of the same iris to extract discriminatory information via the Principal Components Transform (PCT). Experimental results on a subset of the MBGC NIR iris database demonstrate the utility of this scheme to achieve improved recognition accuracy when low-resolution probe images are compared against high-resolution gallery images.
{"title":"Information fusion in low-resolution iris videos using Principal Components Transform","authors":"Raghavender R. Jillela, A. Ross, P. Flynn","doi":"10.1109/WACV.2011.5711512","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711512","url":null,"abstract":"The focus of this work is on improving the recognition performance of low-resolution iris video frames acquired under varying illumination. To facilitate this, an image-level fusion scheme with modest computational requirements is proposed. The proposed algorithm uses the evidence of multiple image frames of the same iris to extract discriminatory information via the Principal Components Transform (PCT). Experimental results on a subset of the MBGC NIR iris database demonstrate the utility of this scheme to achieve improved recognition accuracy when low-resolution probe images are compared against high-resolution gallery images.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126148836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711536
M. R. Bales, D. Forsthoefel, D. S. Wills, L. Wills
Illumination changes present challenging problems to video surveillance algorithms tasked with identifying and tracking objects. Illumination changes can drastically alter the appearance of a scene, causing truly salient features to be lost amid otherwise stable background. We describe an illumination change compensation method that identifies large, stable, chromatically distinct background features-called BigBackground regions — which are used as calibration anchors for scene correction. The benefits of this method are demonstrated for a computationally low-cost kinematic tracking application as it attempts to track objects during illumination changes. The BigBackground-based method is compared with other compensation techniques, and is found to successfully track 60% to 80% more objects during illumination changes. Video sequences of pedestrian and vehicular traffic are used for evaluation.
{"title":"Illumination change compensation techniques to improve kinematic tracking","authors":"M. R. Bales, D. Forsthoefel, D. S. Wills, L. Wills","doi":"10.1109/WACV.2011.5711536","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711536","url":null,"abstract":"Illumination changes present challenging problems to video surveillance algorithms tasked with identifying and tracking objects. Illumination changes can drastically alter the appearance of a scene, causing truly salient features to be lost amid otherwise stable background. We describe an illumination change compensation method that identifies large, stable, chromatically distinct background features-called BigBackground regions — which are used as calibration anchors for scene correction. The benefits of this method are demonstrated for a computationally low-cost kinematic tracking application as it attempts to track objects during illumination changes. The BigBackground-based method is compared with other compensation techniques, and is found to successfully track 60% to 80% more objects during illumination changes. Video sequences of pedestrian and vehicular traffic are used for evaluation.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114764053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711505
R. Abiantun, Utsav Prabhu, Keshav Seshadri, J. Heo, M. Savvides
Traditional approaches to face recognition have utilized aligned facial images containing both shape and texture information. This paper analyzes the contributions of the individual facial shape and texture components to face recognition. These two components are evaluated independently and we investigate methods to combine the information gained from each of them to enhance face recognition performance. The contributions of this paper are the following: (1) to the best of our knowledge, it is the first large-scale study of how face recognition is influenced by shape and texture as all of our results are benchmarked against traditional approaches on the challenging NIST FRGC ver2.0 experiment 4 dataset, (2) we empirically show that shape information is reasonably discriminative, (3) we demonstrate significant improvement in performance by registering texture with dense shape information, and finally (4) show that fusing shape and texture information consistently boosts recognition results across different subspace-based algorithms.
{"title":"An analysis of facial shape and texture for recognition: A large scale evaluation on FRGC ver2.0","authors":"R. Abiantun, Utsav Prabhu, Keshav Seshadri, J. Heo, M. Savvides","doi":"10.1109/WACV.2011.5711505","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711505","url":null,"abstract":"Traditional approaches to face recognition have utilized aligned facial images containing both shape and texture information. This paper analyzes the contributions of the individual facial shape and texture components to face recognition. These two components are evaluated independently and we investigate methods to combine the information gained from each of them to enhance face recognition performance. The contributions of this paper are the following: (1) to the best of our knowledge, it is the first large-scale study of how face recognition is influenced by shape and texture as all of our results are benchmarked against traditional approaches on the challenging NIST FRGC ver2.0 experiment 4 dataset, (2) we empirically show that shape information is reasonably discriminative, (3) we demonstrate significant improvement in performance by registering texture with dense shape information, and finally (4) show that fusing shape and texture information consistently boosts recognition results across different subspace-based algorithms.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124531572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-01-05DOI: 10.1109/WACV.2011.5711483
Bohyung Han, Jihun Hamm, Jack Sim
In automatic video summarization, visual summary is constructed typically based on the analysis of low-level features with little consideration of video semantics. However, the contextual and semantic information of a video is marginally related to low-level features in practice although they are useful to compute visual similarity between frames. Therefore, we propose a novel video summarization technique, where the semantically important information is extracted from a set of keyframes given by human and the summary of a video is constructed based on the automatic temporal segmentation using the analysis of inter-frame similarity to the keyframes. Toward this goal, we model a video sequence with a dissimilarity matrix based on bidirectional similarity measure between every pair of frames, and subsequently characterize the structure of the video by a nonlinear manifold embedding. Then, we formulate video summarization as a variant of the 0–1 knapsack problem, which is solved by dynamic programming efficiently. The effectiveness of our algorithm is illustrated quantitatively and qualitatively using realistic videos collected from YouTube.
{"title":"Personalized video summarization with human in the loop","authors":"Bohyung Han, Jihun Hamm, Jack Sim","doi":"10.1109/WACV.2011.5711483","DOIUrl":"https://doi.org/10.1109/WACV.2011.5711483","url":null,"abstract":"In automatic video summarization, visual summary is constructed typically based on the analysis of low-level features with little consideration of video semantics. However, the contextual and semantic information of a video is marginally related to low-level features in practice although they are useful to compute visual similarity between frames. Therefore, we propose a novel video summarization technique, where the semantically important information is extracted from a set of keyframes given by human and the summary of a video is constructed based on the automatic temporal segmentation using the analysis of inter-frame similarity to the keyframes. Toward this goal, we model a video sequence with a dissimilarity matrix based on bidirectional similarity measure between every pair of frames, and subsequently characterize the structure of the video by a nonlinear manifold embedding. Then, we formulate video summarization as a variant of the 0–1 knapsack problem, which is solved by dynamic programming efficiently. The effectiveness of our algorithm is illustrated quantitatively and qualitatively using realistic videos collected from YouTube.","PeriodicalId":424724,"journal":{"name":"2011 IEEE Workshop on Applications of Computer Vision (WACV)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124903179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：