We present a technique for learning a manifold of human motion data using Convolutional Autoencoders. Our approach is capable of learning a manifold on the complete CMU database of human motion. This manifold can be treated as a prior probability distribution over human motion data, which has many applications in animation research, including projecting invalid or corrupt motion onto the manifold for removing error, computing similarity between motions using geodesic distance along the manifold, and interpolation of motion along the manifold for avoiding blending artefacts.
{"title":"Learning motion manifolds with convolutional autoencoders","authors":"Daniel Holden, Jun Saito, T. Komura, T. Joyce","doi":"10.1145/2820903.2820918","DOIUrl":"https://doi.org/10.1145/2820903.2820918","url":null,"abstract":"We present a technique for learning a manifold of human motion data using Convolutional Autoencoders. Our approach is capable of learning a manifold on the complete CMU database of human motion. This manifold can be treated as a prior probability distribution over human motion data, which has many applications in animation research, including projecting invalid or corrupt motion onto the manifold for removing error, computing similarity between motions using geodesic distance along the manifold, and interpolation of motion along the manifold for avoiding blending artefacts.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"58 3 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82450021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. Pruvost, B. Scherrer, M. Aramaki, S. Ystad, R. Kronland-Martinet
This brief introduces a novel framework for the interactive and real-time synthesis of solids' interaction sounds driven by a game engine. The sound synthesizer used in this work relies on an action-object paradigm, itself based on the notion of perceptual invariants. An intuitive control strategy, based on those invariants and inspired by physics, was developed. The action and the object can be controlled independently, simultaneously, and continuously. This allows the synthesis of sounds for solids' interactions whose nature evolves continuously over time (e.g. from rolling to slipping) and/or where the objects' properties (shape, size and material) vary continuously in time.
{"title":"Perception-based interactive sound synthesis of morphing solids' interactions","authors":"L. Pruvost, B. Scherrer, M. Aramaki, S. Ystad, R. Kronland-Martinet","doi":"10.1145/2820903.2820914","DOIUrl":"https://doi.org/10.1145/2820903.2820914","url":null,"abstract":"This brief introduces a novel framework for the interactive and real-time synthesis of solids' interaction sounds driven by a game engine. The sound synthesizer used in this work relies on an action-object paradigm, itself based on the notion of perceptual invariants. An intuitive control strategy, based on those invariants and inspired by physics, was developed. The action and the object can be controlled independently, simultaneously, and continuously. This allows the synthesis of sounds for solids' interactions whose nature evolves continuously over time (e.g. from rolling to slipping) and/or where the objects' properties (shape, size and material) vary continuously in time.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"286 3 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79630415","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we propose a method for hiding binary images into the gradient domain of a host color image, by modifying the gradient vectors to a chosen hiding vector orientation in CIELAB color space. The color changes are constrained by the just-noticeable difference (JND) to guarantee the imperceptibility of embedded data. Generally, one pair of pixels is used to embed one binary bit. Using multiple hiding vectors, indexed color images or multiple binary images can be embedded in one image simultaneously. Hence high capacity can be achieved in comparison with the existing methods.
{"title":"Gradient domain binary image hiding using color difference metric","authors":"Lu Hao, J. Feng, Bingfeng Zhou","doi":"10.1145/2820903.2820919","DOIUrl":"https://doi.org/10.1145/2820903.2820919","url":null,"abstract":"In this paper, we propose a method for hiding binary images into the gradient domain of a host color image, by modifying the gradient vectors to a chosen hiding vector orientation in CIELAB color space. The color changes are constrained by the just-noticeable difference (JND) to guarantee the imperceptibility of embedded data. Generally, one pair of pixels is used to embed one binary bit. Using multiple hiding vectors, indexed color images or multiple binary images can be embedded in one image simultaneously. Hence high capacity can be achieved in comparison with the existing methods.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"23 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89679196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Freehand drawing is one of the most flexible and efficient ways of expressing creative ideas. However, it can often also be tedious and technically challenging to animate complex dimensional environments or dynamic choreographies. We present a case study of how freehand drawing tools can be integrated into an end-to-end 3D content creation platform to reap the benefits from both worlds. Creative opportunities and challenges in achieving this type of integration are discussed. We also present examples from short films demonstrating the potential of how these techniques can be deployed in production environments.
{"title":"Grease pencil: integrating animated freehand drawings into 3D production environments","authors":"Joshua Leung, Daniel M. Lara","doi":"10.1145/2820903.2820924","DOIUrl":"https://doi.org/10.1145/2820903.2820924","url":null,"abstract":"Freehand drawing is one of the most flexible and efficient ways of expressing creative ideas. However, it can often also be tedious and technically challenging to animate complex dimensional environments or dynamic choreographies. We present a case study of how freehand drawing tools can be integrated into an end-to-end 3D content creation platform to reap the benefits from both worlds. Creative opportunities and challenges in achieving this type of integration are discussed. We also present examples from short films demonstrating the potential of how these techniques can be deployed in production environments.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"27 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89594121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
X. Burgos-Artizzu, J. Fleureau, Olivier Dumas, Thierry Tapie, F. Clerc, N. Mollet
One of the main issues of current Head-Mounted Displays (HMD) is that they hide completely the wearer's face. This can be an issue in social experiences where two or more users want to share the 3D immersive experience. We propose a novel method to recover the face of the user in real-time. First, we learn the user appearance offline by building a 3D textured model of his head from a series of pictures. Then, by calibrating the camera and tracking the HMD's position in real-time we reproject the model on top of the video frames mimicking exactly the user's head pose. Finally, we remove the HMD and replace the occluded part of the face in a seamingless manner by performing image in-painting with the background. We further propose an extension to detect facial expressions on the visible part of the face and use it to change the upper face model accordingly. We show the promise of our method via some qualitative results on a variety of users.
{"title":"Real-time expression-sensitive HMD face reconstruction","authors":"X. Burgos-Artizzu, J. Fleureau, Olivier Dumas, Thierry Tapie, F. Clerc, N. Mollet","doi":"10.1145/2820903.2820910","DOIUrl":"https://doi.org/10.1145/2820903.2820910","url":null,"abstract":"One of the main issues of current Head-Mounted Displays (HMD) is that they hide completely the wearer's face. This can be an issue in social experiences where two or more users want to share the 3D immersive experience. We propose a novel method to recover the face of the user in real-time. First, we learn the user appearance offline by building a 3D textured model of his head from a series of pictures. Then, by calibrating the camera and tracking the HMD's position in real-time we reproject the model on top of the video frames mimicking exactly the user's head pose. Finally, we remove the HMD and replace the occluded part of the face in a seamingless manner by performing image in-painting with the background. We further propose an extension to detect facial expressions on the visible part of the face and use it to change the upper face model accordingly. We show the promise of our method via some qualitative results on a variety of users.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"105 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80867605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we propose a depth-aided patch based inpainting method to perform the disocclusion of holes that appear when synthesizing virtual views from RGB-D scenes. Depth information is added to each key step of the classical patch-based algorithm from [Criminisi et al. 2004] to guide the synthesis of missing structures and textures. These contributions result in a new inpainting method which is efficient compared to state-of-the-art approaches (both in visual quality and computational burden), while requiring only a single easy-to-adjust additional parameter.
本文提出了一种基于深度辅助补片的补图方法，用于对RGB-D场景合成虚拟视图时出现的孔洞进行去遮挡。在经典的[Criminisi et al. 2004]基于patch的算法的每个关键步骤中加入深度信息，以指导缺失结构和纹理的合成。这些贡献产生了一种新的绘制方法，与最先进的方法相比(在视觉质量和计算负担方面)，它更有效，同时只需要一个易于调整的附加参数。
{"title":"Depth-aware patch-based image disocclusion for virtual view synthesis","authors":"P. Buyssens, M. Daisy, D. Tschumperlé, O. Lézoray","doi":"10.1145/2820903.2820920","DOIUrl":"https://doi.org/10.1145/2820903.2820920","url":null,"abstract":"In this paper we propose a depth-aided patch based inpainting method to perform the disocclusion of holes that appear when synthesizing virtual views from RGB-D scenes. Depth information is added to each key step of the classical patch-based algorithm from [Criminisi et al. 2004] to guide the synthesis of missing structures and textures. These contributions result in a new inpainting method which is efficient compared to state-of-the-art approaches (both in visual quality and computational burden), while requiring only a single easy-to-adjust additional parameter.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"60 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77192183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Linear blending techniques based on generalized barycentric cordinates have been well recognized in a digital production workplace for shape deformation due to its simplicity. However, the dense weights, non-rigid deformations and the lack of an intuitive control interface limit its practical use. In this paper we present a novel linear blending scheme utilizing existing barycentric coordinates to overcome these difficulties. The scheme enables cage vertices associated with sparse weights to be inferred automatically by a user's manipulation on constrained vertices or handles in real-time. For this scheme, we have developed two new techniques. The first one is a weight reduction technique to reduce the number of control points on the cage, while still keeping the surface quality. Another technique computes positions of the cage vertices, in order to preserve the rigidity of the shape by minimizing nonlinear rigidity energies. Our prototype system demonstrates that our linear blending scheme can deal with rigid and non-rigid deformation more consistently and efficiently than previous approaches.
{"title":"A linear blending scheme for rigid and non-rigid deformations","authors":"Gengdai Liu, K. Anjyo","doi":"10.1145/2820903.2820912","DOIUrl":"https://doi.org/10.1145/2820903.2820912","url":null,"abstract":"Linear blending techniques based on generalized barycentric cordinates have been well recognized in a digital production workplace for shape deformation due to its simplicity. However, the dense weights, non-rigid deformations and the lack of an intuitive control interface limit its practical use. In this paper we present a novel linear blending scheme utilizing existing barycentric coordinates to overcome these difficulties. The scheme enables cage vertices associated with sparse weights to be inferred automatically by a user's manipulation on constrained vertices or handles in real-time. For this scheme, we have developed two new techniques. The first one is a weight reduction technique to reduce the number of control points on the cage, while still keeping the surface quality. Another technique computes positions of the cage vertices, in order to preserve the rigidity of the shape by minimizing nonlinear rigidity energies. Our prototype system demonstrates that our linear blending scheme can deal with rigid and non-rigid deformation more consistently and efficiently than previous approaches.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75801235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Referring to existing illustrations helps novice drawers to realize their ideas. To find such helpful references from a large image collection, we first build a semantic vector representation of illustrations by training convolutional neural networks. As the proposed vector space correctly reflects the semantic meanings of illustrations, users can efficiently search for references with similar attributes. Besides the search with a single query, a semantic morphing algorithm that searches the intermediate illustrations that gradually connect two queries is proposed. Several experiments were conducted to demonstrate the effectiveness of our methods.
{"title":"Illustration2Vec: a semantic vector representation of illustrations","authors":"Masaki Saito, Yusuke Matsui","doi":"10.1145/2820903.2820907","DOIUrl":"https://doi.org/10.1145/2820903.2820907","url":null,"abstract":"Referring to existing illustrations helps novice drawers to realize their ideas. To find such helpful references from a large image collection, we first build a semantic vector representation of illustrations by training convolutional neural networks. As the proposed vector space correctly reflects the semantic meanings of illustrations, users can efficiently search for references with similar attributes. Besides the search with a single query, a semantic morphing algorithm that searches the intermediate illustrations that gradually connect two queries is proposed. Several experiments were conducted to demonstrate the effectiveness of our methods.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"161 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73657167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
T. Viitanen, M. Koskela, P. Jääskeläinen, Heikki O. Kultala, J. Takala
Powerful hardware accelerators have been recently developed that put interactive ray-tracing even in the reach of mobile devices. However, supplying the rendering unit with up-to date acceleration trees remains difficult, so the rendered scenes are mostly static. The restricted memory bandwidth of a mobile device is a challenge with applying GPU-based tree construction algorithms. This paper describes MergeTree, a BVH tree constructor architecture based on the HLBVH algorithm, whose main features of interest are a streaming hierarchy emitter, an external sorting algorithm with provably minimal memory usage, and a hardware priority queue used to accelerate the external sort. In simulations, the resulting unit is faster by a factor of three than the state-of-the art hardware builder based on the binned SAH sweep algorithm.
{"title":"MergeTree: a HLBVH constructor for mobile systems","authors":"T. Viitanen, M. Koskela, P. Jääskeläinen, Heikki O. Kultala, J. Takala","doi":"10.1145/2820903.2820916","DOIUrl":"https://doi.org/10.1145/2820903.2820916","url":null,"abstract":"Powerful hardware accelerators have been recently developed that put interactive ray-tracing even in the reach of mobile devices. However, supplying the rendering unit with up-to date acceleration trees remains difficult, so the rendered scenes are mostly static. The restricted memory bandwidth of a mobile device is a challenge with applying GPU-based tree construction algorithms. This paper describes MergeTree, a BVH tree constructor architecture based on the HLBVH algorithm, whose main features of interest are a streaming hierarchy emitter, an external sorting algorithm with provably minimal memory usage, and a hardware priority queue used to accelerate the external sort. In simulations, the resulting unit is faster by a factor of three than the state-of-the art hardware builder based on the binned SAH sweep algorithm.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"6 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86700466","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zeyu Wang, Xiaohan Jin, Fei Xue, Xin He, R. Li, H. Zha
As panoramas provide a brand new viewpoint for the public, relevant cameras and software such as RICOH Theta, Microsoft Photosynth are embracing more and more users. However, the display methods for panoramas remain monotonous. In this paper, we propose a novel representation method called Content-Aware Cube Unwrapping using the effective and interactive techniques of orientational rectification, image modification and energy estimation. Thus, a number of fascinating applications will come into reality. For instance, six surfaces of a Rubiks cube can be automatically rendered from a vertically oriented panorama, without cutting any person or significant object apart. Moreover, seam carving and inserting are applied to each surface to enhance the key content and to make the scenery more consistent.
{"title":"Panorama to cube: a content-aware representation method","authors":"Zeyu Wang, Xiaohan Jin, Fei Xue, Xin He, R. Li, H. Zha","doi":"10.1145/2820903.2820911","DOIUrl":"https://doi.org/10.1145/2820903.2820911","url":null,"abstract":"As panoramas provide a brand new viewpoint for the public, relevant cameras and software such as RICOH Theta, Microsoft Photosynth are embracing more and more users. However, the display methods for panoramas remain monotonous. In this paper, we propose a novel representation method called Content-Aware Cube Unwrapping using the effective and interactive techniques of orientational rectification, image modification and energy estimation. Thus, a number of fascinating applications will come into reality. For instance, six surfaces of a Rubiks cube can be automatically rendered from a vertically oriented panorama, without cutting any person or significant object apart. Moreover, seam carving and inserting are applied to each surface to enhance the key content and to make the scenery more consistent.","PeriodicalId":21720,"journal":{"name":"SIGGRAPH Asia 2015 Technical Briefs","volume":"29 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2015-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89653202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：