首页 > 最新文献

Computer Graphics Forum最新文献

英文 中文
SketchAnim: Real-time sketch animation transfer from videos SketchAnim:从视频实时传输草图动画
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15176
Gaurav Rai, Shreyas Gupta, Ojaswa Sharma

Animation of hand-drawn sketches is an adorable art. It allows the animator to generate animations with expressive freedom and requires significant expertise. In this work, we introduce a novel sketch animation framework designed to address inherent challenges, such as motion extraction, motion transfer, and occlusion. The framework takes an exemplar video input featuring a moving object and utilizes a robust motion transfer technique to animate the input sketch. We show comparative evaluations that demonstrate the superior performance of our method over existing sketch animation techniques. Notably, our approach exhibits a higher level of user accessibility in contrast to conventional sketch-based animation systems, positioning it as a promising contributor to the field of sketch animation. https://graphics-research-group.github.io/SketchAnim/

手绘草图动画是一门可爱的艺术。它允许动画制作者自由生成动画,但需要大量的专业知识。在这项工作中,我们引入了一个新颖的草图动画框架,旨在解决运动提取、运动转移和遮挡等固有难题。该框架采用以移动物体为特征的示例视频输入,并利用稳健的运动转移技术为输入草图制作动画。我们进行了比较评估,证明我们的方法比现有的草图动画技术性能更优越。值得注意的是,与传统的基于草图的动画系统相比,我们的方法显示出更高的用户可访问性,使其在草图动画领域大有可为。https://graphics-research-group.github.io/SketchAnim/。
{"title":"SketchAnim: Real-time sketch animation transfer from videos","authors":"Gaurav Rai,&nbsp;Shreyas Gupta,&nbsp;Ojaswa Sharma","doi":"10.1111/cgf.15176","DOIUrl":"https://doi.org/10.1111/cgf.15176","url":null,"abstract":"<p>Animation of hand-drawn sketches is an adorable art. It allows the animator to generate animations with expressive freedom and requires significant expertise. In this work, we introduce a novel sketch animation framework designed to address inherent challenges, such as motion extraction, motion transfer, and occlusion. The framework takes an exemplar video input featuring a moving object and utilizes a robust motion transfer technique to animate the input sketch. We show comparative evaluations that demonstrate the superior performance of our method over existing sketch animation techniques. Notably, our approach exhibits a higher level of user accessibility in contrast to conventional sketch-based animation systems, positioning it as a promising contributor to the field of sketch animation. https://graphics-research-group.github.io/SketchAnim/</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707470","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Creating a 3D Mesh in A-pose from a Single Image for Character Rigging 在 A-pose 中从单一图像创建 3D 网格,用于角色装配
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15177
Seunghwan Lee, C. Karen Liu

Learning-based methods for 3D content generation have shown great potential to create 3D characters from text prompts, videos, and images. However, current methods primarily focus on generating static 3D meshes, overlooking the crucial aspect of creating an animatable 3D meshes. Directly using 3D meshes generated by existing methods to create underlying skeletons for animation presents many challenges because the generated mesh might exhibit geometry artifacts or assume arbitrary poses that complicate the subsequent rigging process. This work proposes a new framework for generating a 3D animatable mesh from a single 2D image depicting the character. We do so by enforcing the generated 3D mesh to assume an A-pose, which can mitigate the geometry artifacts and facilitate the use of existing automatic rigging methods. Our approach aims to leverage the generative power of existing models across modalities without the need for new data or large-scale training. We evaluate the effectiveness of our framework with qualitative results, as well as ablation studies and quantitative comparisons with existing 3D mesh generation models.

基于学习的三维内容生成方法在根据文本提示、视频和图像创建三维角色方面显示出巨大的潜力。然而,目前的方法主要侧重于生成静态三维网格,忽略了创建可动画化三维网格这一关键环节。直接使用现有方法生成的三维网格来创建动画底层骨架会面临许多挑战,因为生成的网格可能会出现几何假象或任意姿势,从而使后续的装配过程复杂化。本作品提出了一种新的框架,用于从描绘角色的单张二维图像生成三维动画网格。我们的方法是强制生成的三维网格采用 A 姿态,这样可以减少几何假象,方便使用现有的自动装配方法。我们的方法旨在利用现有跨模态模型的生成能力,而无需新数据或大规模训练。我们通过定性结果、消融研究以及与现有三维网格生成模型的定量比较来评估我们框架的有效性。
{"title":"Creating a 3D Mesh in A-pose from a Single Image for Character Rigging","authors":"Seunghwan Lee,&nbsp;C. Karen Liu","doi":"10.1111/cgf.15177","DOIUrl":"https://doi.org/10.1111/cgf.15177","url":null,"abstract":"<p>Learning-based methods for 3D content generation have shown great potential to create 3D characters from text prompts, videos, and images. However, current methods primarily focus on generating static 3D meshes, overlooking the crucial aspect of creating an animatable 3D meshes. Directly using 3D meshes generated by existing methods to create underlying skeletons for animation presents many challenges because the generated mesh might exhibit geometry artifacts or assume arbitrary poses that complicate the subsequent rigging process. This work proposes a new framework for generating a 3D animatable mesh from a single 2D image depicting the character. We do so by enforcing the generated 3D mesh to assume an A-pose, which can mitigate the geometry artifacts and facilitate the use of existing automatic rigging methods. Our approach aims to leverage the generative power of existing models across modalities without the need for new data or large-scale training. We evaluate the effectiveness of our framework with qualitative results, as well as ablation studies and quantitative comparisons with existing 3D mesh generation models.</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Learning to Move Like Professional Counter-Strike Players 学习像职业反恐精英玩家那样移动
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15173
D. Durst, F. Xie, V. Sarukkai, B. Shacklett, I. Frosio, C. Tessler, J. Kim, C. Taylor, G. Bernstein, S. Choudhury, P. Hanrahan, K. Fatahalian

In multiplayer, first-person shooter games like Counter-Strike: Global Offensive (CS:GO), coordinated movement is a critical component of high-level strategic play. However, the complexity of team coordination and the variety of conditions present in popular game maps make it impractical to author hand-crafted movement policies for every scenario. We show that it is possible to take a data-driven approach to creating human-like movement controllers for CS:GO. We curate a team movement dataset comprising 123 hours of professional game play traces, and use this dataset to train a transformer-based movement model that generates human-like team movement for all players in a “Retakes” round of the game. Importantly, the movement prediction model is efficient. Performing inference for all players takes less than 0.5 ms per game step (amortized cost) on a single CPU core, making it plausible for use in commercial games today. Human evaluators assess that our model behaves more like humans than both commercially-available bots and procedural movement controllers scripted by experts (16% to 59% higher by TrueSkill rating of “human-like”). Using experiments involving in-game bot vs. bot self-play, we demonstrate that our model performs simple forms of teamwork, makes fewer common movement mistakes, and yields movement distributions, player lifetimes, and kill locations similar to those observed in professional CS:GO match play.

在《反恐精英:全球攻势》(CS:GO)等多人第一人称射击游戏中,协调移动是高水平战略游戏的关键组成部分:在《反恐精英:全球攻势》(CS:GO)等多人第一人称射击游戏中,协调移动是高水平战略游戏的重要组成部分。然而,由于团队协调的复杂性和流行游戏地图中存在的各种情况,要针对每种情况制定手工制作的移动策略是不切实际的。我们的研究表明,采用数据驱动的方法为 CS:GO 创建类人动作控制器是可行的。我们策划了一个团队移动数据集,其中包括 123 个小时的职业比赛轨迹,并利用该数据集训练了一个基于变压器的移动模型,该模型可在游戏的 "重拍 "回合中为所有玩家生成类似人类的团队移动。重要的是,运动预测模型非常高效。在单个 CPU 内核上对所有球员进行推理,每个游戏步骤所需的时间不到 0.5 毫秒(摊销成本),因此可以在当今的商业游戏中使用。人类评估人员认为,与市面上的机器人和专家编写的程序化动作控制器相比,我们的模型表现得更像人类(根据 TrueSkill 的 "类人 "评级,高出 16% 至 59%)。通过游戏中机器人与机器人自我对战的实验,我们证明了我们的模型可以进行简单形式的团队合作,较少犯常见的移动错误,并产生与专业 CS:GO 比赛中观察到的类似的移动分布、玩家生命周期和击杀位置。
{"title":"Learning to Move Like Professional Counter-Strike Players","authors":"D. Durst,&nbsp;F. Xie,&nbsp;V. Sarukkai,&nbsp;B. Shacklett,&nbsp;I. Frosio,&nbsp;C. Tessler,&nbsp;J. Kim,&nbsp;C. Taylor,&nbsp;G. Bernstein,&nbsp;S. Choudhury,&nbsp;P. Hanrahan,&nbsp;K. Fatahalian","doi":"10.1111/cgf.15173","DOIUrl":"https://doi.org/10.1111/cgf.15173","url":null,"abstract":"<p>In multiplayer, first-person shooter games like Counter-Strike: Global Offensive (CS:GO), coordinated movement is a critical component of high-level strategic play. However, the complexity of team coordination and the variety of conditions present in popular game maps make it impractical to author hand-crafted movement policies for every scenario. We show that it is possible to take a data-driven approach to creating human-like movement controllers for CS:GO. We curate a team movement dataset comprising 123 hours of professional game play traces, and use this dataset to train a transformer-based movement model that generates human-like team movement for all players in a “Retakes” round of the game. Importantly, the movement prediction model is efficient. Performing inference for all players takes less than 0.5 ms per game step (amortized cost) on a single CPU core, making it plausible for use in commercial games today. Human evaluators assess that our model behaves more like humans than both commercially-available bots and procedural movement controllers scripted by experts (16% to 59% higher by TrueSkill rating of “human-like”). Using experiments involving in-game bot vs. bot self-play, we demonstrate that our model performs simple forms of teamwork, makes fewer common movement mistakes, and yields movement distributions, player lifetimes, and kill locations similar to those observed in professional CS:GO match play.</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reactive Gaze during Locomotion in Natural Environments 自然环境中运动时的反应性凝视
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15168
J. K. Melgare, D. Rohmer, S. R. Musse, M-P. Cani

Animating gaze behavior is crucial for creating believable virtual characters, providing insights into their perception and interaction with the environment. In this paper, we present an efficient yet natural-looking gaze animation model applicable to real-time walking characters exploring natural environments. We address the challenge of dynamic gaze adaptation by combining findings from neuroscience with a data-driven saliency model. Specifically, our model determines gaze focus by considering the character's locomotion, environment stimuli, and terrain conditions. Our model is compatible with both automatic navigation through pre-defined character trajectories and user-guided interactive locomotion, and can be configured according to the desired degree of visual exploration of the environment. Our perceptual evaluation shows that our solution significantly improves the state-of-the-art saliency-based gaze animation with respect to the character's apparent awareness of the environment, the naturalness of the motion, and the elements to which it pays attention.

注视行为动画对于创建可信的虚拟角色至关重要,它能让人们深入了解虚拟角色的感知以及与环境的互动。在本文中,我们提出了一种高效而自然的注视动画模型,适用于探索自然环境的实时行走角色。我们将神经科学的研究成果与数据驱动的显著性模型相结合,解决了动态注视适应的难题。具体来说,我们的模型通过考虑角色的运动、环境刺激和地形条件来确定注视焦点。我们的模型兼容通过预定义角色轨迹进行的自动导航和用户引导的交互式运动,并可根据所需的环境视觉探索程度进行配置。我们的感知评估结果表明,我们的解决方案在角色对环境的明显感知、运动的自然度以及角色所关注的元素方面,显著改善了最先进的基于显著性的注视动画。
{"title":"Reactive Gaze during Locomotion in Natural Environments","authors":"J. K. Melgare,&nbsp;D. Rohmer,&nbsp;S. R. Musse,&nbsp;M-P. Cani","doi":"10.1111/cgf.15168","DOIUrl":"https://doi.org/10.1111/cgf.15168","url":null,"abstract":"<p>Animating gaze behavior is crucial for creating believable virtual characters, providing insights into their perception and interaction with the environment. In this paper, we present an efficient yet natural-looking gaze animation model applicable to real-time walking characters exploring natural environments. We address the challenge of dynamic gaze adaptation by combining findings from neuroscience with a data-driven saliency model. Specifically, our model determines gaze focus by considering the character's locomotion, environment stimuli, and terrain conditions. Our model is compatible with both automatic navigation through pre-defined character trajectories and user-guided interactive locomotion, and can be configured according to the desired degree of visual exploration of the environment. Our perceptual evaluation shows that our solution significantly improves the state-of-the-art saliency-based gaze animation with respect to the character's apparent awareness of the environment, the naturalness of the motion, and the elements to which it pays attention.</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters VMP:用于可靠跟踪物理字符运动的多功能运动先验器
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15175
Agon Serifi, Ruben Grandia, Espen Knoop, Markus Gross, Moritz Bächer

Recent progress in physics-based character control has made it possible to learn policies from unstructured motion data. However, it remains challenging to train a single control policy that works with diverse and unseen motions, and can be deployed to real-world physical robots. In this paper, we propose a two-stage technique that enables the control of a character with a full-body kinematic motion reference, with a focus on imitation accuracy. In a first stage, we extract a latent space encoding by training a variational autoencoder, taking short windows of motion from unstructured data as input. We then use the embedding from the time-varying latent code to train a conditional policy in a second stage, providing a mapping from kinematic input to dynamics-aware output. By keeping the two stages separate, we benefit from self-supervised methods to get better latent codes and explicit imitation rewards to avoid mode collapse. We demonstrate the efficiency and robustness of our method in simulation, with unseen user-specified motions, and on a bipedal robot, where we bring dynamic motions to the real world.

基于物理的角色控制领域的最新进展使得从非结构化运动数据中学习策略成为可能。然而,要训练出一种可适用于各种未知运动的单一控制策略,并将其部署到真实世界的物理机器人上,仍然具有挑战性。在本文中,我们提出了一种分两个阶段的技术,可以利用全身运动参考来控制角色,重点是模仿的准确性。在第一阶段,我们以非结构化数据中的短运动窗口为输入,通过训练变异自动编码器来提取潜在空间编码。然后,我们在第二阶段利用时变潜码的嵌入来训练条件策略,提供从运动输入到动态感知输出的映射。通过将两个阶段分开,我们可以利用自我监督方法获得更好的潜码,并利用明确的模仿奖励来避免模式崩溃。我们在模拟中演示了我们方法的效率和鲁棒性,包括用户指定的未知运动,以及在双足机器人上将动态运动带入现实世界。
{"title":"VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters","authors":"Agon Serifi,&nbsp;Ruben Grandia,&nbsp;Espen Knoop,&nbsp;Markus Gross,&nbsp;Moritz Bächer","doi":"10.1111/cgf.15175","DOIUrl":"https://doi.org/10.1111/cgf.15175","url":null,"abstract":"<p>Recent progress in physics-based character control has made it possible to learn policies from unstructured motion data. However, it remains challenging to train a single control policy that works with diverse and unseen motions, and can be deployed to real-world physical robots. In this paper, we propose a two-stage technique that enables the control of a character with a full-body kinematic motion reference, with a focus on imitation accuracy. In a first stage, we extract a latent space encoding by training a variational autoencoder, taking short windows of motion from unstructured data as input. We then use the embedding from the time-varying latent code to train a conditional policy in a second stage, providing a mapping from kinematic input to dynamics-aware output. By keeping the two stages separate, we benefit from self-supervised methods to get better latent codes and explicit imitation rewards to avoid mode collapse. We demonstrate the efficiency and robustness of our method in simulation, with unseen user-specified motions, and on a bipedal robot, where we bring dynamic motions to the real world.</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Garment Animation NeRF with Color Editing 带有色彩编辑功能的服装动画 NeRF
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15178
Renke Wang, Meng Zhang, Jun Li, Jian Yang

Generating high-fidelity garment animations through traditional workflows, from modeling to rendering, is both tedious and expensive. These workflows often require repetitive steps in response to updates in character motion, rendering viewpoint changes, or appearance edits. Although recent neural rendering offers an efficient solution for computationally intensive processes, it struggles with rendering complex garment animations containing fine wrinkle details and realistic garment-and-body occlusions, while maintaining structural consistency across frames and dense view rendering. In this paper, we propose a novel approach to directly synthesize garment animations from body motion sequences without the need for an explicit garment proxy. Our approach infers garment dynamic features from body motion, providing a preliminary overview of garment structure. Simultaneously, we capture detailed features from synthesized reference images of the garment's front and back, generated by a pre-trained image model. These features are then used to construct a neural radiance field that renders the garment animation video. Additionally, our technique enables garment recoloring by decomposing its visual elements. We demonstrate the generalizability of our method across unseen body motions and camera views, ensuring detailed structural consistency. Furthermore, we showcase its applicability to color editing on both real and synthetic garment data. Compared to existing neural rendering techniques, our method exhibits qualitative and quantitative improvements in garment dynamics and wrinkle detail modeling. Code is available at https://github.com/wrk226/GarmentAnimationNeRF.

通过从建模到渲染的传统工作流程生成高保真服装动画既繁琐又昂贵。这些工作流程通常需要重复步骤,以应对角色运动更新、渲染视角变化或外观编辑。虽然最新的神经渲染技术为计算密集型流程提供了高效的解决方案,但在渲染包含精细褶皱细节和逼真的服装与人体遮挡物的复杂服装动画时,同时保持跨帧和密集视图渲染的结构一致性方面,它却显得力不从心。在本文中,我们提出了一种新方法,可直接从人体运动序列合成服装动画,而无需明确的服装代理。我们的方法可从身体运动中推断服装动态特征,提供服装结构的初步概览。与此同时,我们还能从预先训练好的图像模型生成的服装正面和背面合成参考图像中捕捉详细特征。然后利用这些特征构建神经辐射场,渲染服装动画视频。此外,我们的技术还能通过分解服装的视觉元素来实现服装的重新着色。我们展示了我们的方法在未知身体运动和摄像机视图中的通用性,确保了细节结构的一致性。此外,我们还展示了在真实和合成服装数据上进行色彩编辑的适用性。与现有的神经渲染技术相比,我们的方法在服装动态和褶皱细节建模方面有质的和量的改进。代码见 https://github.com/wrk226/GarmentAnimationNeRF。
{"title":"Garment Animation NeRF with Color Editing","authors":"Renke Wang,&nbsp;Meng Zhang,&nbsp;Jun Li,&nbsp;Jian Yang","doi":"10.1111/cgf.15178","DOIUrl":"https://doi.org/10.1111/cgf.15178","url":null,"abstract":"<p>Generating high-fidelity garment animations through traditional workflows, from modeling to rendering, is both tedious and expensive. These workflows often require repetitive steps in response to updates in character motion, rendering viewpoint changes, or appearance edits. Although recent neural rendering offers an efficient solution for computationally intensive processes, it struggles with rendering complex garment animations containing fine wrinkle details and realistic garment-and-body occlusions, while maintaining structural consistency across frames and dense view rendering. In this paper, we propose a novel approach to directly synthesize garment animations from body motion sequences without the need for an explicit garment proxy. Our approach infers garment dynamic features from body motion, providing a preliminary overview of garment structure. Simultaneously, we capture detailed features from synthesized reference images of the garment's front and back, generated by a pre-trained image model. These features are then used to construct a neural radiance field that renders the garment animation video. Additionally, our technique enables garment recoloring by decomposing its visual elements. We demonstrate the generalizability of our method across unseen body motions and camera views, ensuring detailed structural consistency. Furthermore, we showcase its applicability to color editing on both real and synthetic garment data. Compared to existing neural rendering techniques, our method exhibits qualitative and quantitative improvements in garment dynamics and wrinkle detail modeling. Code is available at https://github.com/wrk226/GarmentAnimationNeRF.</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Pose-to-Motion: Cross-Domain Motion Retargeting with Pose Prior 从姿势到动作:利用姿势先验进行跨域运动重定位
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15170
Qingqing Zhao, Peizhuo Li, Wang Yifan, Sorkine-Hornung Olga, Gordon Wetzstein

Creating plausible motions for a diverse range of characters is a long-standing goal in computer graphics. Current learning-based motion synthesis methods rely on large-scale motion datasets, which are often difficult if not impossible to acquire. On the other hand, pose data is more accessible, since static posed characters are easier to create and can even be extracted from images using recent advancements in computer vision. In this paper, we tap into this alternative data source and introduce a neural motion synthesis approach through retargeting, which generates plausible motion of various characters that only have pose data by transferring motion from one single existing motion capture dataset of another drastically different characters. Our experiments show that our method effectively combines the motion features of the source character with the pose features of the target character, and performs robustly with small or noisy pose data sets, ranging from a few artist-created poses to noisy poses estimated directly from images. Additionally, a conducted user study indicated that a majority of participants found our retargeted motion to be more enjoyable to watch, more lifelike in appearance, and exhibiting fewer artifacts. Our code and dataset can be accessed here.

为各种角色创建可信的动作是计算机图形学的一个长期目标。目前基于学习的运动合成方法依赖于大规模运动数据集,而这些数据集通常很难获取,甚至根本无法获取。另一方面,姿势数据则更容易获取,因为静态姿势角色更容易创建,甚至可以利用最新的计算机视觉技术从图像中提取出来。在本文中,我们利用这一替代数据源,通过重定向引入了一种神经运动合成方法,该方法通过从一个单一的现有运动捕捉数据集转移另一个截然不同角色的运动,生成只有姿势数据的各种角色的可信运动。我们的实验表明,我们的方法有效地结合了源角色的运动特征和目标角色的姿势特征,并能在姿势数据集较小或有噪声的情况下稳健运行,包括从艺术家创建的几个姿势到直接从图像估算出的噪声姿势。此外,一项用户研究表明,大多数参与者认为我们的重定向动作观看起来更令人愉悦,外观更逼真,而且假象更少。请点击此处查看我们的代码和数据集。
{"title":"Pose-to-Motion: Cross-Domain Motion Retargeting with Pose Prior","authors":"Qingqing Zhao,&nbsp;Peizhuo Li,&nbsp;Wang Yifan,&nbsp;Sorkine-Hornung Olga,&nbsp;Gordon Wetzstein","doi":"10.1111/cgf.15170","DOIUrl":"https://doi.org/10.1111/cgf.15170","url":null,"abstract":"<div>\u0000 <p>Creating plausible motions for a diverse range of characters is a long-standing goal in computer graphics. Current learning-based motion synthesis methods rely on large-scale motion datasets, which are often difficult if not impossible to acquire. On the other hand, pose data is more accessible, since static posed characters are easier to create and can even be extracted from images using recent advancements in computer vision. In this paper, we tap into this alternative data source and introduce a neural motion synthesis approach through retargeting, which generates plausible motion of various characters that only have pose data by transferring motion from one single existing motion capture dataset of another drastically different characters. Our experiments show that our method effectively combines the motion features of the source character with the pose features of the target character, and performs robustly with small or noisy pose data sets, ranging from a few artist-created poses to noisy poses estimated directly from images. Additionally, a conducted user study indicated that a majority of participants found our retargeted motion to be more enjoyable to watch, more lifelike in appearance, and exhibiting fewer artifacts. Our code and dataset can be accessed here.</p>\u0000 </div>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/cgf.15170","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707495","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Long-term Motion In-betweening via Keyframe Prediction 通过关键帧预测实现长期运动间隔
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15171
Seokhyeon Hong, Haemin Kim, Kyungmin Cho, Junyong Noh

Motion in-betweening has emerged as a promising approach to enhance the efficiency of motion creation due to its flexibility and time performance. However, previous in-betweening methods are limited to generating short transitions due to growing pose ambiguity when the number of missing frames increases. This length-related constraint makes the optimization hard and it further causes another constraint on the target pose, limiting the degrees of freedom for artists to use. In this paper, we introduce a keyframe-driven approach that effectively solves the pose ambiguity problem, allowing robust in-betweening performance on various lengths of missing frames. To incorporate keyframe-driven motion synthesis, we introduce a keyframe score that measures the likelihood of a frame being used as a keyframe as well as an adaptive keyframe selection method that maintains appropriate temporal distances between resulting keyframes. Additionally, we employ phase manifolds to further resolve the pose ambiguity and incorporate trajectory conditions to guide the approximate movement of the character. Comprehensive evaluations, encompassing both quantitative and qualitative analyses, were conducted to compare our method with state-of-the-art in-betweening approaches across various transition lengths. The code for the paper is available at https://github.com/seokhyeonhong/long-mib

运动中间转换因其灵活性和时间性能,已成为提高运动创建效率的一种有前途的方法。然而,由于缺失帧数增加时姿势模糊性增加,以往的中间插入方法仅限于生成短过渡。这种与长度相关的限制使优化变得困难,并进一步对目标姿势造成另一种限制,从而限制了艺术家使用的自由度。在本文中,我们介绍了一种关键帧驱动方法,它能有效解决姿势模糊问题,并在各种长度的缺失帧上实现稳健的夹帧性能。为了结合关键帧驱动的运动合成,我们引入了一种关键帧评分,用于衡量帧被用作关键帧的可能性,以及一种自适应关键帧选择方法,用于保持生成的关键帧之间适当的时间距离。此外,我们还采用相位流形来进一步解决姿势模糊的问题,并结合轨迹条件来指导角色的近似运动。我们进行了全面的评估,包括定量和定性分析,将我们的方法与各种过渡长度的先进中间处理方法进行了比较。本文代码见 https://github.com/seokhyeonhong/long-mib。
{"title":"Long-term Motion In-betweening via Keyframe Prediction","authors":"Seokhyeon Hong,&nbsp;Haemin Kim,&nbsp;Kyungmin Cho,&nbsp;Junyong Noh","doi":"10.1111/cgf.15171","DOIUrl":"https://doi.org/10.1111/cgf.15171","url":null,"abstract":"<p>Motion in-betweening has emerged as a promising approach to enhance the efficiency of motion creation due to its flexibility and time performance. However, previous in-betweening methods are limited to generating short transitions due to growing pose ambiguity when the number of missing frames increases. This length-related constraint makes the optimization hard and it further causes another constraint on the target pose, limiting the degrees of freedom for artists to use. In this paper, we introduce a keyframe-driven approach that effectively solves the pose ambiguity problem, allowing robust in-betweening performance on various lengths of missing frames. To incorporate keyframe-driven motion synthesis, we introduce a keyframe score that measures the likelihood of a frame being used as a keyframe as well as an adaptive keyframe selection method that maintains appropriate temporal distances between resulting keyframes. Additionally, we employ phase manifolds to further resolve the pose ambiguity and incorporate trajectory conditions to guide the approximate movement of the character. Comprehensive evaluations, encompassing both quantitative and qualitative analyses, were conducted to compare our method with state-of-the-art in-betweening approaches across various transition lengths. The code for the paper is available at https://github.com/seokhyeonhong/long-mib</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reconstruction of implicit surfaces from fluid particles using convolutional neural networks 利用卷积神经网络重构流体颗粒的隐含表面
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15181
C. Zhao, T. Shinar, C. Schroeder

In this paper, we present a novel network-based approach for reconstructing signed distance functions from fluid particles. The method uses a weighting kernel to transfer particles to a regular grid, which forms the input to a convolutional neural network. We propose a regression-based regularization to reduce surface noise without penalizing high-curvature features. The reconstruction exhibits improved spatial surface smoothness and temporal coherence compared with existing state of the art surface reconstruction methods. The method is insensitive to particle sampling density and robustly handles thin features, isolated particles, and sharp edges.

在本文中,我们提出了一种基于网络的新方法,用于从流体粒子中重建带符号的距离函数。该方法使用加权核将粒子转移到规则网格中,形成卷积神经网络的输入。我们提出了一种基于回归的正则化方法,以减少表面噪声,同时不影响高曲率特征。与现有的表面重建方法相比,这种重建方法的空间表面平滑度和时间连贯性都有所提高。该方法对颗粒采样密度不敏感,并能稳健地处理薄特征、孤立颗粒和尖锐边缘。
{"title":"Reconstruction of implicit surfaces from fluid particles using convolutional neural networks","authors":"C. Zhao,&nbsp;T. Shinar,&nbsp;C. Schroeder","doi":"10.1111/cgf.15181","DOIUrl":"https://doi.org/10.1111/cgf.15181","url":null,"abstract":"<div>\u0000 <p>In this paper, we present a novel network-based approach for reconstructing signed distance functions from fluid particles. The method uses a weighting kernel to transfer particles to a regular grid, which forms the input to a convolutional neural network. We propose a regression-based regularization to reduce surface noise without penalizing high-curvature features. The reconstruction exhibits improved spatial surface smoothness and temporal coherence compared with existing state of the art surface reconstruction methods. The method is insensitive to particle sampling density and robustly handles thin features, isolated particles, and sharp edges.</p>\u0000 </div>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/cgf.15181","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
PartwiseMPC: Interactive Control of Contact-Guided Motions PartwiseMPC:接触引导运动的交互式控制
IF 2.7 4区 计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING Pub Date : 2024-10-09 DOI: 10.1111/cgf.15174
N. Khoshsiyar, R. Gou, T. Zhou, S. Andrews, M. van de Panne

Physics-based character motions remain difficult to create and control. We make two contributions towards simpler specification and faster generation of physics-based control. First, we introduce a novel partwise model predictive control (MPC) method that exploits independent planning for body parts when this proves beneficial, while defaulting to whole-body motion planning when that proves to be more effective. Second, we introduce a new approach to motion specification, based on specifying an ordered set of contact keyframes. These each specify a small number of pairwise contacts between the body and the environment, and serve as loose specifications of motion strategies. Unlike regular keyframes or traditional trajectory optimization constraints, they are heavily under-constrained and have flexible timing. We demonstrate a range of challenging contact-rich motions that can be generated online at interactive rates using this framework. We further show the generalization capabilities of the method.

基于物理的角色动作仍然难以创建和控制。我们在简化规范和快速生成基于物理的控制方面做出了两项贡献。首先,我们引入了一种新颖的部分模型预测控制(MPC)方法,在证明对身体各部分有利时,利用独立规划,而在证明更有效时,默认使用全身运动规划。其次,我们引入了一种新的运动规范方法,该方法基于指定一组有序的接触关键帧。这些关键帧分别指定了身体与环境之间的少量成对接触,可作为运动策略的松散规范。与常规的关键帧或传统的轨迹优化约束不同,这些关键帧的约束严重不足,并且具有灵活的时间安排。我们展示了一系列具有挑战性的富接触运动,这些运动可以使用此框架以交互式速率在线生成。我们进一步展示了该方法的通用能力。
{"title":"PartwiseMPC: Interactive Control of Contact-Guided Motions","authors":"N. Khoshsiyar,&nbsp;R. Gou,&nbsp;T. Zhou,&nbsp;S. Andrews,&nbsp;M. van de Panne","doi":"10.1111/cgf.15174","DOIUrl":"https://doi.org/10.1111/cgf.15174","url":null,"abstract":"<p>Physics-based character motions remain difficult to create and control. We make two contributions towards simpler specification and faster generation of physics-based control. First, we introduce a novel partwise model predictive control (MPC) method that exploits independent planning for body parts when this proves beneficial, while defaulting to whole-body motion planning when that proves to be more effective. Second, we introduce a new approach to motion specification, based on specifying an ordered set of contact keyframes. These each specify a small number of pairwise contacts between the body and the environment, and serve as loose specifications of motion strategies. Unlike regular keyframes or traditional trajectory optimization constraints, they are heavily under-constrained and have flexible timing. We demonstrate a range of challenging contact-rich motions that can be generated online at interactive rates using this framework. We further show the generalization capabilities of the method.</p>","PeriodicalId":10687,"journal":{"name":"Computer Graphics Forum","volume":"43 8","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142707498","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Computer Graphics Forum
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1