Pub Date : 2024-05-01Epub Date: 2024-05-17DOI: 10.1145/3654705
Kevin Barkevich, Reynold Bailey, Gabriel J Diaz
Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.
从移动和基于视频的眼球跟踪器中估计注视方向的算法通常涉及跟踪眼球的某个特征，该特征在眼球摄像头图像中的移动方式与注视方向的变化相一致，例如瞳孔的中心或边界。由于部分遮挡和环境反射等原因，使用传统计算机视觉技术跟踪这些特征可能比较困难。虽然最近使用机器学习（ML）进行瞳孔跟踪的努力在使用分割性能的标准措施进行评估时取得了优异的结果，但人们对这些网络如何影响最终注视估计的质量却知之甚少。这项研究客观评估了几种基于 ML 的当代眼部特征跟踪方法在使用基于特征或基于模型的方法进行后续注视估计时产生的影响。衡量标准包括注视估计的准确度和精确度，以及丢失率。
{"title":"Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality.","authors":"Kevin Barkevich, Reynold Bailey, Gabriel J Diaz","doi":"10.1145/3654705","DOIUrl":"10.1145/3654705","url":null,"abstract":"<p><p>Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.</p>","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"7 2","pages":""},"PeriodicalIF":1.4,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11308822/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141908591","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A well-known issue with the widely used Smoothed Particle Hydrodynamics (SPH) method is the neighborhood deficiency. Near the surface, the SPH interpolant fails to accurately capture the underlying fields due to a lack of neighboring particles. These errors may introduce ghost forces or other visual artifacts into the simulation. In this work we investigate three different popular methods to correct the first-order spatial derivative SPH operators up to linear accuracy, namely the Kernel Gradient Correction (KGC), Moving Least Squares (MLS) and Reproducing Kernel Particle Method (RKPM). We provide a thorough, theoretical comparison in which we remark strong resemblance between the aforementioned methods. We support this by an analysis using synthetic test scenarios. Additionally, we apply the correction methods in simulations with boundary handling, viscosity, surface tension, vorticity and elastic solids to showcase the reduction or elimination of common numerical artifacts like ghost forces. Lastly, we show that incorporating the correction algorithms in a state-of-the-art SPH solver only incurs a negligible reduction in computational performance.
{"title":"A comparison of linear consistent correction methods for first-order SPH derivatives","authors":"Lukas Westhofen, S. Jeske, Jan Bender","doi":"10.1145/3606933","DOIUrl":"https://doi.org/10.1145/3606933","url":null,"abstract":"A well-known issue with the widely used Smoothed Particle Hydrodynamics (SPH) method is the neighborhood deficiency. Near the surface, the SPH interpolant fails to accurately capture the underlying fields due to a lack of neighboring particles. These errors may introduce ghost forces or other visual artifacts into the simulation. In this work we investigate three different popular methods to correct the first-order spatial derivative SPH operators up to linear accuracy, namely the Kernel Gradient Correction (KGC), Moving Least Squares (MLS) and Reproducing Kernel Particle Method (RKPM). We provide a thorough, theoretical comparison in which we remark strong resemblance between the aforementioned methods. We support this by an analysis using synthetic test scenarios. Additionally, we apply the correction methods in simulations with boundary handling, viscosity, surface tension, vorticity and elastic solids to showcase the reduction or elimination of common numerical artifacts like ghost forces. Lastly, we show that incorporating the correction algorithms in a state-of-the-art SPH solver only incurs a negligible reduction in computational performance.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 20"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44400780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fabian Löschner, José Antonio Fernández-Fernández, S. Jeske, Andreas Longva, Jan Bender
We explore micropolar materials for the simulation of volumetric deformable solids. In graphics, micropolar models have only been used in the form of one-dimensional Cosserat rods, where a rotating frame is attached to each material point on the one-dimensional centerline. By carrying this idea over to volumetric solids, every material point is associated with a microrotation, an independent degree of freedom that can be coupled to the displacement through a material's strain energy density. The additional degrees of freedom give us more control over bending and torsion modes of a material. We propose a new orthotropic micropolar curvature energy that allows us to make materials stiff to bending in specific directions. For the simulation of dynamic micropolar deformables we propose a novel incremental potential formulation with a consistent FEM discretization that is well suited for the use in physically-based animation. This allows us to easily couple micropolar deformables with dynamic collisions through a contact model inspired from the Incremental Potential Contact (IPC) approach. For the spatial discretization with FEM we discuss the challenges related to the rotational degrees of freedom and propose a scheme based on the interpolation of angular velocities followed by quaternion time integration at the quadrature points. In our evaluation we validate the consistency and accuracy of our discretization approach and demonstrate several compelling use cases for micropolar materials. This includes explicit control over bending and torsion stiffness, deformation through prescription of a volumetric curvature field and robust interaction of micropolar deformables with dynamic collisions.
{"title":"Micropolar Elasticity in Physically-Based Animation","authors":"Fabian Löschner, José Antonio Fernández-Fernández, S. Jeske, Andreas Longva, Jan Bender","doi":"10.1145/3606922","DOIUrl":"https://doi.org/10.1145/3606922","url":null,"abstract":"We explore micropolar materials for the simulation of volumetric deformable solids. In graphics, micropolar models have only been used in the form of one-dimensional Cosserat rods, where a rotating frame is attached to each material point on the one-dimensional centerline. By carrying this idea over to volumetric solids, every material point is associated with a microrotation, an independent degree of freedom that can be coupled to the displacement through a material's strain energy density. The additional degrees of freedom give us more control over bending and torsion modes of a material. We propose a new orthotropic micropolar curvature energy that allows us to make materials stiff to bending in specific directions. For the simulation of dynamic micropolar deformables we propose a novel incremental potential formulation with a consistent FEM discretization that is well suited for the use in physically-based animation. This allows us to easily couple micropolar deformables with dynamic collisions through a contact model inspired from the Incremental Potential Contact (IPC) approach. For the spatial discretization with FEM we discuss the challenges related to the rotational degrees of freedom and propose a scheme based on the interpolation of angular velocities followed by quaternion time integration at the quadrature points. In our evaluation we validate the consistency and accuracy of our discretization approach and demonstrate several compelling use cases for micropolar materials. This includes explicit control over bending and torsion stiffness, deformation through prescription of a volumetric curvature field and robust interaction of micropolar deformables with dynamic collisions.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"6 1","pages":"1 - 24"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46223097","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Efficiently solving large-scale box-constrained convex quadratic programs (QPs) is an important computational challenge in physical simulation. We propose a new multilevel preconditioning scheme based on the active-set method and combine it with modified proportioning with reduced gradient projections (MPRGP) to efficiently solve such QPs arising from pressure Poisson equations with non-negative pressure constraints in fluid animation. Our method employs a purely algebraic multigrid method to ensure the solvability of the coarser level systems and to merge only algebraically-connected components, thereby avoiding performance degradation of the preconditioner. We present a filtering scheme to efficiently apply our multilevel preconditioning only to unconstrained subsystems of the pressure Poisson system while reusing the hierarchy constructed per simulation step. We demonstrate the effectiveness of our method over previous approaches in various examples.
{"title":"A Multilevel Active-Set Preconditioner for Box-Constrained Pressure Poisson Solvers","authors":"Tetsuya Takahashi, Christopher Peter Batty","doi":"10.1145/3606939","DOIUrl":"https://doi.org/10.1145/3606939","url":null,"abstract":"Efficiently solving large-scale box-constrained convex quadratic programs (QPs) is an important computational challenge in physical simulation. We propose a new multilevel preconditioning scheme based on the active-set method and combine it with modified proportioning with reduced gradient projections (MPRGP) to efficiently solve such QPs arising from pressure Poisson equations with non-negative pressure constraints in fluid animation. Our method employs a purely algebraic multigrid method to ensure the solvability of the coarser level systems and to merge only algebraically-connected components, thereby avoiding performance degradation of the preconditioner. We present a filtering scheme to efficiently apply our multilevel preconditioning only to unconstrained subsystems of the pressure Poisson system while reusing the hierarchy constructed per simulation step. We demonstrate the effectiveness of our method over previous approaches in various examples.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":"6 1","pages":"1 - 22"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41481533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Haozhe Su, Xuan Li, Tao Xue, Chenfanfu Jiang, Mridul Aanjaneya
We present a generalized constitutive model for versatile physics simulation of inviscid fluids, Newtonian viscosity, hyperelasticity, viscoplasticity, elastoplasticity, and other physical effects that arise due to a mixture of these behaviors. The key ideas behind our formulation are the design of a generalized Kirchhoff stress tensor that can describe hyperelasticity, Newtonian viscosity and inviscid fluids, and the use of pre-projection and post-correction rules for simulating material behaviors that involve plasticity, including elastoplasticity and viscoplasticity. We show how our generalized Kirchhoff stress tensor can be coupled together into a generalized constitutive model that allows the simulation of diverse material behaviors by only changing parameter values. We present several side-by-side comparisons with physics simulations for specific constitutive models to show that our generalized model produces visually similar results. More notably, our formulation allows for inverse learning of unknown material properties directly from data using differentiable physics simulations. We present several 3D simulations to highlight the robustness of our method, even with multiple different materials. To the best of our knowledge, our approach is the first to recover the knowledge of unknown material properties without making explicit assumptions about the data.
{"title":"A Generalized Constitutive Model for Versatile MPM Simulation and Inverse Learning with Differentiable Physics","authors":"Haozhe Su, Xuan Li, Tao Xue, Chenfanfu Jiang, Mridul Aanjaneya","doi":"10.1145/3606925","DOIUrl":"https://doi.org/10.1145/3606925","url":null,"abstract":"We present a generalized constitutive model for versatile physics simulation of inviscid fluids, Newtonian viscosity, hyperelasticity, viscoplasticity, elastoplasticity, and other physical effects that arise due to a mixture of these behaviors. The key ideas behind our formulation are the design of a generalized Kirchhoff stress tensor that can describe hyperelasticity, Newtonian viscosity and inviscid fluids, and the use of pre-projection and post-correction rules for simulating material behaviors that involve plasticity, including elastoplasticity and viscoplasticity. We show how our generalized Kirchhoff stress tensor can be coupled together into a generalized constitutive model that allows the simulation of diverse material behaviors by only changing parameter values. We present several side-by-side comparisons with physics simulations for specific constitutive models to show that our generalized model produces visually similar results. More notably, our formulation allows for inverse learning of unknown material properties directly from data using differentiable physics simulations. We present several 3D simulations to highlight the robustness of our method, even with multiple different materials. To the best of our knowledge, our approach is the first to recover the knowledge of unknown material properties without making explicit assumptions about the data.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 20"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45015309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alvin Shi, Haomiao Wu, J. Parr, A. M. Darke, Theodore Kim
We present an isotropic, hyperelastic model specifically designed for the efficient simulation of tightly coiled hairs whose curl radii approach 5 mm. Our model is robust to large bends and torsions, even when they appear at the scale of the strand discretization. The terms of our model are consistently quadratic with respect to their primary variables, do not require per-edge frames or any parallel transport operators, and can efficiently take large timesteps on the order of ~1/30 of a second. Additionally, we show that it is possible to obtain fast, closed-form eigensystems for all the terms in the energy. Our eigenanalysis is sufficiently generic that it generalizes to other models. Our entirely vertex-based formulation integrates naturally with existing finite element codes, and we demonstrate its efficiency and robustness in a variety of scenarios.
{"title":"Lifted Curls","authors":"Alvin Shi, Haomiao Wu, J. Parr, A. M. Darke, Theodore Kim","doi":"10.1145/3606920","DOIUrl":"https://doi.org/10.1145/3606920","url":null,"abstract":"We present an isotropic, hyperelastic model specifically designed for the efficient simulation of tightly coiled hairs whose curl radii approach 5 mm. Our model is robust to large bends and torsions, even when they appear at the scale of the strand discretization. The terms of our model are consistently quadratic with respect to their primary variables, do not require per-edge frames or any parallel transport operators, and can efficiently take large timesteps on the order of ~1/30 of a second. Additionally, we show that it is possible to obtain fast, closed-form eigensystems for all the terms in the energy. Our eigenanalysis is sufficiently generic that it generalizes to other models. Our entirely vertex-based formulation integrates naturally with existing finite element codes, and we demonstrate its efficiency and robustness in a variety of scenarios.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 19"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43905858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Li Huang, Fan Yang, Chen Wei, Yu Ju Chen, Chun Yuan, Ming-Xin Gao
This paper introduces a hair simulator optimized for real-time applications, including console and cloud gaming, avatar live-streaming, and metaverse environments. We view the collisions between strands as a mechanism to preserve the overall volume of the hair and adopt explicit Material Point Method (MPM) to resolve the strand-strand collision. For simulating single-strand behavior, a semi-implicit Discrete Elastic Rods (DER) model is used. We build upon a highly efficient GPU MPM framework recently presented by Fei et al. [2021b] and propose several schemes to largely improve the performance of building and solving the semi-implicit DER systems on GPU. We demonstrate the efficiency of our pipeline by a few practical scenes that achieve up to 260 frames-per-second (FPS) with more than two thousand simulated strands on Nvidia GeForce RTX 3080.
{"title":"Towards Realtime","authors":"Li Huang, Fan Yang, Chen Wei, Yu Ju Chen, Chun Yuan, Ming-Xin Gao","doi":"10.1145/3606937","DOIUrl":"https://doi.org/10.1145/3606937","url":null,"abstract":"This paper introduces a hair simulator optimized for real-time applications, including console and cloud gaming, avatar live-streaming, and metaverse environments. We view the collisions between strands as a mechanism to preserve the overall volume of the hair and adopt explicit Material Point Method (MPM) to resolve the strand-strand collision. For simulating single-strand behavior, a semi-implicit Discrete Elastic Rods (DER) model is used. We build upon a highly efficient GPU MPM framework recently presented by Fei et al. [2021b] and propose several schemes to largely improve the performance of building and solving the semi-implicit DER systems on GPU. We demonstrate the efficiency of our pipeline by a few practical scenes that achieve up to 260 frames-per-second (FPS) with more than two thousand simulated strands on Nvidia GeForce RTX 3080.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 18"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43818989","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Skinning transformations enable digital characters to be animated with minimal user input. Physics simulations can improve the detailed dynamic movement of an animated character; however, such details are typically added in the post-processing stage after the overall animation is specified. We propose a novel interactive framework that unifies skinning transformations and kinematic simulations using position-based dynamics (PBD). Our framework allows an arbitrarily skinned character to be partially manipulated by the user, and a kinematic physics solver automatically complements the behavior of the entire character. This is achieved by introducing new steps into the PBD algorithm: (i) lightweight optimization to identify the skinning transformations, which is similar to inverse kinematics, and (ii) a position-based constraint to restrict the PBD solver to the complementary subspace of the skinning deformation. Our method combines the best of the two methods: the controllability and shape preservation of the skinning transformation, and the efficiency, simplicity, and unconditional stability of the PBD solver. Our interface allows novices to create vibrant animations without tedious editing.
{"title":"Two-Way Coupling of Skinning Transformations and Position Based Dynamics","authors":"Yuehua Wu, Nobuyuki Umetani","doi":"10.1145/3606930","DOIUrl":"https://doi.org/10.1145/3606930","url":null,"abstract":"Skinning transformations enable digital characters to be animated with minimal user input. Physics simulations can improve the detailed dynamic movement of an animated character; however, such details are typically added in the post-processing stage after the overall animation is specified. We propose a novel interactive framework that unifies skinning transformations and kinematic simulations using position-based dynamics (PBD). Our framework allows an arbitrarily skinned character to be partially manipulated by the user, and a kinematic physics solver automatically complements the behavior of the entire character. This is achieved by introducing new steps into the PBD algorithm: (i) lightweight optimization to identify the skinning transformations, which is similar to inverse kinematics, and (ii) a position-based constraint to restrict the PBD solver to the complementary subspace of the skinning deformation. Our method combines the best of the two methods: the controllability and shape preservation of the skinning transformation, and the efficiency, simplicity, and unconditional stability of the PBD solver. Our interface allows novices to create vibrant animations without tedious editing.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 18"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49447777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kaixiang Xie, Pei Xu, S. Andrews, V. Zordan, P. Kry
Deep reinforcement learning (DRL) methods have demonstrated impressive results for skilled motion synthesis of physically based characters, and while these methods perform well in terms of tracking reference motions or achieving complex tasks, several concerns arise when evaluating the naturalness of the motion. In this paper, we conduct a preliminary study of specific quantitative metrics for measuring the naturalness of motion produced by DRL control policies beyond their visual appearance. Namely, we propose to study the stiffness of the control policy, in anticipation that it will influence how the character behaves in the presence of external perturbation. Second, we establish two baselines for strength that allow evaluating the use of joint torques in comparison to human performance. Third, we propose the study of variability to reveal the unnatural precision of control policies and how they compare to real human motion. In sum, we aim to establish repeatable measures to assess the naturalness of control policies produced by DRL methods, and we present a set of comparisons from state-of-the-art systems. Finally, we propose simple modifications to improve realism on these axes.
{"title":"Too Stiff, Too Strong, Too Smart","authors":"Kaixiang Xie, Pei Xu, S. Andrews, V. Zordan, P. Kry","doi":"10.1145/3606935","DOIUrl":"https://doi.org/10.1145/3606935","url":null,"abstract":"Deep reinforcement learning (DRL) methods have demonstrated impressive results for skilled motion synthesis of physically based characters, and while these methods perform well in terms of tracking reference motions or achieving complex tasks, several concerns arise when evaluating the naturalness of the motion. In this paper, we conduct a preliminary study of specific quantitative metrics for measuring the naturalness of motion produced by DRL control policies beyond their visual appearance. Namely, we propose to study the stiffness of the control policy, in anticipation that it will influence how the character behaves in the presence of external perturbation. Second, we establish two baselines for strength that allow evaluating the use of joint torques in comparison to human performance. Third, we propose the study of variability to reveal the unnatural precision of control policies and how they compare to real human motion. In sum, we aim to establish repeatable measures to assess the naturalness of control policies produced by DRL methods, and we present a set of comparisons from state-of-the-art systems. Finally, we propose simple modifications to improve realism on these axes.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 17"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44235174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hitoshi Teshima, Naoki Wake, Diego Thomas, Yuta Nakashima, Hiroshi Kawasaki, K. Ikeuchi
Recent increase of remote-work, online meeting and tele-operation task makes people find that gesture for avatars and communication robots is more important than we have thought. It is one of the key factors to achieve smooth and natural communication between humans and AI systems and has been intensively researched. Current gesture generation methods are mostly based on deep neural network using text, audio and other information as the input, however, they generate gestures mainly based on audio, which is called a beat gesture. Although the ratio of the beat gesture is more than 70% of actual human gestures, content based gestures sometimes play an important role to make avatars more realistic and human-like. In this paper, we propose a attention-based contrastive learning for text-to-gesture (ACT2G), where generated gestures represent content of the text by estimating attention weight for each word from the input text. In the method, since text and gesture features calculated by the attention weight are mapped to the same latent space by contrastive learning, once text is given as input, the network outputs a feature vector which can be used to generate gestures related to the content. User study confirmed that the gestures generated by ACT2G were better than existing methods. In addition, it was demonstrated that wide variation of gestures were generated from the same text by changing attention weights by creators.
{"title":"ACT2G","authors":"Hitoshi Teshima, Naoki Wake, Diego Thomas, Yuta Nakashima, Hiroshi Kawasaki, K. Ikeuchi","doi":"10.1145/3606940","DOIUrl":"https://doi.org/10.1145/3606940","url":null,"abstract":"Recent increase of remote-work, online meeting and tele-operation task makes people find that gesture for avatars and communication robots is more important than we have thought. It is one of the key factors to achieve smooth and natural communication between humans and AI systems and has been intensively researched. Current gesture generation methods are mostly based on deep neural network using text, audio and other information as the input, however, they generate gestures mainly based on audio, which is called a beat gesture. Although the ratio of the beat gesture is more than 70% of actual human gestures, content based gestures sometimes play an important role to make avatars more realistic and human-like. In this paper, we propose a attention-based contrastive learning for text-to-gesture (ACT2G), where generated gestures represent content of the text by estimating attention weight for each word from the input text. In the method, since text and gesture features calculated by the attention weight are mapped to the same latent space by contrastive learning, once text is given as input, the network outputs a feature vector which can be used to generate gestures related to the content. User study confirmed that the gestures generated by ACT2G were better than existing methods. In addition, it was demonstrated that wide variation of gestures were generated from the same text by changing attention weights by creators.","PeriodicalId":74536,"journal":{"name":"Proceedings of the ACM on computer graphics and interactive techniques","volume":" ","pages":"1 - 17"},"PeriodicalIF":0.0,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45872962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号