Displays最新文献_第2页

ITSRN++: Stronger and better implicit transformer network for screen content image continuous super-resolution

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-22 DOI: 10.1016/j.displa.2024.102865

Sheng Shen , Huanjing Yue , Kun Li , Jingyu Yang

Nowadays, online screen sharing and remote cooperation are becoming ubiquitous. However, the screen content may be downsampled and compressed during transmission, while it may be displayed on large screens or the users would zoom in for detail observation at the receiver side. Therefore, developing a strong and effective screen content image (SCI) super-resolution (SR) method is demanded. We observe that the weight-sharing upsampler (such as deconvolution or pixel shuffle) could be harmful to sharp and thin edges in SCIs, and the fixed scale upsampler makes it inflexible to fit screens with various sizes. To solve this problem, we propose an implicit transformer network for SCI continuous SR (termed as ITSRN++). Specifically, we propose a modulation based transformer as the upsampler, which modulates the pixel features in discrete space via a periodic nonlinear function to generate features in continuous space. To better restore the high-frequency details in screen content images, we further propose dual branch block (DBB) as the feature extraction backbone, where convolution and attention branches are utilized parallelly on the same linear transformed value. Besides, we construct a large-scale SCI2K dataset to facilitate the research on SCI SR. Experimental results on nine datasets demonstrate that the proposed method achieves state-of-the-art performance for SCI SR and also works well for natural image SR.

{"title":"ITSRN++: Stronger and better implicit transformer network for screen content image continuous super-resolution","authors":"Sheng Shen , Huanjing Yue , Kun Li , Jingyu Yang","doi":"10.1016/j.displa.2024.102865","DOIUrl":"10.1016/j.displa.2024.102865","url":null,"abstract":"<div><div>Nowadays, online screen sharing and remote cooperation are becoming ubiquitous. However, the screen content may be downsampled and compressed during transmission, while it may be displayed on large screens or the users would zoom in for detail observation at the receiver side. Therefore, developing a strong and effective screen content image (SCI) super-resolution (SR) method is demanded. We observe that the weight-sharing upsampler (such as deconvolution or pixel shuffle) could be harmful to sharp and thin edges in SCIs, and the fixed scale upsampler makes it inflexible to fit screens with various sizes. To solve this problem, we propose an implicit transformer network for SCI continuous SR (termed as ITSRN++). Specifically, we propose a modulation based transformer as the upsampler, which modulates the pixel features in discrete space via a periodic nonlinear function to generate features in continuous space. To better restore the high-frequency details in screen content images, we further propose dual branch block (DBB) as the feature extraction backbone, where convolution and attention branches are utilized parallelly on the same linear transformed value. Besides, we construct a large-scale SCI2K dataset to facilitate the research on SCI SR. Experimental results on nine datasets demonstrate that the proposed method achieves state-of-the-art performance for SCI SR and also works well for natural image SR.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"86 ","pages":"Article 102865"},"PeriodicalIF":3.7,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743274","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Orientation stabilization of light-controlling layers using liquid crystal polymers 利用液晶聚合物稳定光控层的方向

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-21 DOI: 10.1016/j.displa.2024.102893

Xinmin Yu , Tangwu Li , Jingxin Sang , Ming Xiao , Jianhua Shang , Jiatong Sun

Photo-alignment technology provides an advanced optical alignment method and high alignment quality compared to conventional rubbing techniques. However, challenges of photo-stability and response speed have hindered their practical application. This work is concerned with the stability of the photo-alignment layer, with azo dyes (SD1) employed as the photo-alignment layer material. Two solutions were investigated: the passivation of SD1 films with reactive mesogen (RM) material RM257 and the utilization of polymer–azo dye composites in lieu of the photo-alignment layer. Experimental methods, including spin-coating and UV polymerization, were employed, while optimal monomer concentrations were determined by photo-stability tests. The results demonstrate that the passivation layer enhances photo-stability. Atomic force microscopy (AFM) is employed to assess film roughness and enhance film quality. Furthermore, electro-optical tests indicate that the approach has no negative impact on the performance of LCD. These findings provide a more optimized approach to stabilizing liquid crystal (LC) alignment for advanced display technologies.

与传统的摩擦技术相比，光对准技术提供了一种先进的光学对准方法和较高的对准质量。然而，光稳定性和响应速度方面的挑战阻碍了其实际应用。这项研究关注光对准层的稳定性，采用偶氮染料（SD1）作为光对准层材料。研究了两种解决方案：用反应介质（RM）材料 RM257 对 SD1 薄膜进行钝化，以及用聚合物-偶氮染料复合材料代替光对准层。实验采用了旋涂和紫外聚合等方法，并通过光稳定性测试确定了最佳单体浓度。结果表明，钝化层可增强光稳定性。原子力显微镜（AFM）被用来评估薄膜粗糙度和提高薄膜质量。此外，电光测试表明，这种方法对液晶显示器的性能没有负面影响。这些发现为稳定先进显示技术的液晶（LC）排列提供了一种更优化的方法。

{"title":"Orientation stabilization of light-controlling layers using liquid crystal polymers","authors":"Xinmin Yu , Tangwu Li , Jingxin Sang , Ming Xiao , Jianhua Shang , Jiatong Sun","doi":"10.1016/j.displa.2024.102893","DOIUrl":"10.1016/j.displa.2024.102893","url":null,"abstract":"<div><div>Photo-alignment technology provides an advanced optical alignment method and high alignment quality compared to conventional rubbing techniques. However, challenges of photo-stability and response speed have hindered their practical application. This work is concerned with the stability of the photo-alignment layer, with azo dyes (SD1) employed as the photo-alignment layer material. Two solutions were investigated: the passivation of SD1 films with reactive mesogen (RM) material RM257 and the utilization of polymer–azo dye composites in lieu of the photo-alignment layer. Experimental methods, including spin-coating and UV polymerization, were employed, while optimal monomer concentrations were determined by photo-stability tests. The results demonstrate that the passivation layer enhances photo-stability. Atomic force microscopy (AFM) is employed to assess film roughness and enhance film quality. Furthermore, electro-optical tests indicate that the approach has no negative impact on the performance of LCD. These findings provide a more optimized approach to stabilizing liquid crystal (LC) alignment for advanced display technologies.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"86 ","pages":"Article 102893"},"PeriodicalIF":3.7,"publicationDate":"2024-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Frequency-spatial interaction network for gaze estimation 用于凝视估计的频率-空间交互网络

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-21 DOI: 10.1016/j.displa.2024.102878

Yuanning Jia , Zhi Liu , Ying Lv , Xiaofeng Lu , Xuefeng Liu , Jie Chen

Gaze estimation is a fundamental task in the field of computer vision, which determines the direction a person is looking at. With advancements in Convolutional Neural Networks (CNNs) and the availability of large-scale datasets, appearance-based models have made significant progress. Nonetheless, CNNs exhibit limitations in extracting global information from features, resulting in a constraint on gaze estimation performance. Inspired by the properties of the Fourier transform in signal processing, we propose the Frequency-Spatial Interaction network for Gaze estimation (FSIGaze), which integrates residual modules and Frequency-Spatial Synergistic (FSS) modules. To be specific, its FSS module is a dual-branch structure with a spatial branch and a frequency branch. The frequency branch employs Fast Fourier Transformation to transfer a latent representation to the frequency domain and applies adaptive frequency filter to achieve an image-size receptive field. The spatial branch, on the other hand, can extract local detailed features. Acknowledging the synergistic benefits of global and local information in gaze estimation, we introduce a Dual-domain Interaction Block (DIB) to enhance the capability of the model. Furthermore, we implement a multi-task learning strategy, incorporating eye region detection as an auxiliary task to refine facial features. Extensive experiments demonstrate that our model surpasses other state-of-the-art gaze estimation models on three three-dimensional (3D) datasets and delivers competitive results on two two-dimensional (2D) datasets.

凝视估计是计算机视觉领域的一项基本任务，它能确定一个人正在注视的方向。随着卷积神经网络（CNN）的进步和大规模数据集的可用性，基于外观的模型取得了重大进展。然而，卷积神经网络在从特征中提取全局信息方面表现出局限性，从而制约了凝视估计的性能。受信号处理中傅立叶变换特性的启发，我们提出了用于注视估计的频率-空间交互网络（FSIGaze），它集成了残差模块和频率-空间协同（FSS）模块。具体来说，其 FSS 模块是一个双分支结构，包括空间分支和频率分支。频率分支采用快速傅里叶变换将潜在表征转移到频域，并应用自适应频率滤波器实现图像大小的感受野。空间分支则可以提取局部细节特征。考虑到全局和局部信息在凝视估计中的协同优势，我们引入了双域交互块（DIB）来增强模型的能力。此外，我们还实施了多任务学习策略，将眼部区域检测作为完善面部特征的辅助任务。广泛的实验证明，我们的模型在三个三维（3D）数据集上超越了其他最先进的凝视估计模型，在两个二维（2D）数据集上也取得了具有竞争力的结果。

{"title":"Frequency-spatial interaction network for gaze estimation","authors":"Yuanning Jia , Zhi Liu , Ying Lv , Xiaofeng Lu , Xuefeng Liu , Jie Chen","doi":"10.1016/j.displa.2024.102878","DOIUrl":"10.1016/j.displa.2024.102878","url":null,"abstract":"<div><div>Gaze estimation is a fundamental task in the field of computer vision, which determines the direction a person is looking at. With advancements in Convolutional Neural Networks (CNNs) and the availability of large-scale datasets, appearance-based models have made significant progress. Nonetheless, CNNs exhibit limitations in extracting global information from features, resulting in a constraint on gaze estimation performance. Inspired by the properties of the Fourier transform in signal processing, we propose the Frequency-Spatial Interaction network for Gaze estimation (FSIGaze), which integrates residual modules and Frequency-Spatial Synergistic (FSS) modules. To be specific, its FSS module is a dual-branch structure with a spatial branch and a frequency branch. The frequency branch employs Fast Fourier Transformation to transfer a latent representation to the frequency domain and applies adaptive frequency filter to achieve an image-size receptive field. The spatial branch, on the other hand, can extract local detailed features. Acknowledging the synergistic benefits of global and local information in gaze estimation, we introduce a Dual-domain Interaction Block (DIB) to enhance the capability of the model. Furthermore, we implement a multi-task learning strategy, incorporating eye region detection as an auxiliary task to refine facial features. Extensive experiments demonstrate that our model surpasses other state-of-the-art gaze estimation models on three three-dimensional (3D) datasets and delivers competitive results on two two-dimensional (2D) datasets.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"86 ","pages":"Article 102878"},"PeriodicalIF":3.7,"publicationDate":"2024-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706178","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AdapSyn: Anomaly detection based on triplet training with adaptive anomaly synthesis AdapSyn：基于自适应异常合成的三元组训练的异常检测

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-18 DOI: 10.1016/j.displa.2024.102885

Shijie Zhou , Chunyu Lin , Zisong Chen , Baoqing Guo , Yao Zhao

In the field of anomaly detection (AD), Few-Shot Anomaly Detection (FSAD) has gained significant attention in recent years. The goal of anomaly detection is to identify defects at the image level and localize them at the pixel level. Including defect data in training helps to improve model performance, and FSAD methods require a large amount of data to achieve better results. However, defect data is often difficult to obtain in experiments and applications. This paper proposes a more realistic method for simulating anomaly data, incorporating the synthesized anomaly data into the training process and applying it to the FSAD domain through multi-class mixed training. The anomaly data generated using our synthesis method closely resembles real anomalies. The corresponding anomaly synthesis method synthesizes the anomaly data from normal samples for the adaptively selected polygonal mesh region. We enhance the model’s ability to distinguish between positive and negative samples by incorporating synthesized anomaly data and normal data as triplets during training. This results in that more detailed features for normal samples will be noticed. During the testing phase, we obtain the feature distribution of normal images for a few unknown class normal samples to quickly adapt to the detection task of new categories. The effectiveness of the anomaly synthesis method was validated through experiments. Comparisons with advanced methods in the FSAD domain demonstrated that our method achieved competitive performance.

近年来，在异常检测（AD）领域，少镜头异常检测（FSAD）备受关注。异常检测的目标是在图像层面识别缺陷，并在像素层面对缺陷进行定位。在训练中加入缺陷数据有助于提高模型性能，而 FSAD 方法需要大量数据才能取得更好的效果。然而，缺陷数据在实验和应用中往往难以获得。本文提出了一种更切合实际的模拟异常数据的方法，将合成的异常数据纳入训练过程，并通过多类混合训练将其应用于 FSAD 领域。使用我们的合成方法生成的异常数据与真实异常数据非常相似。相应的异常合成方法从自适应选择的多边形网格区域的正常样本中合成异常数据。在训练过程中，我们将合成的异常数据和正常数据作为三元组，从而增强了模型区分正样本和负样本的能力。这样，正常样本的详细特征就会被注意到。在测试阶段，我们会获取一些未知类别正常样本的正常图像特征分布，以快速适应新类别的检测任务。通过实验验证了异常合成方法的有效性。与 FSAD 领域先进方法的比较表明，我们的方法取得了具有竞争力的性能。

{"title":"AdapSyn: Anomaly detection based on triplet training with adaptive anomaly synthesis","authors":"Shijie Zhou , Chunyu Lin , Zisong Chen , Baoqing Guo , Yao Zhao","doi":"10.1016/j.displa.2024.102885","DOIUrl":"10.1016/j.displa.2024.102885","url":null,"abstract":"<div><div>In the field of anomaly detection (AD), Few-Shot Anomaly Detection (FSAD) has gained significant attention in recent years. The goal of anomaly detection is to identify defects at the image level and localize them at the pixel level. Including defect data in training helps to improve model performance, and FSAD methods require a large amount of data to achieve better results. However, defect data is often difficult to obtain in experiments and applications. This paper proposes a more realistic method for simulating anomaly data, incorporating the synthesized anomaly data into the training process and applying it to the FSAD domain through multi-class mixed training. The anomaly data generated using our synthesis method closely resembles real anomalies. The corresponding anomaly synthesis method synthesizes the anomaly data from normal samples for the adaptively selected polygonal mesh region. We enhance the model’s ability to distinguish between positive and negative samples by incorporating synthesized anomaly data and normal data as triplets during training. This results in that more detailed features for normal samples will be noticed. During the testing phase, we obtain the feature distribution of normal images for a few unknown class normal samples to quickly adapt to the detection task of new categories. The effectiveness of the anomaly synthesis method was validated through experiments. Comparisons with advanced methods in the FSAD domain demonstrated that our method achieved competitive performance.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"86 ","pages":"Article 102885"},"PeriodicalIF":3.7,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706176","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

3D-MSFC: A 3D multi-scale features compression method for object detection 3D-MSFC：用于物体检测的三维多尺度特征压缩方法

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-17 DOI: 10.1016/j.displa.2024.102880

Zhengxin Li , Chongzhen Tian , Hui Yuan , Xin Lu , Hossein Malekmohamadi

As machine vision tasks rapidly evolve, a new concept of compression, namely video coding for machines (VCM), has emerged. However, current VCM methods are only suitable for 2D machine vision tasks. With the popularization of autonomous driving, the demand for 3D machine vision tasks has significantly increased, leading to an explosive growth in LiDAR data that requires efficient transmission. To address this need, we propose a machine vision-based point cloud coding paradigm inspired by VCM. Specifically, we introduce a 3D multi-scale features compression (3D-MSFC) method, tailored for 3D object detection. Experimental results demonstrate that 3D-MSFC achieves less than a 3% degradation in object detection accuracy at a compression ratio of 2796

\times

. Furthermore, its low-profile variant, 3D-MSFC-L, achieves less than a 2% degradation in accuracy at a compression ratio of 463

\times

. The above results indicate that our proposed method can provide an ultra-high compression ratio while ensuring no significant drop in accuracy, greatly reducing the amount of data required for transmission during each detection. This can significantly lower bandwidth consumption and save substantial costs in application scenarios such as smart cities.

随着机器视觉任务的快速发展，出现了一种新的压缩概念，即机器视频编码（VCM）。然而，目前的 VCM 方法仅适用于二维机器视觉任务。随着自动驾驶的普及，三维机器视觉任务的需求大幅增加，导致需要高效传输的激光雷达数据爆炸式增长。为了满足这一需求，我们受 VCM 的启发，提出了一种基于机器视觉的点云编码范例。具体来说，我们引入了一种三维多尺度特征压缩（3D-MSFC）方法，专门用于三维物体检测。实验结果表明，在压缩率为 2796× 的情况下，3D-MSFC 的物体检测准确率下降不到 3%。此外，其低调变体 3D-MSFC-L 在压缩比为 463× 时，精度下降不到 2%。上述结果表明，我们提出的方法可以提供超高的压缩比，同时确保精度不会明显下降，大大减少了每次检测所需的传输数据量。这可以大大降低带宽消耗，为智慧城市等应用场景节省大量成本。

{"title":"3D-MSFC: A 3D multi-scale features compression method for object detection","authors":"Zhengxin Li , Chongzhen Tian , Hui Yuan , Xin Lu , Hossein Malekmohamadi","doi":"10.1016/j.displa.2024.102880","DOIUrl":"10.1016/j.displa.2024.102880","url":null,"abstract":"<div><div>As machine vision tasks rapidly evolve, a new concept of compression, namely video coding for machines (VCM), has emerged. However, current VCM methods are only suitable for 2D machine vision tasks. With the popularization of autonomous driving, the demand for 3D machine vision tasks has significantly increased, leading to an explosive growth in LiDAR data that requires efficient transmission. To address this need, we propose a machine vision-based point cloud coding paradigm inspired by VCM. Specifically, we introduce a 3D multi-scale features compression (3D-MSFC) method, tailored for 3D object detection. Experimental results demonstrate that 3D-MSFC achieves less than a 3% degradation in object detection accuracy at a compression ratio of 2796<span><math><mo>×</mo></math></span>. Furthermore, its low-profile variant, 3D-MSFC-L, achieves less than a 2% degradation in accuracy at a compression ratio of 463<span><math><mo>×</mo></math></span>. The above results indicate that our proposed method can provide an ultra-high compression ratio while ensuring no significant drop in accuracy, greatly reducing the amount of data required for transmission during each detection. This can significantly lower bandwidth consumption and save substantial costs in application scenarios such as smart cities.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"85 ","pages":"Article 102880"},"PeriodicalIF":3.7,"publicationDate":"2024-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706428","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring the brain physiological activity and quantified assessment of VR cybersickness using EEG signals 利用脑电信号探索大脑生理活动并量化评估 VR 网络晕眩症

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-17 DOI: 10.1016/j.displa.2024.102879

Mutian Liu , Banghua Yang , Peng Zan , Luting Chen , Baozeng Wang , Xinxing Xia

Cybersickness in virtual reality (VR) significantly impedes user experience enhancement. Sensory conflict theory explains cybersickness as arising from brain conflicts, making a brain physiology-based examination essential for cybersickness research. In this study, we analyze the impact of cybersickness on brain neural activity and achieve the quantified assessment of cybersickness using cybersickness-related electroencephalography (EEG) data. We conduct a cybersickness induction experiment by view rotation and simultaneously collect EEG signals from 36 subjects. We investigate both brain functional connectivity and neural oscillation power aiming to demonstrate the specific variation trends of brain physiological characteristics across varying degrees of cybersickness. Filtering raw EEG highlights cybersickness-related features, facilitating the quantified assessment of cybersickness through a Convolutional Temporal-Transformer Network, named CTTNet. The results demonstrate that cybersickness leads to a significant reduction in the power of the beta and gamma frequency bands in the frontal lobe, accompanied by weakened internal connectivity within these bands. Conversely, as the severity of cybersickness increases, connectivity between the posterior brain regions and the frontal lobe in the mid-to-high frequency bands is enhanced. CTTNet achieves accurate evaluation of cybersickness by effectively capturing temporal-spatial EEG features and the long-term temporal dependencies of cybersickness. A significant and robust relationship between cybersickness and cerebral physiological characteristics is demonstrated. These findings hold the potential to offer valuable insights for the future real-time assessment and mitigation of cybersickness, particularly focusing on brain dynamics.

虚拟现实（VR）中的 "晕机 "现象严重阻碍了用户体验的提升。感官冲突理论认为晕机是由大脑冲突引起的，因此基于大脑生理学的检查对晕机研究至关重要。在本研究中，我们分析了网络晕眩对大脑神经活动的影响，并利用与网络晕眩相关的脑电图（EEG）数据实现了对网络晕眩的量化评估。我们通过视图旋转进行晕机诱导实验，同时收集 36 名受试者的脑电图信号。我们对大脑功能连接和神经振荡功率进行了研究，旨在展示不同程度的晕机状态下大脑生理特征的具体变化趋势。通过对原始脑电图进行过滤，可突出与晕机相关的特征，从而有助于通过卷积时变网络（名为 CTTNet）对晕机进行量化评估。研究结果表明，晕网导致额叶β和γ频段的功率显著降低，同时这些频段的内部连接性减弱。相反，随着晕机症严重程度的增加，大脑后部区域与额叶之间在中高频段的连接性增强。CTTNet 通过有效捕捉时空脑电图特征和晕机症的长期时间依赖性，实现了对晕机症的准确评估。结果表明，晕机与大脑生理特征之间存在重要而稳健的关系。这些发现有可能为未来实时评估和缓解晕机症提供有价值的见解，尤其是侧重于大脑动态的评估和缓解。

{"title":"Exploring the brain physiological activity and quantified assessment of VR cybersickness using EEG signals","authors":"Mutian Liu , Banghua Yang , Peng Zan , Luting Chen , Baozeng Wang , Xinxing Xia","doi":"10.1016/j.displa.2024.102879","DOIUrl":"10.1016/j.displa.2024.102879","url":null,"abstract":"<div><div>Cybersickness in virtual reality (VR) significantly impedes user experience enhancement. Sensory conflict theory explains cybersickness as arising from brain conflicts, making a brain physiology-based examination essential for cybersickness research. In this study, we analyze the impact of cybersickness on brain neural activity and achieve the quantified assessment of cybersickness using cybersickness-related electroencephalography (EEG) data. We conduct a cybersickness induction experiment by view rotation and simultaneously collect EEG signals from 36 subjects. We investigate both brain functional connectivity and neural oscillation power aiming to demonstrate the specific variation trends of brain physiological characteristics across varying degrees of cybersickness. Filtering raw EEG highlights cybersickness-related features, facilitating the quantified assessment of cybersickness through a Convolutional Temporal-Transformer Network, named CTTNet. The results demonstrate that cybersickness leads to a significant reduction in the power of the beta and gamma frequency bands in the frontal lobe, accompanied by weakened internal connectivity within these bands. Conversely, as the severity of cybersickness increases, connectivity between the posterior brain regions and the frontal lobe in the mid-to-high frequency bands is enhanced. CTTNet achieves accurate evaluation of cybersickness by effectively capturing temporal-spatial EEG features and the long-term temporal dependencies of cybersickness. A significant and robust relationship between cybersickness and cerebral physiological characteristics is demonstrated. These findings hold the potential to offer valuable insights for the future real-time assessment and mitigation of cybersickness, particularly focusing on brain dynamics.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"85 ","pages":"Article 102879"},"PeriodicalIF":3.7,"publicationDate":"2024-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

VR-empowered interior design: Enhancing efficiency and quality through immersive experiences VR 助力室内设计：通过身临其境的体验提高效率和质量

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-16 DOI: 10.1016/j.displa.2024.102887

Pengjun Wu , Yao Liu , Huijie Chen , Xiaowen Li , Hui Wang

Traditional interior design methods often struggle to meet the increasingly diverse needs of modern users. As a result, virtual reality (VR) technology has emerged as an innovative solution. This study explores the application of VR technology in interior design (ID), with a focus on enhancing efficiency and quality in the design process. To achieve this, we developed an ID platform based on VR technology and employed Level of Detail (LOD) techniques for 3D model simplification, significantly improving rendering efficiency and effectively reducing rendering times. Performance testing results indicate that the average rendering times decreased from 721.50 ms, 811.20 ms, and 748.10 ms to 335.80 ms, 431.10 ms, and 371.30 ms, respectively, while the frame rates (FPS) increased from 100.80, 85.00, and 93.40 to 107.50, 91.00, and 100.20. These experimental results demonstrate that VR-based design not only accelerates the design process but also significantly enhances the visual quality of interior spaces, providing users with a more immersive experience.

传统的室内设计方法往往难以满足现代用户日益多样化的需求。因此，虚拟现实（VR）技术作为一种创新解决方案应运而生。本研究探讨了虚拟现实技术在室内设计（ID）中的应用，重点是提高设计过程的效率和质量。为此，我们开发了一个基于 VR 技术的 ID 平台，并采用细节水平（LOD）技术简化三维模型，显著提高了渲染效率，有效缩短了渲染时间。性能测试结果表明，平均渲染时间分别从 721.50 毫秒、811.20 毫秒和 748.10 毫秒减少到 335.80 毫秒、431.10 毫秒和 371.30 毫秒，而帧速率（FPS）则从 100.80、85.00 和 93.40 提高到 107.50、91.00 和 100.20。这些实验结果表明，基于 VR 的设计不仅能加快设计过程，还能显著提升室内空间的视觉质量，为用户提供更加身临其境的体验。

引用次数: 0

SPFont: Stroke potential features embedded GAN for Chinese calligraphy font generation SPFont：用于生成中国书法字体的嵌入笔画潜力特征的 GAN

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-16 DOI: 10.1016/j.displa.2024.102876

Fangmei Chen, Chen Wang, Xingchen Yao, Fuming Sun

Chinese calligraphy font generation is an extremely challenging problem. Firstly, Chinese calligraphy fonts have complex structures. The accuracy and artistic quality of the generated fonts will be affected by the order and layout of the strokes as well as the relationships between them. Secondly, the number of Chinese characters is large, but existing calligraphy works are scarce. Hence, it is difficult to establish a comprehensive and high-quality Chinese calligraphy dataset. In this paper, we propose an unsupervised calligraphy font generation network SPFont. It is based on a generative adversarial network (GAN) framework. The generator includes a style feature encoder, a content feature encoder, a stroke potential feature fusion module (SPFM) and a decoder. The SPFM module, by overlaying lower-level style and content features, better preserves fine details of the font such as stroke thickness, curve shapes and other characteristics. The SPFM module and the extracted style features are fused and then fed into the decoder, allowing it to consider the influence of style, content and stroke potential simultaneously during the generation process. Experimental results demonstrate that our model generates Chinese calligraphy fonts with higher quality compared to previous methods.

中国书法字体生成是一个极具挑战性的问题。首先，中国书法字体结构复杂。笔画的顺序和布局以及它们之间的关系会影响生成字体的准确性和艺术质量。其次，汉字数量庞大，但现有的书法作品却很少。因此，很难建立一个全面而高质量的汉字书法数据集。本文提出了一种无监督书法字体生成网络 SPFont。它基于生成对抗网络（GAN）框架。生成器包括风格特征编码器、内容特征编码器、笔画潜在特征融合模块（SPFM）和解码器。SPFM 模块通过叠加较低级别的样式和内容特征，能更好地保留字体的细节，如笔画粗细、曲线形状和其他特征。SPFM 模块和提取的风格特征融合后输入解码器，使其在生成过程中同时考虑风格、内容和笔势的影响。实验结果表明，与之前的方法相比，我们的模型生成的书法字体质量更高。

{"title":"SPFont: Stroke potential features embedded GAN for Chinese calligraphy font generation","authors":"Fangmei Chen, Chen Wang, Xingchen Yao, Fuming Sun","doi":"10.1016/j.displa.2024.102876","DOIUrl":"10.1016/j.displa.2024.102876","url":null,"abstract":"<div><div>Chinese calligraphy font generation is an extremely challenging problem. Firstly, Chinese calligraphy fonts have complex structures. The accuracy and artistic quality of the generated fonts will be affected by the order and layout of the strokes as well as the relationships between them. Secondly, the number of Chinese characters is large, but existing calligraphy works are scarce. Hence, it is difficult to establish a comprehensive and high-quality Chinese calligraphy dataset. In this paper, we propose an unsupervised calligraphy font generation network SPFont. It is based on a generative adversarial network (GAN) framework. The generator includes a style feature encoder, a content feature encoder, a stroke potential feature fusion module (SPFM) and a decoder. The SPFM module, by overlaying lower-level style and content features, better preserves fine details of the font such as stroke thickness, curve shapes and other characteristics. The SPFM module and the extracted style features are fused and then fed into the decoder, allowing it to consider the influence of style, content and stroke potential simultaneously during the generation process. Experimental results demonstrate that our model generates Chinese calligraphy fonts with higher quality compared to previous methods.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"85 ","pages":"Article 102876"},"PeriodicalIF":3.7,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HHGraphSum: Hierarchical heterogeneous graph learning for extractive document summarization HHGraphSum：用于提取文档摘要的分层异构图学习

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-16 DOI: 10.1016/j.displa.2024.102884

Pengyi Hao , Cunqi Wu , Cong Bai

Extractive summarization aims to select important sentences from the document to generate a summary. However, current extractive document summarization methods fail to fully consider the semantic information among sentences and the various relations in the entire document. Therefore, a novel end-to-end framework named hierarchical heterogeneous graph learning for document summarization (HHGraphSum) is proposed in this paper. In this framework, a hierarchical heterogeneous graph is constructed for the whole document, where the representation of sentences is learnt by several levels of graph neural network. The combination of single-direction message passing and bidirectional message passing helps graph learning obtain effective relations among sentences and words. For capturing the rich semantic information, space–time collaborative learning is designed to generate the primary features of sentences which are enhanced in graph learning. For generating a less redundant and more precise summary, a LSTM based predictor and a blocking strategy are explored. Evaluations both on a single-document dataset and a multi-document dataset demonstrate the effectiveness of the HHGraphSum. The code of HHGraphSum is available on Github:https://github.com/Devin100086/HHGraphSum.

提取式摘要旨在从文档中选择重要句子生成摘要。然而，目前的提取式文档摘要方法未能充分考虑句子之间的语义信息以及整个文档中的各种关系。因此，本文提出了一个新颖的端到端框架，名为用于文档摘要的分层异构图学习（HHGraphSum）。在这个框架中，为整个文档构建了一个分层异构图，其中句子的表示是通过多层图神经网络学习的。单向信息传递和双向信息传递相结合，有助于图学习获得句子和词之间的有效关系。为了捕捉丰富的语义信息，设计了时空协作学习来生成句子的主要特征，这些特征在图学习中得到了增强。为了生成更少冗余、更精确的摘要，我们探索了基于 LSTM 的预测器和阻塞策略。在单文档数据集和多文档数据集上进行的评估证明了 HHGraphSum 的有效性。HHGraphSum 的代码可在 Github:https://github.com/Devin100086/HHGraphSum 上获取。

{"title":"HHGraphSum: Hierarchical heterogeneous graph learning for extractive document summarization","authors":"Pengyi Hao , Cunqi Wu , Cong Bai","doi":"10.1016/j.displa.2024.102884","DOIUrl":"10.1016/j.displa.2024.102884","url":null,"abstract":"<div><div>Extractive summarization aims to select important sentences from the document to generate a summary. However, current extractive document summarization methods fail to fully consider the semantic information among sentences and the various relations in the entire document. Therefore, a novel end-to-end framework named hierarchical heterogeneous graph learning for document summarization (HHGraphSum) is proposed in this paper. In this framework, a hierarchical heterogeneous graph is constructed for the whole document, where the representation of sentences is learnt by several levels of graph neural network. The combination of single-direction message passing and bidirectional message passing helps graph learning obtain effective relations among sentences and words. For capturing the rich semantic information, space–time collaborative learning is designed to generate the primary features of sentences which are enhanced in graph learning. For generating a less redundant and more precise summary, a LSTM based predictor and a blocking strategy are explored. Evaluations both on a single-document dataset and a multi-document dataset demonstrate the effectiveness of the HHGraphSum. The code of HHGraphSum is available on Github:<span><span>https://github.com/Devin100086/HHGraphSum</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"86 ","pages":"Article 102884"},"PeriodicalIF":3.7,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706219","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning domain-adaptive palmprint anti-spoofing feature from multi-source domains 从多源域学习域自适应掌纹防欺骗特征

IF 3.7 2区工程技术 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Displays

Pub Date : 2024-11-16 DOI: 10.1016/j.displa.2024.102871

Chengcheng Liu , Huikai Shao , Dexing Zhong

Palmprint anti-spoofing is essential for securing palmprint recognition systems. Although some anti-spoofing methods excel on closed datasets, their ability to generalize across unknown domains is often limited. This paper introduces the Domain-Adaptive Palmprint Anti-Spoofing Network (DAPANet), which leverages multiple known spoofing domains to extract domain-invariant spoofing clues from unlabeled domains. DAPANet tackles the domain adaptation challenge using three strategies: global domain alignment, subdomain alignment, and the separation of distinct subdomains. The framework consists of a public feature extraction module, a domain adaptation module, a domain classifier, and a fusion classifier. Initially, the public feature extraction module extracts palmprint features. Subsequently, the domain adaptation module aligns target domain features with source domain features to generate domain-specific outputs. The domain classifier provides initial classifiable features, which are then integrated by DAPANet, employing a unified fusion classifier for decision-making. Comprehensive experiments conducted on XJTU-PalmReplay database across various cross-domain scenarios confirm the efficacy of the proposed method.

掌纹防欺骗对于确保掌纹识别系统的安全至关重要。虽然一些反欺骗方法在封闭数据集上表现出色，但它们在未知领域的通用能力往往有限。本文介绍了领域自适应掌纹反欺骗网络（DAPANet），该网络利用多个已知的欺骗领域，从未标明的领域中提取与领域无关的欺骗线索。DAPANet 采用三种策略应对域适应挑战：全域对齐、子域对齐和分离不同的子域。该框架由公共特征提取模块、域适应模块、域分类器和融合分类器组成。首先，公共特征提取模块提取掌纹特征。随后，域适配模块将目标域特征与源域特征对齐，生成特定域的输出。领域分类器提供可分类的初始特征，然后由 DAPANet 对这些特征进行整合，采用统一的融合分类器进行决策。在 XJTU-PalmReplay 数据库上进行的各种跨域场景的综合实验证实了所提方法的有效性。

{"title":"Learning domain-adaptive palmprint anti-spoofing feature from multi-source domains","authors":"Chengcheng Liu , Huikai Shao , Dexing Zhong","doi":"10.1016/j.displa.2024.102871","DOIUrl":"10.1016/j.displa.2024.102871","url":null,"abstract":"<div><div>Palmprint anti-spoofing is essential for securing palmprint recognition systems. Although some anti-spoofing methods excel on closed datasets, their ability to generalize across unknown domains is often limited. This paper introduces the Domain-Adaptive Palmprint Anti-Spoofing Network (DAPANet), which leverages multiple known spoofing domains to extract domain-invariant spoofing clues from unlabeled domains. DAPANet tackles the domain adaptation challenge using three strategies: global domain alignment, subdomain alignment, and the separation of distinct subdomains. The framework consists of a public feature extraction module, a domain adaptation module, a domain classifier, and a fusion classifier. Initially, the public feature extraction module extracts palmprint features. Subsequently, the domain adaptation module aligns target domain features with source domain features to generate domain-specific outputs. The domain classifier provides initial classifiable features, which are then integrated by DAPANet, employing a unified fusion classifier for decision-making. Comprehensive experiments conducted on XJTU-PalmReplay database across various cross-domain scenarios confirm the efficacy of the proposed method.</div></div>","PeriodicalId":50570,"journal":{"name":"Displays","volume":"86 ","pages":"Article 102871"},"PeriodicalIF":3.7,"publicationDate":"2024-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706220","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0