Pattern Recognition Letters最新文献_第5页

Label-noise learning via uncertainty-aware neighborhood sample selection 通过不确定性感知邻域样本选择进行标签噪声学习

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.09.012

Yiliang Zhang, Yang Lu, Hanzi Wang

Existing deep learning methods often require a large amount of high-quality labeled data. Yet, the presence of noisy labels in the real-world training data seriously affects the generalization ability of the model. Sample selection techniques, the current dominant approach to mitigating the effects of noisy labels on models, use the consistency of sample predictions and observed labels to make clean selections. However, these methods rely heavily on the accuracy of the sample predictions and inevitably suffer when the model predictions are unstable. To address these issues, we propose an uncertainty-aware neighborhood sample selection method. Especially, it calibrates for sample prediction by neighbor prediction and reassigns model attention to the selected samples based on sample uncertainty. By alleviating the influence of prediction bias on sample selection and avoiding the occurrence of prediction bias, our proposed method achieves excellent performance in extensive experiments. In particular, we achieved an average of 5% improvement in asymmetric noise scenarios.

现有的深度学习方法通常需要大量高质量的标记数据。然而，真实世界训练数据中存在的噪声标签会严重影响模型的泛化能力。样本选择技术是目前减轻噪声标签对模型影响的主流方法，它利用样本预测和观察到的标签的一致性来进行干净的选择。然而，这些方法在很大程度上依赖于样本预测的准确性，当模型预测不稳定时，这些方法不可避免地会受到影响。为了解决这些问题，我们提出了一种不确定性感知邻域样本选择方法。特别是，它通过邻域预测对样本预测进行校准，并根据样本的不确定性重新分配模型对所选样本的关注度。通过减轻预测偏差对样本选择的影响和避免预测偏差的发生，我们提出的方法在大量实验中取得了优异的性能。特别是在非对称噪声场景下，我们平均提高了 5%。

引用次数: 0

DDOWOD: DiffusionDet for open-world object detection DDOWOD：用于开放世界物体检测的 DiffusionDet

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.10.002

Jiaqi Fan , Enming Zhang , Ying Wei , Yuefeng Wang , Jiakun Xia , Junwei Liu , Xinghong Liu , Shuailei Ma

Open-world object detection (OWOD) poses a significant challenge in computer vision, requiring models to detect unknown objects and incrementally learn new categories. To explore this field, we propose the DDOWOD based on the DiffusionDet. It is more likely to cover unknown objects hidden in the background and can reduce the model’s bias towards known class objects during training due to its ability to randomly generate boxes and reconstruct the characteristics of the GT from them. Also, to improve the insufficient quality of pseudo-labels which leads to reduced accuracy in recognizing unknown classes, we use the Segment Anything Model (SAM) as the teacher model in distillation learning to endow DDOWOD with rich visual knowledge. Surprisingly, compared to other existing models, our DDOWOD is more suitable for using SAM as the teacher. Furthermore, we proposed the Stepwise distillation (SD) which is a new incremental learning method specialized for our DDOWOD to avoid catastrophic forgetting during the training. Our approach utilizes all previously trained models from past tasks rather than solely relying on the last one. DDOWOD has achieved excellent performance. U-Recall is 53.2, 51.5, 50.7 in OWOD split and U-AP is 21.9 in IntensiveSet.

开放世界物体检测（OWOD）是计算机视觉领域的一项重大挑战，需要模型检测未知物体并逐步学习新的类别。为了探索这一领域，我们提出了基于 DiffusionDet 的 DDOWOD，它更有可能覆盖隐藏在背景中的未知物体，并能随机生成方框并从中重建 GT 的特征，从而减少模型在训练过程中对已知类别物体的偏差。同时，为了改善伪标签质量不足导致识别未知类别准确率降低的问题，我们在蒸馏学习中使用了 "任意分段模型"（Segment Anything Model，SAM）作为教师模型，为 DDOWOD 赋予丰富的视觉知识。令人惊讶的是，与其他现有模型相比，我们的 DDOWOD 更适合使用 SAM 作为教师模型。此外，我们还提出了逐步蒸馏法（SD），这是一种新的增量学习方法，专门用于我们的 DDOWOD，以避免训练过程中的灾难性遗忘。我们的方法利用了以往任务中所有训练过的模型，而不是仅仅依赖于最后一个模型。DDOWOD 取得了优异的性能。在 OWOD 分案中，U-Recall 为 53.2，51.5，50.7；在 IntensiveSet 中，U-AP 为 21.9。

{"title":"DDOWOD: DiffusionDet for open-world object detection","authors":"Jiaqi Fan , Enming Zhang , Ying Wei , Yuefeng Wang , Jiakun Xia , Junwei Liu , Xinghong Liu , Shuailei Ma","doi":"10.1016/j.patrec.2024.10.002","DOIUrl":"10.1016/j.patrec.2024.10.002","url":null,"abstract":"<div><div>Open-world object detection (OWOD) poses a significant challenge in computer vision, requiring models to detect unknown objects and incrementally learn new categories. To explore this field, we propose the DDOWOD based on the DiffusionDet. It is more likely to cover unknown objects hidden in the background and can reduce the model’s bias towards known class objects during training due to its ability to randomly generate boxes and reconstruct the characteristics of the GT from them. Also, to improve the insufficient quality of pseudo-labels which leads to reduced accuracy in recognizing unknown classes, we use the Segment Anything Model (SAM) as the teacher model in distillation learning to endow DDOWOD with rich visual knowledge. Surprisingly, compared to other existing models, our DDOWOD is more suitable for using SAM as the teacher. Furthermore, we proposed the Stepwise distillation (SD) which is a new incremental learning method specialized for our DDOWOD to avoid catastrophic forgetting during the training. Our approach utilizes all previously trained models from past tasks rather than solely relying on the last one. DDOWOD has achieved excellent performance. U-Recall is 53.2, 51.5, 50.7 in OWOD split and U-AP is 21.9 in IntensiveSet.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 170-177"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142421626","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Main genes in breast cancer primary tumor and first metastasis in lymph nodes revealed by information-theory-based genetic networks pattern analysis 基于信息论的基因网络模式分析揭示乳腺癌原发肿瘤和淋巴结首次转移的主要基因

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.07.006

Irving Ulises Martínez Vargas , Moises Omar León Pineda , Matías Alvarado Mentado

In this paper, we use pattern analysis in genetic networks to identify differentially expressed genes in primary breast cancer tumors and their first metastasis in lymph nodes, using human biopsies from the GEO and GDCDP databases. By applying Information-Theory-based algorithms to process gene expression profile matrices, we obtained the genetic networks of the following tissues: (1) breast cancer-free, (2) primary breast cancer tumors, and (3) first metastasis of breast cancer in lymph nodes. Topological analysis of the genetic networks delves for identifying patterns of pairs of genes with higher mutual information than a threshold; then, among these genes, the ones with highest degree are elected. We propose the plausible hypothesis that the elected genes, having principal roles in each network, could be relevant as biomarkers regarding the genetic information. A subsequent gene ontology-based analysis of the molecular and functional characteristics of these genes reveals specific signaling pathways signatures in cancer-free tissue and in the tumor microenvironment associated with primary and metastatic requirements. Furthermore, a state-of-the-art review of the functional roles of genes reveals tumor suppressor genes in cancer-free tissue and proliferation- and migration-associated genes in cancer.

在本文中，我们使用遗传网络中的模式分析来识别原发性乳腺癌肿瘤及其淋巴结首次转移的差异表达基因，使用来自GEO和GDCDP数据库的人体活检。通过应用基于信息理论的算法处理基因表达谱矩阵，我们获得了以下组织的遗传网络：(1)无乳腺癌，(2)原发性乳腺癌肿瘤，(3)乳腺癌在淋巴结的首次转移。遗传网络的拓扑分析探讨了识别具有更高互信息的基因对的模式；然后，在这些基因中，选择度最高的基因。我们提出了一个合理的假设，即选择的基因在每个网络中都起着主要作用，可以作为遗传信息的生物标志物。随后对这些基因的分子和功能特征进行了基于基因本体论的分析，揭示了无癌组织和肿瘤微环境中与原发性和转移性要求相关的特定信号通路特征。此外，对基因功能作用的最新回顾揭示了肿瘤抑制基因在无癌组织中的作用以及肿瘤中增殖和迁移相关基因的作用。

{"title":"Main genes in breast cancer primary tumor and first metastasis in lymph nodes revealed by information-theory-based genetic networks pattern analysis","authors":"Irving Ulises Martínez Vargas , Moises Omar León Pineda , Matías Alvarado Mentado","doi":"10.1016/j.patrec.2024.07.006","DOIUrl":"10.1016/j.patrec.2024.07.006","url":null,"abstract":"<div><div>In this paper, we use pattern analysis in genetic networks to identify differentially expressed genes in primary breast cancer tumors and their first metastasis in lymph nodes, using human biopsies from the GEO and GDCDP databases. By applying Information-Theory-based algorithms to process gene expression profile matrices, we obtained the genetic networks of the following tissues: (1) breast cancer-free, (2) primary breast cancer tumors, and (3) first metastasis of breast cancer in lymph nodes. Topological analysis of the genetic networks delves for identifying patterns of pairs of genes with higher mutual information than a threshold; then, among these genes, the ones with highest degree are elected. We propose the plausible hypothesis that the elected genes, having principal roles in each network, could be relevant as biomarkers regarding the genetic information. A subsequent gene ontology-based analysis of the molecular and functional characteristics of these genes reveals specific signaling pathways signatures in cancer-free tissue and in the tumor microenvironment associated with primary and metastatic requirements. Furthermore, a state-of-the-art review of the functional roles of genes reveals tumor suppressor genes in cancer-free tissue and proliferation- and migration-associated genes in cancer.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 369-376"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141700570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhancing abusive language detection: A domain-adapted approach leveraging BERT pre-training tasks 加强滥用语言检测：利用 BERT 预训练任务的领域适应性方法

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.05.007

Horacio Jarquín-Vásquez, Hugo Jair Escalante, Manuel Montes-y-Gómez

The widespread adoption of deep learning approaches in natural language processing is largely attributed to their exceptional performance across diverse tasks. Notably, Transformer-based models, such as BERT, have gained popularity for their remarkable efficacy and their ease of adaptation (via fine-tuning) across various domains. Despite their success, fine-tuning these models for informal language, particularly instances involving offensive expressions, presents a major challenge due to limitations in vocabulary coverage and contextual information for such tasks. To address these challenges, we propose the domain adaptation of the BERT language model for the task of detecting abusive language. Our approach involves constraining the language model with the adaptation and paradigm shift of two default pre-trained tasks, the design of two datasets specifically engineered to support the adapted pre-training tasks, and the proposal of a dynamic weighting loss function. The evaluation of these adapted configurations on six datasets dedicated to abusive language detection reveals promising outcomes, with a significant enhancement observed compared to the base model. Furthermore, our proposed methods yield competitive results when compared to state-of-the-art approaches, establishing a robust and easily trainable model for the effective identification of abusive language.

深度学习方法在自然语言处理中的广泛应用主要归功于它们在不同任务中的出色表现。值得注意的是，基于transformer的模型，例如BERT，由于其显著的功效和跨不同领域的适应性（通过微调）而获得了广泛的欢迎。尽管他们取得了成功，但由于词汇覆盖范围和上下文信息的限制，将这些模型微调到非正式语言，特别是涉及攻击性表达的情况下，面临着重大挑战。为了解决这些挑战，我们提出了BERT语言模型的领域适应来检测滥用语言的任务。我们的方法包括使用两个默认预训练任务的自适应和范式转换来约束语言模型，设计两个专门用于支持自适应预训练任务的数据集，并提出动态加权损失函数。在六个专门用于滥用语言检测的数据集上对这些调整配置的评估显示了有希望的结果，与基本模型相比，观察到显着增强。此外，与最先进的方法相比，我们提出的方法产生了具有竞争力的结果，为有效识别辱骂性语言建立了一个鲁棒且易于训练的模型。

{"title":"Enhancing abusive language detection: A domain-adapted approach leveraging BERT pre-training tasks","authors":"Horacio Jarquín-Vásquez, Hugo Jair Escalante, Manuel Montes-y-Gómez","doi":"10.1016/j.patrec.2024.05.007","DOIUrl":"10.1016/j.patrec.2024.05.007","url":null,"abstract":"<div><div><span>The widespread adoption of deep learning<span> approaches in natural language processing is largely attributed to their exceptional performance across diverse tasks. Notably, Transformer-based models, such as </span></span>BERT<span>, have gained popularity for their remarkable efficacy and their ease of adaptation (via fine-tuning) across various domains. Despite their success, fine-tuning these models for informal language, particularly instances involving offensive expressions, presents a major challenge due to limitations in vocabulary coverage and contextual information for such tasks. To address these challenges, we propose the domain adaptation<span> of the BERT<span> language model<span> for the task of detecting abusive language. Our approach involves constraining the language model with the adaptation and paradigm shift of two default pre-trained tasks, the design of two datasets specifically engineered to support the adapted pre-training tasks, and the proposal of a dynamic weighting loss function. The evaluation of these adapted configurations on six datasets dedicated to abusive language detection reveals promising outcomes, with a significant enhancement observed compared to the base model. Furthermore, our proposed methods yield competitive results when compared to state-of-the-art approaches, establishing a robust and easily trainable model for the effective identification of abusive language.</span></span></span></span></div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 361-368"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141037648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Pseudo-label refinement via hierarchical contrastive learning for source-free unsupervised domain adaptation 通过分层对比学习完善伪标签，实现无源无监督领域适配

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.10.006

Deng Li , Jianguang Zhang , Kunhong Wu , Yucheng Shi , Yahong Han

Source-free unsupervised domain adaptation aims to adapt a source model to an unlabeled target domain without accessing the source data due to privacy considerations. Existing works mainly solve the problem by self-training methods and representation learning. However, these works typically learn the representation on a single semantic level and barely exploit the rich hierarchical semantic information to obtain clear decision boundaries, which makes it hard for these methods to achieve satisfactory generalization performance. In this paper, we propose a novel hierarchical contrastive domain adaptation algorithm that exploits self-supervised contrastive learning on both fine-grained instances and coarse-grained cluster semantics. On the one hand, we propose an adaptive prototype pseudo-labeling strategy to obtain much more reliable labels. On the other hand, we propose hierarchical contrastive representation learning on both fine-grained instance-wise level and coarse-grained cluster level to reduce the negative effect of label noise and stabilize the whole training procedure. Extensive experiments are conducted on primary unsupervised domain adaptation benchmark datasets, and the results demonstrate the effectiveness of the proposed method.

无源无监督领域适配旨在将源模型适配到未标记的目标领域，而无需访问源数据（出于隐私考虑）。现有研究主要通过自我训练方法和表征学习来解决这一问题。然而，这些工作通常是在单一语义层次上学习表示，几乎不利用丰富的分层语义信息来获得清晰的决策边界，这使得这些方法难以达到令人满意的泛化性能。在本文中，我们提出了一种新颖的分层对比领域适应算法，该算法同时利用了细粒度实例和粗粒度聚类语义的自监督对比学习。一方面，我们提出了一种自适应原型伪标签策略，以获得更可靠的标签。另一方面，我们提出了在细粒度实例层面和粗粒度聚类层面进行分层对比表示学习的方法，以减少标签噪声的负面影响并稳定整个训练过程。我们在主要的无监督领域适应基准数据集上进行了广泛的实验，结果证明了所提方法的有效性。

{"title":"Pseudo-label refinement via hierarchical contrastive learning for source-free unsupervised domain adaptation","authors":"Deng Li , Jianguang Zhang , Kunhong Wu , Yucheng Shi , Yahong Han","doi":"10.1016/j.patrec.2024.10.006","DOIUrl":"10.1016/j.patrec.2024.10.006","url":null,"abstract":"<div><div>Source-free unsupervised domain adaptation aims to adapt a source model to an unlabeled target domain without accessing the source data due to privacy considerations. Existing works mainly solve the problem by self-training methods and representation learning. However, these works typically learn the representation on a single semantic level and barely exploit the rich hierarchical semantic information to obtain clear decision boundaries, which makes it hard for these methods to achieve satisfactory generalization performance. In this paper, we propose a novel hierarchical contrastive domain adaptation algorithm that exploits self-supervised contrastive learning on both fine-grained instances and coarse-grained cluster semantics. On the one hand, we propose an adaptive prototype pseudo-labeling strategy to obtain much more reliable labels. On the other hand, we propose hierarchical contrastive representation learning on both fine-grained instance-wise level and coarse-grained cluster level to reduce the negative effect of label noise and stabilize the whole training procedure. Extensive experiments are conducted on primary unsupervised domain adaptation benchmark datasets, and the results demonstrate the effectiveness of the proposed method.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 236-242"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142535005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Measuring student behavioral engagement using histogram of actions 利用行动直方图衡量学生的行为参与度

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.11.002

Ahmed Abdelkawy , Aly Farag , Islam Alkabbany , Asem Ali , Chris Foreman , Thomas Tretter , Nicholas Hindy

In this work, we propose a novel method for assessing students’ behavioral engagement by representing student’s actions and their frequencies over an arbitrary time interval as a histogram of actions. This histogram and the student’s gaze are utilized as input to a classifier that determines whether the student is engaged or not. For action recognition, we use students’ skeletons to model their postures and upper body movements. To learn the dynamics of a student’s upper body, a 3D-CNN model is developed. The trained 3D-CNN model recognizes actions within every 2-minute video segment then these actions are used to build the histogram of actions. To evaluate the proposed framework, we build a dataset consisting of 1414 video segments annotated with 13 actions and 963 2-minute video segments annotated with two engagement levels. Experimental results indicate that student actions can be recognized with top-1 accuracy 86.32% and the proposed framework can capture the average engagement of the class with a 90% F1-score.

在这项工作中，我们提出了一种评估学生行为参与度的新方法，即将学生在任意时间间隔内的动作及其频率表示为动作直方图。该直方图和学生的注视被用作分类器的输入，由分类器判断学生是否参与。在动作识别方面，我们使用学生的骨骼来模拟他们的姿势和上半身动作。为了学习学生上半身的动态，我们开发了一个 3D-CNN 模型。经过训练的 3D-CNN 模型可识别每 2 分钟视频片段中的动作，然后利用这些动作建立动作直方图。为了评估所提出的框架，我们建立了一个数据集，其中包括 1414 个标注了 13 个动作的视频片段和 963 个标注了两个参与度的 2 分钟视频片段。实验结果表明，学生动作的识别准确率最高可达 86.32%，建议的框架可以捕捉全班学生的平均参与度，F1 分数高达 90%。

{"title":"Measuring student behavioral engagement using histogram of actions","authors":"Ahmed Abdelkawy , Aly Farag , Islam Alkabbany , Asem Ali , Chris Foreman , Thomas Tretter , Nicholas Hindy","doi":"10.1016/j.patrec.2024.11.002","DOIUrl":"10.1016/j.patrec.2024.11.002","url":null,"abstract":"<div><div>In this work, we propose a novel method for assessing students’ behavioral engagement by representing student’s actions and their frequencies over an arbitrary time interval as a histogram of actions. This histogram and the student’s gaze are utilized as input to a classifier that determines whether the student is engaged or not. For action recognition, we use students’ skeletons to model their postures and upper body movements. To learn the dynamics of a student’s upper body, a 3D-CNN model is developed. The trained 3D-CNN model recognizes actions within every 2-minute video segment then these actions are used to build the histogram of actions. To evaluate the proposed framework, we build a dataset consisting of 1414 video segments annotated with 13 actions and 963 2-minute video segments annotated with two engagement levels. Experimental results indicate that student actions can be recognized with top-1 accuracy 86.32% and the proposed framework can capture the average engagement of the class with a 90% F1-score.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 337-344"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142657603","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Design of a differentiable L-1 norm for pattern recognition and machine learning 设计用于模式识别和机器学习的可微分 L-1 准则

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.09.020

Min Zhang , Yiming Wang , Hongyu Chen , Taihao Li , Shupeng Liu , Xianfeng Gu , Xiaoyin Xu

In various applications of pattern recognition, feature selection, and machine learning, L-1 norm is used as either an objective function or a regularizer. Mathematically, L-1 norm has unique characteristics that make it attractive in machine learning, feature selection, optimization, and regression. Computationally, however, L-1 norm presents a hurdle as it is non-differentiable, making the process of finding a solution difficult. Existing approach therefore relies on numerical approaches. In this work we designed an L-1 norm that is differentiable and, thus, has an analytical solution. The differentiable L-1 norm removes the absolute sign in the conventional definition and is everywhere differentiable. The new L-1 norm is almost everywhere linear, a desirable feature that is also present in the conventional L-1 norm. The only limitation of the new L-1 norm is that near zero, its behavior is not linear, hence we consider the new L-1 norm quasi-linear. Being differentiable, the new L-1 norm and its quasi-linear variation make them amenable to analytic solutions. Hence, it can facilitate the development and implementation of many algorithms involving L-1 norm. Our tests validate the capability of the new L-1 norm in various applications.

在模式识别、特征选择和机器学习的各种应用中，L-1 准则被用作目标函数或正则表达式。在数学上，L-1 准则具有独特的特性，这使它在机器学习、特征选择、优化和回归中具有吸引力。然而，在计算上，L-1 准则是一个障碍，因为它是无差别的，使得寻找解决方案的过程变得困难。因此，现有方法依赖于数值方法。在这项工作中，我们设计了一种 L-1 准则，它是可微分的，因此有一个解析解。可微分的 L-1 准则去掉了传统定义中的绝对符号，在任何地方都是可微分的。新的 L-1 准则几乎到处都是线性的，这也是传统 L-1 准则的一个理想特征。新 L-1 准则的唯一限制是，在零点附近，它的行为不是线性的，因此我们认为新 L-1 准则是准线性的。由于可微分，新 L-1 准则及其准线性变化使它们易于分析求解。因此，它可以促进许多涉及 L-1 准则的算法的开发和实施。我们的测试验证了新 L-1 准则在各种应用中的能力。

{"title":"Design of a differentiable L-1 norm for pattern recognition and machine learning","authors":"Min Zhang , Yiming Wang , Hongyu Chen , Taihao Li , Shupeng Liu , Xianfeng Gu , Xiaoyin Xu","doi":"10.1016/j.patrec.2024.09.020","DOIUrl":"10.1016/j.patrec.2024.09.020","url":null,"abstract":"<div><div>In various applications of pattern recognition, feature selection, and machine learning, L-1 norm is used as either an objective function or a regularizer. Mathematically, L-1 norm has unique characteristics that make it attractive in machine learning, feature selection, optimization, and regression. Computationally, however, L-1 norm presents a hurdle as it is non-differentiable, making the process of finding a solution difficult. Existing approach therefore relies on numerical approaches. In this work we designed an L-1 norm that is differentiable and, thus, has an analytical solution. The differentiable L-1 norm removes the absolute sign in the conventional definition and is everywhere differentiable. The new L-1 norm is almost everywhere linear, a desirable feature that is also present in the conventional L-1 norm. The only limitation of the new L-1 norm is that near zero, its behavior is not linear, hence we consider the new L-1 norm quasi-linear. Being differentiable, the new L-1 norm and its quasi-linear variation make them amenable to analytic solutions. Hence, it can facilitate the development and implementation of many algorithms involving L-1 norm. Our tests validate the capability of the new L-1 norm in various applications.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 126-132"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142421580","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Online probabilistic knowledge distillation on cryptocurrency trading using Deep Reinforcement Learning 利用深度强化学习对加密货币交易进行在线概率知识提炼

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.10.005

Vasileios Moustakidis , Nikolaos Passalis , Anastasios Tefas

Leveraging Deep Reinforcement Learning (DRL) for training agents for financial trading has gained significant attention in recent years. However, training these agents in noisy financial environments remains challenging and unstable, significantly impacting their performance as trading agents, as the recent literature has also showcased. This paper introduces a novel distillation method for DRL agents, aiming to improve the training stability of DRL agents. The proposed method transfers knowledge from a teacher ensemble to a student model, incorporating both the action probability distribution knowledge from the output layer, as well as the knowledge from the intermediate layers of the teacher’s network. Furthermore, the proposed method also works in an online fashion, allowing for eliminating the separate teacher training process typically involved in many DRL distillation pipelines, simplifying the distillation process. The proposed method is extensively evaluated on a large-scale cryptocurrency trading setup, demonstrating its ability to both lead to significant improvements in trading accuracy and obtained profit, as well as increase the stability of the training process.

近年来，利用深度强化学习（DRL）来训练金融交易代理已受到广泛关注。然而，在嘈杂的金融环境中训练这些代理仍具有挑战性和不稳定性，极大地影响了它们作为交易代理的性能，最近的文献也证明了这一点。本文为 DRL 代理引入了一种新颖的蒸馏方法，旨在提高 DRL 代理的训练稳定性。所提出的方法将知识从教师集合转移到学生模型，既包含输出层的行动概率分布知识，也包含教师网络中间层的知识。此外，所提出的方法还能以在线方式工作，从而省去了许多 DRL 提炼管道中通常涉及的单独教师培训过程，简化了提炼过程。我们在大规模加密货币交易设置上对所提出的方法进行了广泛评估，证明该方法既能显著提高交易准确性和利润，又能提高训练过程的稳定性。

{"title":"Online probabilistic knowledge distillation on cryptocurrency trading using Deep Reinforcement Learning","authors":"Vasileios Moustakidis , Nikolaos Passalis , Anastasios Tefas","doi":"10.1016/j.patrec.2024.10.005","DOIUrl":"10.1016/j.patrec.2024.10.005","url":null,"abstract":"<div><div>Leveraging Deep Reinforcement Learning (DRL) for training agents for financial trading has gained significant attention in recent years. However, training these agents in noisy financial environments remains challenging and unstable, significantly impacting their performance as trading agents, as the recent literature has also showcased. This paper introduces a novel distillation method for DRL agents, aiming to improve the training stability of DRL agents. The proposed method transfers knowledge from a teacher ensemble to a student model, incorporating both the action probability distribution knowledge from the output layer, as well as the knowledge from the intermediate layers of the teacher’s network. Furthermore, the proposed method also works in an online fashion, allowing for eliminating the separate teacher training process typically involved in many DRL distillation pipelines, simplifying the distillation process. The proposed method is extensively evaluated on a large-scale cryptocurrency trading setup, demonstrating its ability to both lead to significant improvements in trading accuracy and obtained profit, as well as increase the stability of the training process.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 243-249"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142535359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Explainable hypergraphs for gait based Parkinson classification 基于帕金森病步态分类的可解释超图

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.09.026

Anirban Dutta Choudhury , Ananda S. Chowdhury

Parkinson Disease (PD) classification using Vertical Ground Reaction Force (VGRF) sensors can help in unobtrusive detection and monitoring of PD patients. State-of-the-art (SOTA) research in PD classification reveals that Deep Learning (DL), at the expense of explainability, performs better than Shallow Learning (SL). In this paper, we introduce a novel explainable weighted hypergraph, where the interconnections of the SOTA features are exploited, leading to more discriminative derived features, and thereby, forming an SL arm. In parallel, we create a DL arm consisting of ResNet architecture to learn the spatio-temporal patterns of the VGRF signals. Probabilities of PD classification scores from the SL and the DL arms are adaptively fused to create a hybrid pipeline. The pipeline achieves an AUC value of 0.979 on the Physionet Parkinson Dataset. This AUC value is found to be superior to the SL as well as the DL arm used in isolation, yielding respective AUCs of 0.878 and 0.852. The proposed pipeline demonstrates explainability through improved permutation feature importance and contrasting examples of use cases, where incorrect misclassification of the DL arm gets rectified by the SL arm and vice versa. We further demonstrate that our solution achieves comparable performance with SOTA methods. To the best of our knowledge, this is the first approach to analyze PD classification with a hypergraph based xAI (Explainable Artificial Intelligence).

使用垂直地面反作用力（VGRF）传感器对帕金森病（PD）进行分类有助于对帕金森病患者进行非侵入式检测和监测。帕金森病分类的最新研究表明，深度学习（DL）以牺牲可解释性为代价，比浅层学习（SL）表现更好。在本文中，我们引入了一种新颖的可解释加权超图，利用 SOTA 特征之间的相互联系，得出更具区分性的衍生特征，从而形成 SL 臂。同时，我们创建了一个由 ResNet 架构组成的 DL 臂，以学习 VGRF 信号的时空模式。来自 SL 和 DL 臂的 PD 分类得分概率被自适应地融合在一起，以创建一个混合管道。该管道在 Physionet 帕金森数据集上的 AUC 值为 0.979。该AUC值优于单独使用的SL和DL臂，前者的AUC值分别为0.878和0.852。所提出的管道通过改进的置换特征重要性和使用案例的对比实例证明了其可解释性，在这些案例中，DL臂的错误分类会被SL臂纠正，反之亦然。我们进一步证明，我们的解决方案与 SOTA 方法的性能相当。据我们所知，这是第一种利用基于超图的 xAI（可解释人工智能）分析 PD 分类的方法。

{"title":"Explainable hypergraphs for gait based Parkinson classification","authors":"Anirban Dutta Choudhury , Ananda S. Chowdhury","doi":"10.1016/j.patrec.2024.09.026","DOIUrl":"10.1016/j.patrec.2024.09.026","url":null,"abstract":"<div><div>Parkinson Disease (PD) classification using Vertical Ground Reaction Force (VGRF) sensors can help in unobtrusive detection and monitoring of PD patients. State-of-the-art (SOTA) research in PD classification reveals that Deep Learning (DL), at the expense of explainability, performs better than Shallow Learning (SL). In this paper, we introduce a novel explainable weighted hypergraph, where the interconnections of the SOTA features are exploited, leading to more discriminative derived features, and thereby, forming an SL arm. In parallel, we create a DL arm consisting of ResNet architecture to learn the spatio-temporal patterns of the VGRF signals. Probabilities of PD classification scores from the SL and the DL arms are adaptively fused to create a hybrid pipeline. The pipeline achieves an AUC value of 0.979 on the Physionet Parkinson Dataset. This AUC value is found to be superior to the SL as well as the DL arm used in isolation, yielding respective AUCs of 0.878 and 0.852. The proposed pipeline demonstrates explainability through improved permutation feature importance and contrasting examples of use cases, where incorrect misclassification of the DL arm gets rectified by the SL arm and vice versa. We further demonstrate that our solution achieves comparable performance with SOTA methods. To the best of our knowledge, this is the first approach to analyze PD classification with a hypergraph based xAI (Explainable Artificial Intelligence).</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 1-7"},"PeriodicalIF":3.9,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142421573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Special section: Best papers of the 15th Mexican conference on pattern recognition (MCPR) 2023 特别部分：第 15 届墨西哥模式识别会议（MCPR）最佳论文 2023

IF 3.9 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Letters

Pub Date : 2024-10-01 DOI: 10.1016/j.patrec.2024.08.017

Ansel Y. Rodríguez González , Humberto Perez-Espinosa , Jesús Ariel Carrasco Ochoa , José Francisco Martínez Trinidad , José Arturo Olvera López

引用次数: 0