Data & Knowledge Engineering最新文献

Unified access to interdisciplinary open data platforms: Open Science Data Network 统一接入跨学科开放数据平台：开放科学数据网络

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-12-19 DOI: 10.1016/j.datak.2025.102552

Vincent-Nam Dang , Nathalie Aussenac-Gilles , Imen Megdiche , Franck Ravat

Open Science is based on a collaborative network to develop transparent, accessible, and shared knowledge. Open Research Data Platforms (ORDPs) are deployed to fulfill the needs for data sharing of a specific community and/or scientific discipline. The high variety of research areas creates a barrier to data sharing between research entities. To enable this research data to be found by the research entities that need it, it is necessary to establish access to different ORDPs that are unknown to these research entities. The goal of this article is to provide a quantitative analysis showing the current limitations of data sharing between ORDPs in Open Science. We then propose a solution to improve data access and sharing based on theoretical foundations and an experimental approach.

We propose to extend our theoretical interoperability model, which helps us to define the necessary steps to interoperate ORDPs. We present and discuss a quantitative evaluation of ORDPs’ interoperability. Based on this exploratory study, we propose a solution that enables research entities to discover unknown ORDPs, thereby facilitating access to relevant data. This solution is the Open Science Data Network (OSDN), a decentralized, distributed, and federated network of ORDPs that integrates a query propagation process and robustness features. To enable the deployment of OSDN at an Open Science scale, we designed our solution by considering its adoption cost relative to a non-organized interoperability approach. With two ORDPs integrated into the OSDN, the adoption cost is estimated to be reduced by at least 17%. This reduction approaches 100% as the number of integrated ORDPs increases.

To demonstrate the feasibility of the solution, we developed a Proof of Concept (POC) and applied it to two research projects from different domains and involving distinct research communities. For the first research project, we measured a 7% increase in the volume of accessed data and an 80% reduction in the time needed to find this data. In addition, researcher from this experiment was able to formulate new intra- and interdisciplinary research questions thanks to the newly accessed data. In the second research project, we observed an increase in data volume of up to a factor of 3968. More importantly, this process led to the discovery of new essential data that was previously missing.

开放科学以协作网络为基础，开发透明、可获取和共享的知识。开放研究数据平台（ordp）的部署是为了满足特定社区和/或科学学科的数据共享需求。研究领域的多样性为研究实体之间的数据共享造成了障碍。为了使需要的研究实体能够找到这些研究数据，有必要建立对这些研究实体未知的不同ordp的访问。本文的目标是提供一个定量分析，显示开放科学中ordp之间数据共享的当前限制。然后，我们提出了一个基于理论基础和实验方法的改进数据访问和共享的解决方案。我们建议扩展我们的理论互操作性模型，它帮助我们定义互操作ordp的必要步骤。我们提出并讨论了ordp互操作性的定量评估。在此探索性研究的基础上，我们提出了一种解决方案，使研究实体能够发现未知的ordp，从而促进相关数据的访问。这个解决方案就是开放科学数据网络（OSDN），它是一个分散、分布式和联合的ordp网络，集成了查询传播过程和健壮性特性。为了能够在开放科学规模上部署OSDN，我们在设计解决方案时考虑了相对于非组织互操作性方法的采用成本。将两个ordp集成到OSDN中，采用成本估计至少降低了17%。随着集成ordp数量的增加，这种减少接近100%。为了证明该解决方案的可行性，我们开发了一个概念验证（POC），并将其应用于来自不同领域和涉及不同研究社区的两个研究项目。对于第一个研究项目，我们测量到访问的数据量增加了7%，查找这些数据所需的时间减少了80%。此外，由于新获得的数据，本次实验的研究人员能够制定新的内部和跨学科的研究问题。在第二个研究项目中，我们观察到数据量增加了3968倍。更重要的是，这一过程导致发现了以前缺失的新的重要数据。

{"title":"Unified access to interdisciplinary open data platforms: Open Science Data Network","authors":"Vincent-Nam Dang , Nathalie Aussenac-Gilles , Imen Megdiche , Franck Ravat","doi":"10.1016/j.datak.2025.102552","DOIUrl":"10.1016/j.datak.2025.102552","url":null,"abstract":"<div><div>Open Science is based on a collaborative network to develop transparent, accessible, and shared knowledge. Open Research Data Platforms (ORDPs) are deployed to fulfill the needs for data sharing of a specific community and/or scientific discipline. The high variety of research areas creates a barrier to data sharing between research entities. To enable this research data to be found by the research entities that need it, it is necessary to establish access to different ORDPs that are unknown to these research entities. The goal of this article is to provide a quantitative analysis showing the current limitations of data sharing between ORDPs in Open Science. We then propose a solution to improve data access and sharing based on theoretical foundations and an experimental approach.</div><div>We propose to extend our theoretical interoperability model, which helps us to define the necessary steps to interoperate ORDPs. We present and discuss a quantitative evaluation of ORDPs’ interoperability. Based on this exploratory study, we propose a solution that enables research entities to discover unknown ORDPs, thereby facilitating access to relevant data. This solution is the Open Science Data Network (OSDN), a decentralized, distributed, and federated network of ORDPs that integrates a query propagation process and robustness features. To enable the deployment of OSDN at an Open Science scale, we designed our solution by considering its adoption cost relative to a non-organized interoperability approach. With two ORDPs integrated into the OSDN, the adoption cost is estimated to be reduced by at least 17%. This reduction approaches 100% as the number of integrated ORDPs increases.</div><div>To demonstrate the feasibility of the solution, we developed a Proof of Concept (POC) and applied it to two research projects from different domains and involving distinct research communities. For the first research project, we measured a 7% increase in the volume of accessed data and an 80% reduction in the time needed to find this data. In addition, researcher from this experiment was able to formulate new intra- and interdisciplinary research questions thanks to the newly accessed data. In the second research project, we observed an increase in data volume of up to a factor of 3968. More importantly, this process led to the discovery of new essential data that was previously missing.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102552"},"PeriodicalIF":2.7,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145790029","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

All-words pronunciation estimation of Japanese homographs 日语同音异义词的全词发音估计

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-12-12 DOI: 10.1016/j.datak.2025.102537

Kanako Komiya , Taichiro Kobayashi , Masayuki Asahara , Hiroyuki Shinnou

The Japanese language has many homographs, which are words that share the same kanji characters, regardless of their pronunciations. Therefore, pronunciation estimation of homographs is necessary to read Japanese sentences accurately. We developed a system to estimate the pronunciations of homographs using a Bidirectional Encoder Representations from the Transformer model. This is the first research paper on pronunciation estimation of all homographs and we achieved this goal using the technique for all-word word sense disambiguation. We used the Corpus of Spontaneous Japanese (CSJ), a transcription of spoken Japanese, as the test data and utilized the non-core data of the Balanced Corpus of Contemporary Written Japanese, for which pronunciations are automatically tagged by a Japanese morphological analyzer, in addition to CSJ, as training data to reduce the cost of transcription. We also investigated the case where pseudo-pronunciation data was assigned to CSJ using a morphological analyzer as training examples. We show that automatically tagged data can improve the accuracy of pronunciation estimation.

Additionally, to evaluate an all-words pronunciation estimation system, we developed a dataset through crowdsourcing. We asked 20 crowdworkers to select pronunciations for the sentences from the Nihon Keizai Shimbun newspaper (the NIKKEI). For this NIKKEI data, multiple correct pronunciations were allowed, and the answer provided by the majority of crowdworkers was treated as the correct answer for evaluation purposes. When comparing the model trained on pseudo-data from BCCWJ with the model trained on pseudo-data from CSJ, the model using BCCWJ pseudo-data demonstrated superior performance.

日语中有许多同音异义词，即具有相同汉字字符的单词，无论其发音如何。因此，同音异义词的读音判断是准确读懂日语句子的必要条件。我们开发了一个系统来估计同音异义词的发音使用双向编码器表示从变压器模型。这是第一篇关于所有同形异义词发音估计的研究论文，我们利用全词词义消歧技术实现了这一目标。我们使用日语口语转录库CSJ作为测试数据，并利用现代书面日语平衡语料库的非核心数据作为训练数据，该语料库的发音除CSJ外还由日语形态分析仪自动标记，以降低转录成本。我们还研究了使用形态学分析器将伪发音数据分配给CSJ作为训练示例的情况。我们证明了自动标记数据可以提高语音估计的准确性。此外，为了评估一个全词发音估计系统，我们通过众包开发了一个数据集。我们请20位众包工作者为《日本经济新闻》（NIKKEI）的句子选择读音。本次日经数据允许多次正确发音，多数众包工作者提供的答案作为评估的正确答案。将BCCWJ伪数据训练的模型与CSJ伪数据训练的模型进行比较，BCCWJ伪数据训练的模型表现出更好的性能。

{"title":"All-words pronunciation estimation of Japanese homographs","authors":"Kanako Komiya , Taichiro Kobayashi , Masayuki Asahara , Hiroyuki Shinnou","doi":"10.1016/j.datak.2025.102537","DOIUrl":"10.1016/j.datak.2025.102537","url":null,"abstract":"<div><div>The Japanese language has many homographs, which are words that share the same kanji characters, regardless of their pronunciations. Therefore, pronunciation estimation of homographs is necessary to read Japanese sentences accurately. We developed a system to estimate the pronunciations of homographs using a Bidirectional Encoder Representations from the Transformer model. This is the first research paper on pronunciation estimation of all homographs and we achieved this goal using the technique for all-word word sense disambiguation. We used the Corpus of Spontaneous Japanese (CSJ), a transcription of spoken Japanese, as the test data and utilized the non-core data of the Balanced Corpus of Contemporary Written Japanese, for which pronunciations are automatically tagged by a Japanese morphological analyzer, in addition to CSJ, as training data to reduce the cost of transcription. We also investigated the case where pseudo-pronunciation data was assigned to CSJ using a morphological analyzer as training examples. We show that automatically tagged data can improve the accuracy of pronunciation estimation.</div><div>Additionally, to evaluate an all-words pronunciation estimation system, we developed a dataset through crowdsourcing. We asked 20 crowdworkers to select pronunciations for the sentences from the Nihon Keizai Shimbun newspaper (the NIKKEI). For this NIKKEI data, multiple correct pronunciations were allowed, and the answer provided by the majority of crowdworkers was treated as the correct answer for evaluation purposes. When comparing the model trained on pseudo-data from BCCWJ with the model trained on pseudo-data from CSJ, the model using BCCWJ pseudo-data demonstrated superior performance.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102537"},"PeriodicalIF":2.7,"publicationDate":"2025-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145790026","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring cutting-edge data ecosystems: A comprehensive analysis 探索前沿数据生态系统：综合分析

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-12-02 DOI: 10.1016/j.datak.2025.102539

Ioannis Chrysakis , David Chaves-Fraga , Giorgos Flouris , Erik Mannens , Anastasia Dimou

Data-driven innovation has recently changed the mindset in data sharing from centralized architectures and monolithic data exploitation by data providers (data platforms) to decentralized architectures and different data sharing options among all involved participants (data ecosystems). Data sharing is further strengthened through the establishment of several legal frameworks (e.g., European Strategy for Data, Data Act, Data Governance Act) and the emerging initiatives that provide the means to build data ecosystems, which is evident in the formulated communities, established use cases, and the technical solutions. However, the data ecosystems have not been thoroughly studied so far. The differences between the various data ecosystems are not clear, making it hard to choose the most suitable for each use case, negatively impacting their adoption. Since the domain is growing fast, a review of the state-of-the-art data ecosystem initiatives is needed to analyze what each initiative offers, identify collaboration prospects, and highlight features for improvement and open research topics. In this paper, we review the state-of-the-art data ecosystem initiatives, describe their innovative aspects, compare their technical and business features, and identify open research challenges. We aim to assist practitioners in choosing the most suitable data ecosystem for their use cases and scientists to explore emerging research opportunities. Furthermore, we will provide a framework that outlines the key criteria for evaluating these initiatives, ensuring that stakeholders can make informed decisions based on their specific needs and objectives. By synthesizing our findings, we hope to foster a deeper understanding of the evolving landscape of data ecosystems and encourage further advancements in this critical field.

数据驱动的创新最近改变了数据共享的思维方式，从数据提供者（数据平台）的集中式架构和单块数据利用，到所有参与者（数据生态系统）的分散架构和不同的数据共享选项。通过建立几个法律框架（例如，欧洲数据战略、数据法案、数据治理法案）和提供构建数据生态系统手段的新举措，数据共享得到进一步加强，这在制定的社区、已建立的用例和技术解决方案中是显而易见的。然而，到目前为止，对数据生态系统的研究还不够深入。各种数据生态系统之间的差异并不清楚，因此很难为每个用例选择最合适的，这对它们的采用产生了负面影响。由于该领域正在快速发展，需要对最先进的数据生态系统计划进行回顾，以分析每个计划提供的内容，确定合作前景，并突出改进的功能和开放的研究主题。在本文中，我们回顾了最新的数据生态系统计划，描述了它们的创新方面，比较了它们的技术和业务特征，并确定了开放的研究挑战。我们的目标是帮助从业者为他们的用例选择最合适的数据生态系统，帮助科学家探索新兴的研究机会。此外，我们将提供一个框架，概述评估这些举措的关键标准，确保利益相关者能够根据他们的具体需求和目标做出明智的决定。通过综合我们的发现，我们希望加深对数据生态系统不断变化的景观的理解，并鼓励这一关键领域的进一步发展。

{"title":"Exploring cutting-edge data ecosystems: A comprehensive analysis","authors":"Ioannis Chrysakis , David Chaves-Fraga , Giorgos Flouris , Erik Mannens , Anastasia Dimou","doi":"10.1016/j.datak.2025.102539","DOIUrl":"10.1016/j.datak.2025.102539","url":null,"abstract":"<div><div>Data-driven innovation has recently changed the mindset in data sharing from centralized architectures and monolithic data exploitation by data providers (data platforms) to decentralized architectures and different data sharing options among all involved participants (data ecosystems). Data sharing is further strengthened through the establishment of several legal frameworks (e.g., European Strategy for Data, Data Act, Data Governance Act) and the emerging initiatives that provide the means to build data ecosystems, which is evident in the formulated communities, established use cases, and the technical solutions. However, the data ecosystems have not been thoroughly studied so far. The differences between the various data ecosystems are not clear, making it hard to choose the most suitable for each use case, negatively impacting their adoption. Since the domain is growing fast, a review of the state-of-the-art data ecosystem initiatives is needed to analyze what each initiative offers, identify collaboration prospects, and highlight features for improvement and open research topics. In this paper, we review the state-of-the-art data ecosystem initiatives, describe their innovative aspects, compare their technical and business features, and identify open research challenges. We aim to assist practitioners in choosing the most suitable data ecosystem for their use cases and scientists to explore emerging research opportunities. Furthermore, we will provide a framework that outlines the key criteria for evaluating these initiatives, ensuring that stakeholders can make informed decisions based on their specific needs and objectives. By synthesizing our findings, we hope to foster a deeper understanding of the evolving landscape of data ecosystems and encourage further advancements in this critical field.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102539"},"PeriodicalIF":2.7,"publicationDate":"2025-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145736659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhancing clustering stability, compactness, and separation in multimodal data environments 增强多模态数据环境中的聚类稳定性、紧凑性和分离性

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-11-26 DOI: 10.1016/j.datak.2025.102536

Fillipe dos Santos Silva , Júlio Cesar dos Reis , Marcelo da Silva Reis

Effective customer segmentation, crucial for tailored marketing strategies, relies on stable and distinct clustering methods. Traditional clustering approaches often focus on structured data, limiting their effectiveness when handling multimodal information. This study originally introduces a multimodal framework to enhance clustering stability, compactness, and separation by integrating categorical, numerical, and textual data. Our framework addresses existing limitations through three core components: a transformer-based embedding model for textual analysis, a data fusion layer for integrating diverse data types, and a generative model for refining cluster consistency. We rigorously assess the effectiveness of our framework using five stability metrics: Adjusted Rand Index (ARI), Adjusted Mutual Information Score (AMIS), BagClust (BG), Hierarchical Agglomerative Nesting (HAN), and Optimal Transport Alignment (OTA). Additionally, we use the Davies–Bouldin Score (DBS) to evaluate cluster compactness and separation. Real-world datasets (Yelp, Melbourne Airbnb, PetFinder.my, Women’s Clothing Reviews) were used to benchmark our approach against four existing methods. Results demonstrate that our framework achieves superior clustering stability, compactness, and separation, advancing multimodal learning for more nuanced customer segmentation.

有效的客户细分对于量身定制的营销策略至关重要，它依赖于稳定而独特的聚类方法。传统的聚类方法通常侧重于结构化数据，这限制了它们在处理多模态信息时的有效性。本研究最初引入了一个多模态框架，通过整合分类、数值和文本数据来增强聚类的稳定性、紧凑性和分离性。我们的框架通过三个核心组件解决了现有的限制：一个用于文本分析的基于转换器的嵌入模型，一个用于集成不同数据类型的数据融合层，以及一个用于精炼集群一致性的生成模型。我们使用五个稳定性指标严格评估我们框架的有效性：调整Rand指数（ARI）、调整互信息评分（AMIS）、bagcluster （BG）、分层凝聚嵌套（HAN）和最优传输对齐（OTA）。此外，我们使用Davies-Bouldin评分（DBS）来评估聚类的紧密性和分离性。真实世界的数据集（Yelp，墨尔本Airbnb， PetFinder）。我的《女装评论》（Women’s Clothing Reviews）将我们的方法与四种现有方法进行对比。结果表明，我们的框架实现了卓越的聚类稳定性、紧凑性和分离性，推进了多模态学习，以实现更细致的客户细分。

{"title":"Enhancing clustering stability, compactness, and separation in multimodal data environments","authors":"Fillipe dos Santos Silva , Júlio Cesar dos Reis , Marcelo da Silva Reis","doi":"10.1016/j.datak.2025.102536","DOIUrl":"10.1016/j.datak.2025.102536","url":null,"abstract":"<div><div>Effective customer segmentation, crucial for tailored marketing strategies, relies on stable and distinct clustering methods. Traditional clustering approaches often focus on structured data, limiting their effectiveness when handling multimodal information. This study originally introduces a multimodal framework to enhance clustering stability, compactness, and separation by integrating categorical, numerical, and textual data. Our framework addresses existing limitations through three core components: a transformer-based embedding model for textual analysis, a data fusion layer for integrating diverse data types, and a generative model for refining cluster consistency. We rigorously assess the effectiveness of our framework using five stability metrics: Adjusted Rand Index (ARI), Adjusted Mutual Information Score (AMIS), BagClust (BG), Hierarchical Agglomerative Nesting (HAN), and Optimal Transport Alignment (OTA). Additionally, we use the Davies–Bouldin Score (DBS) to evaluate cluster compactness and separation. Real-world datasets (Yelp, Melbourne Airbnb, PetFinder.my, Women’s Clothing Reviews) were used to benchmark our approach against four existing methods. Results demonstrate that our framework achieves superior clustering stability, compactness, and separation, advancing multimodal learning for more nuanced customer segmentation.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102536"},"PeriodicalIF":2.7,"publicationDate":"2025-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145618297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

S3: A simple strong sample-effective multimodal dialog system S3：一个简单的强样本有效的多模式对话系统

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-11-25 DOI: 10.1016/j.datak.2025.102538

Elisei Rykov , Alexander Panchenko

In this work, we present a conceptually simple yet powerful baseline for multimodal dialog task, an S

^{3}

model, that achieves near state-of-the-art results on four compelling leaderboards: MMMU, RealWorldQA, POPE, and AI Journey Contest 2023. The system is based on a pre-trained large language model, pre-trained modality encoders for image and audio, and a trainable modality projector. The proposed effective data mixture for training such an architecture demonstrates that a multimodal model based on a strong language model and trained on a small amount of multimodal data can perform efficiently in the task of multimodal dialog.

在这项工作中，我们提出了一个概念简单但功能强大的多模态对话任务基线，一个S3模型，在四个引人注目的排行榜上取得了接近最先进的结果：MMMU， RealWorldQA， POPE和AI Journey Contest 2023。该系统基于预训练的大型语言模型、预训练的图像和音频模态编码器以及可训练的模态投影仪。本文提出的训练多模态结构的有效数据混合表明，基于强语言模型并在少量多模态数据上训练的多模态模型可以有效地执行多模态对话任务。

引用次数: 0

“Detectors Lead, LLMs Follow”: Integrating LLMs and traditional models on implicit hate speech detection to generate faithful and plausible explanations “检测器领先，法学硕士跟随”：整合法学硕士和传统的隐式仇恨言论检测模型，生成忠实和可信的解释

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-11-24 DOI: 10.1016/j.datak.2025.102535

Greta Damo , Nicolás Benjamín Ocampo , Elena Cabrio, Serena Villata

Social media platforms face a growing challenge in addressing abusive content and hate speech, particularly as traditional natural language processing methods often struggle with detecting nuanced and implicit instances. To tackle this issue, our study enhances Large Language Models (LLMs) in the detection and explanation of implicit hate speech, outperforming classical approaches. We focus on two key objectives: (1) determining whether jointly predicting and generating explanations for why a message is hateful improves LLMs’ accuracy, especially for implicit cases, and (2) evaluating whether incorporating information from BERT-based models can further boost detection and explanation performance. Our method evaluates and enhances LLMs’ ability to detect hate speech and explain their predictions. By combining binary classification (Hate Speech vs. Non-Hate Speech) with natural language explanations, our approach provides clearer insights into why a message is considered hateful, advancing the accuracy and interpretability of hate speech detection.

社交媒体平台在处理辱骂性内容和仇恨言论方面面临着越来越大的挑战，尤其是传统的自然语言处理方法往往难以发现微妙和隐含的实例。为了解决这个问题，我们的研究增强了大型语言模型（llm）在隐性仇恨言论的检测和解释方面的性能，优于经典方法。我们专注于两个关键目标：(1)确定联合预测和生成解释为什么消息是可恨的是否可以提高llm的准确性，特别是对于隐式情况，以及(2)评估结合基于bert的模型的信息是否可以进一步提高检测和解释性能。我们的方法评估并提高了法学硕士检测仇恨言论和解释其预测的能力。通过将二元分类（仇恨言论与非仇恨言论）与自然语言解释相结合，我们的方法可以更清楚地了解为什么一条信息被认为是仇恨的，从而提高仇恨言论检测的准确性和可解释性。

{"title":"“Detectors Lead, LLMs Follow”: Integrating LLMs and traditional models on implicit hate speech detection to generate faithful and plausible explanations","authors":"Greta Damo , Nicolás Benjamín Ocampo , Elena Cabrio, Serena Villata","doi":"10.1016/j.datak.2025.102535","DOIUrl":"10.1016/j.datak.2025.102535","url":null,"abstract":"<div><div>Social media platforms face a growing challenge in addressing abusive content and hate speech, particularly as traditional natural language processing methods often struggle with detecting nuanced and implicit instances. To tackle this issue, our study enhances Large Language Models (LLMs) in the detection and explanation of implicit hate speech, outperforming classical approaches. We focus on two key objectives: (1) determining whether jointly predicting and generating explanations for why a message is hateful improves LLMs’ accuracy, especially for implicit cases, and (2) evaluating whether incorporating information from BERT-based models can further boost detection and explanation performance. Our method evaluates and enhances LLMs’ ability to detect hate speech and explain their predictions. By combining binary classification (Hate Speech vs. Non-Hate Speech) with natural language explanations, our approach provides clearer insights into why a message is considered hateful, advancing the accuracy and interpretability of hate speech detection.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102535"},"PeriodicalIF":2.7,"publicationDate":"2025-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145617829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Human vs. Automated data annotation: Labeling the data set for an ML-driven support ticket classifier 人工与自动数据注释：为ml驱动的支持票证分类器标记数据集

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-11-23 DOI: 10.1016/j.datak.2025.102534

Simon Fuchs, Janik Schnellbach, Holger Wittges, Helmut Krcmar

In general, supervised Machine Learning approaches using labeled training data currently promise the best classification results with respect to classification accuracy. Therefore, data annotation is a key component of most Machine Learning projects implemented. However, creating labels for a training data set is often an elaborate project involving arduous and repetitive work, which is why data scientists often try to minimize the effort for data annotation by automating the data annotation process itself. In this paper, we present a case study of two data annotation projects on the same data set of support tickets and compare these: one using human annotators and the other using algorithmic Learning Functions in a combination of Active Learning and Weak Supervision. Here, we achieved a weighted confidence score of >94 % for the human-created labels, while also achieving up to 92 % agreement between the labels of our automated project and the labels created by human annotators, with the need for only 10 % of human annotation as the starting input of the automated approach. Additionally, we were able to reproduce the value of 85 % for initial human classification accuracy in support ticket distribution from previous papers. We close with a reflection about the worth of business understanding in data annotation projects and the problem and proposed solutions to ticket ambiguity.

一般来说，使用标记训练数据的监督机器学习方法目前在分类精度方面保证了最好的分类结果。因此，数据注释是大多数机器学习项目的关键组成部分。然而，为训练数据集创建标签通常是一个复杂的项目，涉及艰巨和重复的工作，这就是为什么数据科学家经常试图通过自动化数据注释过程本身来减少数据注释的工作量。在本文中，我们对同一支持票数据集上的两个数据标注项目进行了案例研究，并对它们进行了比较：一个使用人工标注，另一个使用主动学习和弱监督相结合的算法学习函数。在这里，我们实现了人工创建标签的加权置信度得分为>； 94%，同时我们的自动化项目的标签和人类注释者创建的标签之间也实现了高达92%的一致性，只需要10%的人类注释作为自动化方法的开始输入。此外，我们能够从以前的论文中重现85%的初始人类分类准确率。最后，我们对数据注释项目中业务理解的价值和问题进行了反思，并提出了解决票歧义的方法。

{"title":"Human vs. Automated data annotation: Labeling the data set for an ML-driven support ticket classifier","authors":"Simon Fuchs, Janik Schnellbach, Holger Wittges, Helmut Krcmar","doi":"10.1016/j.datak.2025.102534","DOIUrl":"10.1016/j.datak.2025.102534","url":null,"abstract":"<div><div>In general, supervised Machine Learning approaches using labeled training data currently promise the best classification results with respect to classification accuracy. Therefore, data annotation is a key component of most Machine Learning projects implemented. However, creating labels for a training data set is often an elaborate project involving arduous and repetitive work, which is why data scientists often try to minimize the effort for data annotation by automating the data annotation process itself. In this paper, we present a case study of two data annotation projects on the same data set of support tickets and compare these: one using human annotators and the other using algorithmic Learning Functions in a combination of Active Learning and Weak Supervision. Here, we achieved a weighted confidence score of >94 % for the human-created labels, while also achieving up to 92 % agreement between the labels of our automated project and the labels created by human annotators, with the need for only 10 % of human annotation as the starting input of the automated approach. Additionally, we were able to reproduce the value of 85 % for initial human classification accuracy in support ticket distribution from previous papers. We close with a reflection about the worth of business understanding in data annotation projects and the problem and proposed solutions to ticket ambiguity.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102534"},"PeriodicalIF":2.7,"publicationDate":"2025-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145684462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Quality matters: A decadal systematic exploration of data quality in IoT environment 质量至关重要：物联网环境下数据质量的十年系统探索

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-11-20 DOI: 10.1016/j.datak.2025.102533

Tarandeep Kaur, Pankaj Deep Kaur

The proliferation of the Internet of Things (IoT) has led to an unprecedented surge in data generation, making the enhancement of data quality indispensable for unlocking IoT's full potential and enabling intelligent, data-driven decision-making. This systematic literature review examines scholarly research from the past decade to unravel the complexities of data quality in IoT. Seven research questions have been formulated, an extensive search of relevant academic databases has been conducted, and criteria for inclusion and exclusion have been defined. Key insights from these selected studies address the research questions, analyzing the multi-faceted landscape of IoT data and its quality. This study explores various data quality dimensions that play a pivotal role in assessing overall data quality, identifying critical gaps and limitations, and offering a roadmap for future research. The comprehensive overview provides a nuanced understanding of the factors influencing data quality and highlights the contributions of various researchers to ensure data quality. The study consolidates perspectives on the significance of data quality as perceived by users, while also emphasizing the paramount importance of security and privacy in managing IoT data. The findings of this SLR will provide valuable insights to researchers and practitioners, advancing efforts to maintain robust data quality across IoT ecosystems.

物联网（IoT）的扩散导致了数据生成的前所未有的激增，这使得数据质量的提高对于释放物联网的全部潜力和实现智能、数据驱动的决策必不可少。这篇系统的文献综述检查了过去十年的学术研究，以揭示物联网数据质量的复杂性。制定了七个研究问题，对相关学术数据库进行了广泛的搜索，并确定了纳入和排除标准。这些精选研究的关键见解解决了研究问题，分析了物联网数据及其质量的多方面景观。本研究探讨了各种数据质量维度，这些维度在评估整体数据质量、识别关键差距和限制以及为未来研究提供路线图方面发挥着关键作用。全面的概述提供了对影响数据质量的因素的细致理解，并强调了各种研究人员对确保数据质量的贡献。该研究巩固了用户对数据质量重要性的看法，同时也强调了安全性和隐私在管理物联网数据中的首要重要性。该单反的研究结果将为研究人员和从业者提供有价值的见解，推动在物联网生态系统中保持稳健数据质量的努力。

{"title":"Quality matters: A decadal systematic exploration of data quality in IoT environment","authors":"Tarandeep Kaur, Pankaj Deep Kaur","doi":"10.1016/j.datak.2025.102533","DOIUrl":"10.1016/j.datak.2025.102533","url":null,"abstract":"<div><div>The proliferation of the Internet of Things (IoT) has led to an unprecedented surge in data generation, making the enhancement of data quality indispensable for unlocking IoT's full potential and enabling intelligent, data-driven decision-making. This systematic literature review examines scholarly research from the past decade to unravel the complexities of data quality in IoT. Seven research questions have been formulated, an extensive search of relevant academic databases has been conducted, and criteria for inclusion and exclusion have been defined. Key insights from these selected studies address the research questions, analyzing the multi-faceted landscape of IoT data and its quality. This study explores various data quality dimensions that play a pivotal role in assessing overall data quality, identifying critical gaps and limitations, and offering a roadmap for future research. The comprehensive overview provides a nuanced understanding of the factors influencing data quality and highlights the contributions of various researchers to ensure data quality. The study consolidates perspectives on the significance of data quality as perceived by users, while also emphasizing the paramount importance of security and privacy in managing IoT data. The findings of this SLR will provide valuable insights to researchers and practitioners, advancing efforts to maintain robust data quality across IoT ecosystems.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102533"},"PeriodicalIF":2.7,"publicationDate":"2025-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145684460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Optimized adaptive depression state prediction and severity estimation from twitter data 基于twitter数据的抑郁状态优化自适应预测和严重程度估计

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-11-19 DOI: 10.1016/j.datak.2025.102532

Pavani Chirasani , Gatram Rama Mohan Babu

Social media sites like Twitter, which offer an abundance of content supplemented with emojis, can be used to identify and treat depression, a widespread mental health issue. Several prediction methods exist to predict the depression. However, the relevant outcome was not good because of inaccurate prediction. The input Twitter emoji data contains more error features, which increases the complexity of predicting depression. These drawbacks resulted in poor prediction and low Accuracy. So, the proposed work aims to design a novel Zebra-based Long former Emoji Analysis (ZLEA) for predicting depression. The Twitter Emoji database was initially collected from the standard website and provided to the Python environment as input. First, the pre-processing function is run to remove the noisy features that are present in the trained database. Moreover, the necessary features were extracted and the depression condition was predicted using the current emoji. Finally, the depression severe state was assessed based on the emoji's grade level, and the performance was confirmed with other conventional research with metrics.

像推特这样的社交媒体网站，提供了丰富的内容，辅以表情符号，可以用来识别和治疗抑郁症，这是一种普遍的心理健康问题。目前已有几种预测凹陷的方法。然而，由于预测不准确，相关结果并不好。输入的Twitter表情符号数据包含更多的错误特征，这增加了预测抑郁症的复杂性。这些缺点导致了较差的预测和较低的准确性。因此，本研究旨在设计一种基于斑马的长前表情符号分析（ZLEA）来预测抑郁症。Twitter表情符号数据库最初是从标准网站收集的，并作为输入提供给Python环境。首先，运行预处理函数去除训练数据库中存在的噪声特征。此外，提取必要的特征，并使用当前表情符号预测抑郁状况。最后，根据表情符号的年级水平评估抑郁严重状态，并通过其他带有指标的常规研究证实其表现。

{"title":"Optimized adaptive depression state prediction and severity estimation from twitter data","authors":"Pavani Chirasani , Gatram Rama Mohan Babu","doi":"10.1016/j.datak.2025.102532","DOIUrl":"10.1016/j.datak.2025.102532","url":null,"abstract":"<div><div>Social media sites like Twitter, which offer an abundance of content supplemented with emojis, can be used to identify and treat depression, a widespread mental health issue. Several prediction methods exist to predict the depression. However, the relevant outcome was not good because of inaccurate prediction. The input Twitter emoji data contains more error features, which increases the complexity of predicting depression. These drawbacks resulted in poor prediction and low Accuracy. So, the proposed work aims to design a novel Zebra-based Long former Emoji Analysis (ZLEA) for predicting depression. The Twitter Emoji database was initially collected from the standard website and provided to the Python environment as input. First, the pre-processing function is run to remove the noisy features that are present in the trained database. Moreover, the necessary features were extracted and the depression condition was predicted using the current emoji. Finally, the depression severe state was assessed based on the emoji's grade level, and the performance was confirmed with other conventional research with metrics.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102532"},"PeriodicalIF":2.7,"publicationDate":"2025-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145617823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Query-based automatic text summarization using query expansion approach 使用查询扩展方法的基于查询的自动文本摘要

IF 2.7 3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Data & Knowledge Engineering

Pub Date : 2025-11-17 DOI: 10.1016/j.datak.2025.102531

Hiteshwar Kumar Azad

The amount of information available on the Web has grown dramatically and continues to grow on a daily basis. The massive amount of Web data poses significant challenges to the reliability and accuracy of current information retrieval systems. The purpose of information retrieval is to discover relevant documents within a huge group of documents whose contents match a user-initiated query. Because most users struggle to formulate well-defined queries, the query expansion technique is critical for retrieving the most relevant information. Obtaining relevant results in a concise manner is a significant challenge in this scenario. Automatic text summarization can condense a lengthy document while retaining its informative content and key concepts. It could be a potential solution to information overload. This paper proposed a query-based automatic text summarization technique that employs query expansion to improve text summarization and provide the relevant information in a concise manner. To produce a relevant text summary, this article employs a query-based extractive text summarization method, which involves selecting sentences based on the four best features retrieved from each sentence. In this process, the words are scored by the expanded query’s score, and the sentences are scored by four important features, including sentence terms, position, similarity to the first sentence, and proper noun. Extensive experiments with different ROUGE variants on various evaluation metrics, including precision, recall, and F-score, were carried out on the DUC 2007 dataset, with gains of approximately 44%, 46%, and 45% respectively, in the best scenario. It is observed that the suggested approach outperforms both DUC participatory systems and cutting-edge approaches in summary generation.

Web上可用的信息量急剧增长，并且每天都在继续增长。海量的网络数据对当前信息检索系统的可靠性和准确性提出了严峻的挑战。信息检索的目的是在内容与用户发起的查询匹配的庞大文档组中发现相关文档。由于大多数用户难以制定定义良好的查询，因此查询扩展技术对于检索最相关的信息至关重要。在这种情况下，以简洁的方式获得相关结果是一项重大挑战。自动文本摘要可以压缩冗长的文档，同时保留其信息内容和关键概念。这可能是解决信息过载的一个潜在方法。本文提出了一种基于查询的文本自动摘要技术，该技术采用查询扩展的方法来改进文本摘要，并以简洁的方式提供相关信息。为了生成相关的文本摘要，本文采用了基于查询的提取文本摘要方法，该方法包括根据从每个句子中检索到的四个最佳特征选择句子。在这个过程中，单词根据扩展查询的分数进行评分，句子根据四个重要特征进行评分，包括句子术语、位置、与第一句的相似度和专有名词。在DUC 2007数据集上对不同的ROUGE变体进行了广泛的实验，包括精度、召回率和f分数，在最佳情况下分别获得约44%、46%和45%的增益。我们观察到，所建议的方法在摘要生成方面优于DUC参与式系统和前沿方法。

{"title":"Query-based automatic text summarization using query expansion approach","authors":"Hiteshwar Kumar Azad","doi":"10.1016/j.datak.2025.102531","DOIUrl":"10.1016/j.datak.2025.102531","url":null,"abstract":"<div><div>The amount of information available on the Web has grown dramatically and continues to grow on a daily basis. The massive amount of Web data poses significant challenges to the reliability and accuracy of current information retrieval systems. The purpose of information retrieval is to discover relevant documents within a huge group of documents whose contents match a user-initiated query. Because most users struggle to formulate well-defined queries, the query expansion technique is critical for retrieving the most relevant information. Obtaining relevant results in a concise manner is a significant challenge in this scenario. Automatic text summarization can condense a lengthy document while retaining its informative content and key concepts. It could be a potential solution to information overload. This paper proposed a query-based automatic text summarization technique that employs query expansion to improve text summarization and provide the relevant information in a concise manner. To produce a relevant text summary, this article employs a query-based extractive text summarization method, which involves selecting sentences based on the four best features retrieved from each sentence. In this process, the words are scored by the expanded query’s score, and the sentences are scored by four important features, including sentence terms, position, similarity to the first sentence, and proper noun. Extensive experiments with different ROUGE variants on various evaluation metrics, including precision, recall, and F-score, were carried out on the DUC 2007 dataset, with gains of approximately 44%, 46%, and 45% respectively, in the best scenario. It is observed that the suggested approach outperforms both DUC participatory systems and cutting-edge approaches in summary generation.</div></div>","PeriodicalId":55184,"journal":{"name":"Data & Knowledge Engineering","volume":"162 ","pages":"Article 102531"},"PeriodicalIF":2.7,"publicationDate":"2025-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145532425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0