Information Systems最新文献_第2页

A Bayesian framework for robust local intrinsic dimensionality estimation 鲁棒局部固有维数估计的贝叶斯框架

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2026-01-02 DOI: 10.1016/j.is.2025.102668

Zaher Joukhadar , Hanxun Huang , Sarah Monazam Erfani , Ricardo J.G.B. Campello , Michael E. Houle , James Bailey

Local Intrinsic Dimensionality (LID) is a measure of data complexity in the vicinity of a query point. In this work, we propose a novel Bayesian framework for LID estimation that improves robustness and accuracy, especially in scenarios with small neighborhood sizes (

k \leq 10

), where maintaining locality is critical. Our framework allows the incorporation of both informative and non-informative priors, enabling the integration of prior knowledge to enhance the estimation process. Using this framework, we derive new LID estimators and provide insights into transitional ones. Furthermore, we propose aggregation methods using linear and logarithmic pooling to combine multiple LID posteriors. These methods allow for principled integration of LID estimates across different training states of Deep Neural Networks (DNNs), such as epochs, thereby improving estimation stability and performance. We also derive a posterior predictive distribution (PPD) for modeling and synthesizing nearest-neighbor distances. Experiments demonstrate that the LID Bayesian estimation methods achieves a balanced approach between reducing variance and lowering squared bias, often leading to lower mean squared error (MSE) values for small number of nearest neighbors.

局部固有维数（LID）是对查询点附近数据复杂度的度量。在这项工作中，我们提出了一种新的贝叶斯框架，用于LID估计，提高了鲁棒性和准确性，特别是在小邻域大小（k≤10）的情况下，保持局部性至关重要。我们的框架允许信息先验和非信息先验的结合，使先验知识的集成能够增强估计过程。使用这个框架，我们得到了新的LID估计器，并提供了对过渡估计器的见解。此外，我们提出了使用线性和对数池的聚合方法来组合多个LID后置。这些方法允许在深度神经网络（dnn）的不同训练状态（如epoch）之间有原则地集成LID估计，从而提高估计的稳定性和性能。我们还推导了一个用于建模和合成最近邻距离的后验预测分布（PPD）。实验表明，LID贝叶斯估计方法在减少方差和降低平方偏差之间取得了平衡，对于少数最近邻，通常会导致较低的均方误差（MSE）值。

{"title":"A Bayesian framework for robust local intrinsic dimensionality estimation","authors":"Zaher Joukhadar , Hanxun Huang , Sarah Monazam Erfani , Ricardo J.G.B. Campello , Michael E. Houle , James Bailey","doi":"10.1016/j.is.2025.102668","DOIUrl":"10.1016/j.is.2025.102668","url":null,"abstract":"<div><div>Local Intrinsic Dimensionality (LID) is a measure of data complexity in the vicinity of a query point. In this work, we propose a novel Bayesian framework for LID estimation that improves robustness and accuracy, especially in scenarios with small neighborhood sizes (<span><math><mrow><mi>k</mi><mo>≤</mo><mn>10</mn></mrow></math></span>), where maintaining locality is critical. Our framework allows the incorporation of both informative and non-informative priors, enabling the integration of prior knowledge to enhance the estimation process. Using this framework, we derive new LID estimators and provide insights into transitional ones. Furthermore, we propose aggregation methods using linear and logarithmic pooling to combine multiple LID posteriors. These methods allow for principled integration of LID estimates across different training states of Deep Neural Networks (DNNs), such as epochs, thereby improving estimation stability and performance. We also derive a posterior predictive distribution (PPD) for modeling and synthesizing nearest-neighbor distances. Experiments demonstrate that the LID Bayesian estimation methods achieves a balanced approach between reducing variance and lowering squared bias, often leading to lower mean squared error (MSE) values for small number of nearest neighbors.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102668"},"PeriodicalIF":3.4,"publicationDate":"2026-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146023093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Applying organizational mining to discover agent systems from event data 应用组织挖掘技术从事件数据中发现代理系统

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-31 DOI: 10.1016/j.is.2025.102669

Qingtan Shen , Artem Polyvyanyy , Nir Lipovetzky , Timotheus Kampik

Agent system mining is a recently introduced type of process mining that takes a bottom-up approach to the data-driven analysis of socio-technical systems that execute business processes in organizations. Instead of the top-down approach used in conventional process mining that studies a system in terms of its global state evolution, agent system mining analyzes the system as if it is composed of autonomous agents, each with its local state and behavior, interacting with other agents and the environment to contribute to the emerging global behavior of the business process. Recently, Agent Miner, the first algorithm for discovering agent systems from event data generated by process-aware information systems, has been proposed. The quality of the agent systems discovered by this algorithm depends on the quality of the agent types (or agents), which are identified from the available information about agent instances in the data. In this paper, we study the suitability and benefits of using methods from the organizational mining subarea of process mining for identifying agent types. The experiments we conduct over real-world datasets confirm the usefulness of such methods for discovering simple, modular, and accurate agent systems. These conclusions are grounded in quality metrics such as the size of discovered models (simplicity), Louvain modularity and the Gini coefficient (modularity), and precision and recall (accuracy). The results confirm the benefits of using organizational mining for identifying agent types when discovering agent systems from event data, leading to the construction of models of superior quality in precision, recall, and simplicity compared to models constructed by state-of-the-art conventional process discovery algorithms.

代理系统挖掘是最近引入的一种流程挖掘类型，它采用自底向上的方法对组织中执行业务流程的社会技术系统进行数据驱动分析。与传统流程挖掘中使用的从全局状态演变研究系统的自顶向下方法不同，代理系统挖掘将系统视为由自治代理组成，每个代理都具有其局部状态和行为，与其他代理和环境相互作用，以促进业务流程的新兴全局行为。最近提出了Agent Miner算法，这是第一个从进程感知信息系统生成的事件数据中发现Agent系统的算法。该算法发现的代理系统的质量取决于代理类型（或代理）的质量，这些类型是从数据中关于代理实例的可用信息中识别出来的。在本文中，我们研究了使用过程挖掘的组织挖掘子领域的方法来识别代理类型的适用性和效益。我们在真实世界数据集上进行的实验证实了这些方法对于发现简单、模块化和准确的代理系统的有用性。这些结论是基于质量指标，如发现模型的大小（简单性），鲁文模块化和基尼系数（模块化），以及精度和召回率（准确性）。结果证实了在从事件数据中发现代理系统时使用组织挖掘来识别代理类型的好处，与使用最先进的常规流程发现算法构建的模型相比，可以构建精度、召回率和简单性更高的模型。

{"title":"Applying organizational mining to discover agent systems from event data","authors":"Qingtan Shen , Artem Polyvyanyy , Nir Lipovetzky , Timotheus Kampik","doi":"10.1016/j.is.2025.102669","DOIUrl":"10.1016/j.is.2025.102669","url":null,"abstract":"<div><div>Agent system mining is a recently introduced type of process mining that takes a bottom-up approach to the data-driven analysis of socio-technical systems that execute business processes in organizations. Instead of the top-down approach used in conventional process mining that studies a system in terms of its global state evolution, agent system mining analyzes the system as if it is composed of autonomous agents, each with its local state and behavior, interacting with other agents and the environment to contribute to the emerging global behavior of the business process. Recently, Agent Miner, the first algorithm for discovering agent systems from event data generated by process-aware information systems, has been proposed. The quality of the agent systems discovered by this algorithm depends on the quality of the agent types (or agents), which are identified from the available information about agent instances in the data. In this paper, we study the suitability and benefits of using methods from the organizational mining subarea of process mining for identifying agent types. The experiments we conduct over real-world datasets confirm the usefulness of such methods for discovering simple, modular, and accurate agent systems. These conclusions are grounded in quality metrics such as the size of discovered models (simplicity), Louvain modularity and the Gini coefficient (modularity), and precision and recall (accuracy). The results confirm the benefits of using organizational mining for identifying agent types when discovering agent systems from event data, leading to the construction of models of superior quality in precision, recall, and simplicity compared to models constructed by state-of-the-art conventional process discovery algorithms.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102669"},"PeriodicalIF":3.4,"publicationDate":"2025-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145925901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Graph-based similarity measures for the structural comparison of process traces 用于过程轨迹结构比较的基于图的相似性度量

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-26 DOI: 10.1016/j.is.2025.102671

Clemens Schreiber , Amine Abbad-Andaloussi , Andrea Burattin , Andreas Oberweis , Barbara Weber

Similarity measures are commonly applied for a variety of process mining techniques, such as trace clustering, conformance checking, and event abstraction. Yet, these measures generally fail to recognize similarity based on structural process features, such as the order of activities, loops, skips, choices, and parallelism. To make this more explicit, we propose a set of properties that allow to evaluate, what kind of structural features are reflected by a similarity measure. We further propose a novel approach leveraging existing graph-based algorithms and instance graphs to extract high-level structural features (loops, skips, choices, and parallelism) from traces, such that they can be used to extend and improve existing similarity measures. These algorithms are well-established in graph theory and can be computed efficiently. Finally, we provide an evaluation of the proposed approach based on synthetic and real-world datasets. The evaluation provides evidence that the additional graph-based features can substantially improve the similarity comparison of traces in several cases. This applies in particular for the comparison of user behavior (e.g., based on eye tracking data) where structural features enable the detection of specific behavioral patterns.

相似性度量通常应用于各种过程挖掘技术，例如跟踪聚类、一致性检查和事件抽象。然而，这些措施通常不能识别基于结构过程特征的相似性，如活动顺序、循环、跳过、选择和并行性。为了使这一点更明确，我们提出了一组属性，允许评估什么样的结构特征是由相似性度量反映出来的。我们进一步提出了一种新的方法，利用现有的基于图的算法和实例图从轨迹中提取高级结构特征（循环、跳过、选择和并行性），这样它们就可以用来扩展和改进现有的相似性度量。这些算法在图论中已经得到了很好的验证，并且可以进行高效的计算。最后，我们基于合成和真实世界的数据集对所提出的方法进行了评估。评估提供的证据表明，在一些情况下，附加的基于图的特征可以大大提高轨迹的相似性比较。这尤其适用于用户行为的比较（例如，基于眼动追踪数据），其中结构特征可以检测特定的行为模式。

{"title":"Graph-based similarity measures for the structural comparison of process traces","authors":"Clemens Schreiber , Amine Abbad-Andaloussi , Andrea Burattin , Andreas Oberweis , Barbara Weber","doi":"10.1016/j.is.2025.102671","DOIUrl":"10.1016/j.is.2025.102671","url":null,"abstract":"<div><div>Similarity measures are commonly applied for a variety of process mining techniques, such as trace clustering, conformance checking, and event abstraction. Yet, these measures generally fail to recognize similarity based on structural process features, such as the order of activities, loops, skips, choices, and parallelism. To make this more explicit, we propose a set of properties that allow to evaluate, what kind of structural features are reflected by a similarity measure. We further propose a novel approach leveraging existing graph-based algorithms and instance graphs to extract high-level structural features (loops, skips, choices, and parallelism) from traces, such that they can be used to extend and improve existing similarity measures. These algorithms are well-established in graph theory and can be computed efficiently. Finally, we provide an evaluation of the proposed approach based on synthetic and real-world datasets. The evaluation provides evidence that the additional graph-based features can substantially improve the similarity comparison of traces in several cases. This applies in particular for the comparison of user behavior (e.g., based on eye tracking data) where structural features enable the detection of specific behavioral patterns.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102671"},"PeriodicalIF":3.4,"publicationDate":"2025-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145925902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Improving the understandability of declarative process discovery results using easyDeclare 使用easyDeclare提高声明性过程发现结果的可理解性

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-22 DOI: 10.1016/j.is.2025.102667

Graziano Blasilli , Lauren S. Ferro , Simone Lenti , Fabrizio Maria Maggi , Andrea Marrella , Tiziana Catarci

Declarative process models allow us to capture the behavior of a business process through temporal constraints on the evolution of process activities. In process mining, declarative process discovery focuses on deriving these constraints from event logs. Although the semantic aspects of declarative processes have been extensively investigated, there has been less focus on designing declarative visual notations that enhance model understanding and support analysts in solving process mining tasks. To improve the human understandability of declarative process models, in this paper, we present easyDeclare, a novel visual notation to specify declarative process models using the Declare language. easyDeclare was developed with consideration of the well-established Moody’s design principles. We conducted extensive user experiments to demonstrate that easyDeclare, when compared with the original graphical representation of Declare, reduces the cognitive load required to interpret Declare models of increasing complexity, making it a promising alternative to enhancing overall comprehension of declarative process discovery tasks.

声明性流程模型允许我们通过流程活动演化的时间约束来捕获业务流程的行为。在流程挖掘中，声明性流程发现侧重于从事件日志中派生这些约束。尽管已经对声明性过程的语义方面进行了广泛的研究，但很少有人关注如何设计声明性可视化符号来增强模型理解并支持分析人员解决过程挖掘任务。为了提高人类对声明性过程模型的可理解性，本文提出了一种新的使用Declare语言来指定声明性过程模型的可视化符号easyDeclare。easyDeclare的开发考虑了穆迪完善的设计原则。我们进行了大量的用户实验来证明，与Declare的原始图形表示形式相比，easyDeclare减少了解释日益复杂的Declare模型所需的认知负荷，使其成为增强声明性过程发现任务的整体理解的有希望的替代方案。

{"title":"Improving the understandability of declarative process discovery results using easyDeclare","authors":"Graziano Blasilli , Lauren S. Ferro , Simone Lenti , Fabrizio Maria Maggi , Andrea Marrella , Tiziana Catarci","doi":"10.1016/j.is.2025.102667","DOIUrl":"10.1016/j.is.2025.102667","url":null,"abstract":"<div><div>Declarative process models allow us to capture the behavior of a business process through temporal constraints on the evolution of process activities. In process mining, declarative process discovery focuses on deriving these constraints from event logs. Although the semantic aspects of declarative processes have been extensively investigated, there has been less focus on designing declarative visual notations that enhance model understanding and support analysts in solving process mining tasks. To improve the human understandability of declarative process models, in this paper, we present <span>easyDeclare</span>, a novel visual notation to specify declarative process models using the <span>Declare</span> language. <span>easyDeclare</span> was developed with consideration of the well-established Moody’s design principles. We conducted extensive user experiments to demonstrate that <span>easyDeclare</span>, when compared with the original graphical representation of <span>Declare</span>, reduces the cognitive load required to interpret <span>Declare</span> models of increasing complexity, making it a promising alternative to enhancing overall comprehension of declarative process discovery tasks.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102667"},"PeriodicalIF":3.4,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145884521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

VCR: Interpretable and interactive debugging of object detection models with visual concepts 具有可视化概念的对象检测模型的可解释和交互式调试

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-12 DOI: 10.1016/j.is.2025.102652

Jie Jeff Xu , Saahir Dhanani , Jorge Piazentin Ono , Wenbin He , Liu Ren , Kexin Rong

Computer vision models can make systematic errors, performing well on average but substantially worse on particular subsets (or slices) of data. In this work, we introduce Visual Concept Reviewer (VCR), a human-in-the-loop slice discovery framework that enables practitioners to interactively discover and understand systematic errors in object-detection models via novel use of visual concepts–semantically meaningful and frequently recurring image segments representing objects, parts, or abstract properties.

Leveraging recent advances in vision foundation models, VCR automatically generates segment-level visual concepts that serve as interpretable primitives for diagnosing issues in object-detection models, while also supporting lightweight human supervision when needed. VCR combines visual concepts with metadata in a tabular format and adapts frequent itemset mining techniques to identify common absences and presences of concepts associated with poor model performance at interactive speeds. VCR also keeps humans in the loop for interpretation and refinement at each step of the slice discovery process. We demonstrate VCR’s effectiveness and scalability through a new evaluation benchmark with 1713 slice discovery settings across three datasets. A user study with six expert industry machine learning scientists and engineers provides qualitative evidence of VCR’s utility in real-world workflows.

计算机视觉模型可能会产生系统错误，平均表现良好，但在特定的数据子集（或切片）上表现得更差。在这项工作中，我们介绍了视觉概念审查器（VCR），这是一个人在循环中的切片发现框架，使从业者能够通过新颖地使用视觉概念（表示对象、部件或抽象属性的语义上有意义且经常重复出现的图像片段）来交互式地发现和理解对象检测模型中的系统错误。利用视觉基础模型的最新进展，VCR自动生成分段级视觉概念，作为对象检测模型中诊断问题的可解释原语，同时在需要时还支持轻量级的人工监督。VCR将可视化概念与表格格式的元数据结合起来，并采用频繁的项集挖掘技术来识别与交互速度较差的模型性能相关的概念的常见缺失和存在。VCR还使人类在切片发现过程的每一步都能进行解释和改进。我们通过一个新的评估基准，在三个数据集上使用1713个切片发现设置，展示了VCR的有效性和可扩展性。六位行业机器学习专家和工程师的用户研究为VCR在实际工作流程中的实用性提供了定性证据。

{"title":"VCR: Interpretable and interactive debugging of object detection models with visual concepts","authors":"Jie Jeff Xu , Saahir Dhanani , Jorge Piazentin Ono , Wenbin He , Liu Ren , Kexin Rong","doi":"10.1016/j.is.2025.102652","DOIUrl":"10.1016/j.is.2025.102652","url":null,"abstract":"<div><div>Computer vision models can make systematic errors, performing well on average but substantially worse on particular subsets (or slices) of data. In this work, we introduce Visual Concept Reviewer (VCR), a human-in-the-loop slice discovery framework that enables practitioners to interactively discover and understand systematic errors in object-detection models via novel use of visual concepts–semantically meaningful and frequently recurring image segments representing objects, parts, or abstract properties.</div><div>Leveraging recent advances in vision foundation models, <span>VCR</span> automatically generates segment-level visual concepts that serve as interpretable primitives for diagnosing issues in object-detection models, while also supporting lightweight human supervision when needed. <span>VCR</span> combines visual concepts with metadata in a tabular format and adapts frequent itemset mining techniques to identify common absences and presences of concepts associated with poor model performance at interactive speeds. <span>VCR</span> also keeps humans in the loop for interpretation and refinement at each step of the slice discovery process. We demonstrate VCR’s effectiveness and scalability through a new evaluation benchmark with 1713 slice discovery settings across three datasets. A user study with six expert industry machine learning scientists and engineers provides qualitative evidence of VCR’s utility in real-world workflows.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102652"},"PeriodicalIF":3.4,"publicationDate":"2025-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145791230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HILTS: Human-LLM collaboration for effective data labeling HILTS：人类-法学硕士协作有效的数据标签

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-12 DOI: 10.1016/j.is.2025.102660

Juliana Barbosa, Eduarda Alencar, Grace Fan, Aécio Santos, Juliana Freire

The growing complexity and volume of data highlight the importance of learning-based classifiers across diverse tasks, from medical diagnosis to environmental monitoring. A common and impactful use case is data triage—efficiently identifying rare, relevant instances in large, imbalanced datasets. This is crucial for enabling domain experts to focus on what matters most. However, traditional supervised learning approaches often struggle with scalability due to the high cost and time required for manual labeling.

We introduce HILTS (Human-In-the-loop Learn To Sample), a framework designed to tackle these limitations. HILTS leverages Large Language Models (LLMs) for automated initial labeling and strategically incorporates human expertise through advanced active learning techniques. It selects diverse and representative samples for pseudo-labeling and identifies highly uncertain or likely incorrect LLM labels for targeted human review. This focused use of human effort maximizes the value of domain expertise while minimizing annotation overhead.

Our system reduces human labeling effort by up to 80% while outperforming few-shot foundation models such as GPT-4 by over 5% in F1-score in some scenarios—all at a significantly lower cost. HILTS also shows clear improvements over fully automated pseudo-labeling approaches and proves especially effective in handling class imbalance in real-world datasets. Its adaptability and efficiency make it a practical and scalable solution for high-stakes, domain-specific data triage tasks.

不断增长的复杂性和数据量凸显了基于学习的分类器在各种任务中的重要性，从医疗诊断到环境监测。一个常见且有影响力的用例是数据分类——在大型、不平衡的数据集中有效地识别罕见的、相关的实例。这对于使领域专家专注于最重要的事情是至关重要的。然而，传统的监督学习方法由于人工标注的高成本和时间要求，往往难以实现可扩展性。我们介绍了HILTS (Human-In-the-loop Learn To Sample)，这是一个旨在解决这些限制的框架。HILTS利用大型语言模型（llm）进行自动初始标记，并通过先进的主动学习技术战略性地结合人类专业知识。它选择多样化和代表性的样本进行伪标记，并识别高度不确定或可能不正确的LLM标签，用于有针对性的人工审查。这种对人力的集中使用使领域专业知识的价值最大化，同时使注释开销最小化。我们的系统减少了多达80%的人工标记工作，同时在某些情况下，在f1得分上比GPT-4等少量基础模型高出5%以上，所有这些都大大降低了成本。HILTS还显示出比全自动伪标签方法有明显的改进，并证明在处理真实数据集中的类不平衡方面特别有效。它的适应性和效率使其成为高风险、特定于领域的数据分类任务的实用且可扩展的解决方案。

{"title":"HILTS: Human-LLM collaboration for effective data labeling","authors":"Juliana Barbosa, Eduarda Alencar, Grace Fan, Aécio Santos, Juliana Freire","doi":"10.1016/j.is.2025.102660","DOIUrl":"10.1016/j.is.2025.102660","url":null,"abstract":"<div><div>The growing complexity and volume of data highlight the importance of learning-based classifiers across diverse tasks, from medical diagnosis to environmental monitoring. A common and impactful use case is data triage—efficiently identifying rare, relevant instances in large, imbalanced datasets. This is crucial for enabling domain experts to focus on what matters most. However, traditional supervised learning approaches often struggle with scalability due to the high cost and time required for manual labeling.</div><div>We introduce HILTS (Human-In-the-loop Learn To Sample), a framework designed to tackle these limitations. HILTS leverages Large Language Models (LLMs) for automated initial labeling and strategically incorporates human expertise through advanced active learning techniques. It selects diverse and representative samples for pseudo-labeling and identifies highly uncertain or likely incorrect LLM labels for targeted human review. This focused use of human effort maximizes the value of domain expertise while minimizing annotation overhead.</div><div>Our system reduces human labeling effort by up to 80% while outperforming few-shot foundation models such as GPT-4 by over 5% in F1-score in some scenarios—all at a significantly lower cost. HILTS also shows clear improvements over fully automated pseudo-labeling approaches and proves especially effective in handling class imbalance in real-world datasets. Its adaptability and efficiency make it a practical and scalable solution for high-stakes, domain-specific data triage tasks.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102660"},"PeriodicalIF":3.4,"publicationDate":"2025-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145791324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ACTER: Activity Customization through Timely and Explainable Recommendations ACTER：通过及时和可解释的建议定制活动

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-11 DOI: 10.1016/j.is.2025.102666

Anna Dalla Vecchia, Niccolò Marastoni, Barbara Oliboni, Elisa Quintarelli

The proliferation of sensors, including wearable devices, has significantly increased the volume of generated data, opening up new opportunities for personalized recommendations. This paper presents ACTER (Activity Customization through Timely and Explainable Recommendations), an integrated framework to provide contextual, timely, explainable, and user-specific recommendations. Thanks to the sequential rule mining algorithm ALBA (AgedLookBackApriori), we extract totally ordered sequential rules to uncover hidden insights from temporal data, ultimately improving a predefined target parameter related to the selected application domain. An aging mechanism is applied to ensure that recommendations remain relevant, giving more weight to newer information while still considering older data. In addition, our framework leverages historical data to also infer personalized, contextual information, allowing us to adapt the predefined context—usually set at the design stage—more dynamically and expressly. The experimental results of the ACTER evaluation confirm that integrating ad-hoc contexts mined from historical data into the recommender system yields more accurate suggestions.

包括可穿戴设备在内的传感器的激增大大增加了生成的数据量，为个性化推荐开辟了新的机会。本文介绍了ACTER（通过及时和可解释的建议进行活动定制），这是一个集成框架，用于提供上下文相关的、及时的、可解释的和特定于用户的建议。得益于顺序规则挖掘算法ALBA (AgedLookBackApriori)，我们提取了完全有序的顺序规则，以从时间数据中发现隐藏的见解，最终改进了与所选应用程序领域相关的预定义目标参数。使用老化机制来确保建议保持相关性，在考虑旧数据的同时给予新信息更多权重。此外，我们的框架还利用历史数据来推断个性化的上下文信息，从而允许我们更动态、更明确地调整预定义的上下文（通常在设计阶段设置）。ACTER评估的实验结果证实，将从历史数据中挖掘的临时上下文集成到推荐系统中可以产生更准确的建议。

{"title":"ACTER: Activity Customization through Timely and Explainable Recommendations","authors":"Anna Dalla Vecchia, Niccolò Marastoni, Barbara Oliboni, Elisa Quintarelli","doi":"10.1016/j.is.2025.102666","DOIUrl":"10.1016/j.is.2025.102666","url":null,"abstract":"<div><div>The proliferation of sensors, including wearable devices, has significantly increased the volume of generated data, opening up new opportunities for personalized recommendations. This paper presents ACTER (Activity Customization through Timely and Explainable Recommendations), an integrated framework to provide contextual, timely, explainable, and user-specific recommendations. Thanks to the sequential rule mining algorithm ALBA (AgedLookBackApriori), we extract totally ordered sequential rules to uncover hidden insights from temporal data, ultimately improving a predefined target parameter related to the selected application domain. An aging mechanism is applied to ensure that recommendations remain relevant, giving more weight to newer information while still considering older data. In addition, our framework leverages historical data to also infer personalized, contextual information, allowing us to adapt the predefined context—usually set at the design stage—more dynamically and expressly. The experimental results of the ACTER evaluation confirm that integrating ad-hoc contexts mined from historical data into the recommender system yields more accurate suggestions.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102666"},"PeriodicalIF":3.4,"publicationDate":"2025-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145737870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

From precision to perception: Human-in-the-loop evaluation of keyword extraction for internet-scale contextual advertising 从精确到感知：互联网规模上下文广告关键字提取的人在循环评估

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-11 DOI: 10.1016/j.is.2025.102665

Jingwen Cai , Sara Leckner , Johanna Björklund

Keyword extraction is a foundational task in natural language processing, underpinning countless real-world applications. One of these is contextual advertising, where keywords help predict the topical congruence between ads and their surrounding media contexts to enhance advertising effectiveness. Recent advances in artificial intelligence have improved keyword extraction capabilities but also introduced concerns about computational cost. Moreover, although the end-user experience is of vital importance, human evaluation of keyword extraction performances remains under-explored. This study provides a comparative evaluation of prevalent keyword extraction algorithms with different levels of complexity represented by TF-IDF, KeyBERT, and Llama 2. To evaluate their effectiveness, a mixed-methods approach is employed, combining quantitative benchmarking with qualitative assessments from 855 participants through four survey-based experiments. The findings demonstrate that KeyBERT achieves an effective balance between user preferences and computational efficiency, compared to the other algorithms. We observe a clear overall preference for gold-standard keywords, but there is a misalignment between algorithmic benchmark performance and user ratings. This reveals a long-overlooked gap between traditional precision-focused metrics and user-perceived algorithm efficiency. The study underscores the importance of human-in-the-loop evaluation methodologies and proposes analytical tools to facilitate their implementation.

关键字提取是自然语言处理的一项基础任务，支撑着无数现实世界的应用。其中之一是上下文广告，其中关键词有助于预测广告与周围媒体上下文之间的主题一致性，以提高广告效果。人工智能的最新进展提高了关键字提取能力，但也引入了对计算成本的担忧。此外，尽管最终用户体验至关重要，但关键字提取性能的人类评估仍未得到充分探索。本研究对以TF-IDF、KeyBERT和Llama 2为代表的不同复杂度的流行关键字提取算法进行了比较评价。为了评估其有效性，采用了一种混合方法，通过四个基于调查的实验，将855名参与者的定量基准与定性评估相结合。研究结果表明，与其他算法相比，KeyBERT在用户偏好和计算效率之间实现了有效的平衡。我们观察到对黄金标准关键字的明显总体偏好，但算法基准性能和用户评级之间存在不一致。这揭示了传统的以精度为中心的指标和用户感知的算法效率之间长期被忽视的差距。该研究强调了人在循环评估方法的重要性，并提出了促进其实施的分析工具。

{"title":"From precision to perception: Human-in-the-loop evaluation of keyword extraction for internet-scale contextual advertising","authors":"Jingwen Cai , Sara Leckner , Johanna Björklund","doi":"10.1016/j.is.2025.102665","DOIUrl":"10.1016/j.is.2025.102665","url":null,"abstract":"<div><div>Keyword extraction is a foundational task in natural language processing, underpinning countless real-world applications. One of these is contextual advertising, where keywords help predict the topical congruence between ads and their surrounding media contexts to enhance advertising effectiveness. Recent advances in artificial intelligence have improved keyword extraction capabilities but also introduced concerns about computational cost. Moreover, although the end-user experience is of vital importance, human evaluation of keyword extraction performances remains under-explored. This study provides a comparative evaluation of prevalent keyword extraction algorithms with different levels of complexity represented by TF-IDF, KeyBERT, and Llama 2. To evaluate their effectiveness, a mixed-methods approach is employed, combining quantitative benchmarking with qualitative assessments from 855 participants through four survey-based experiments. The findings demonstrate that KeyBERT achieves an effective balance between user preferences and computational efficiency, compared to the other algorithms. We observe a clear overall preference for gold-standard keywords, but there is a misalignment between algorithmic benchmark performance and user ratings. This reveals a long-overlooked gap between traditional precision-focused metrics and user-perceived algorithm efficiency. The study underscores the importance of human-in-the-loop evaluation methodologies and proposes analytical tools to facilitate their implementation.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102665"},"PeriodicalIF":3.4,"publicationDate":"2025-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145737933","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Visualizing repetition in process execution variants from partially ordered event data 从部分有序的事件数据中可视化流程执行变体中的重复

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-05 DOI: 10.1016/j.is.2025.102664

Ariba Siddiqui , Francesca Zerbato , Daniel Schuster

Operational processes often exhibit concurrency, where the execution of activities can overlap in time. Moreover, repetitions of activities, both intentional (e.g., iterative tasks) and unintentional (e.g., rework) often occur. Existing process mining techniques and visualizations largely assume sequential event data, making it difficult to analyze repetitions in partially ordered event data, which better captures real-world process behavior. We address this gap by introducing a novel arc-diagram-based visualization that highlights recurring activity patterns within individual process execution variants. This approach allows analysts to intuitively detect repetitions that are otherwise obscured in raw data or traditional variant views. We validate the usefulness and ease of use of the proposed visualization through a user study with process mining experts and provide an implementation of our contribution in an open-source tool, supporting practical adoption.

操作流程通常表现为并发性，其中活动的执行可以在时间上重叠。此外，活动的重复，有意的（例如，迭代任务）和无意的（例如，返工）经常发生。现有的流程挖掘技术和可视化在很大程度上假设事件数据是顺序的，这使得分析部分有序事件数据中的重复变得困难，而部分有序事件数据能够更好地捕捉真实的流程行为。我们通过引入一种新颖的基于弧线图的可视化来解决这一差距，该可视化突出了单个流程执行变体中重复出现的活动模式。这种方法允许分析人员直观地检测在原始数据或传统变体视图中被掩盖的重复。我们通过与过程挖掘专家的用户研究验证了所建议的可视化的有用性和易用性，并在开源工具中提供了我们的贡献的实现，支持实际采用。

引用次数: 0

Efficient allocation of shared resources across multiple processes 跨多个进程有效地分配共享资源

IF 3.4 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Systems

Pub Date : 2025-12-05 DOI: 10.1016/j.is.2025.102663

Kiran Busch, Henrik Leopold

Effective resource allocation is crucial for optimizing business processes. Yet, most existing methods focus solely on single-process optimization, overlooking the interdependencies present in multi-process environments. This limitation results in inefficient resource allocation, and scalability challenges. To address this gap, we propose MuProMAC (Multi-Process Multi-Agent Coordination), a novel reinforcement learning-based method designed to optimize resource allocation across multiple interdependent business processes. Unlike prior methods, MuProMAC is the first online resource allocation method that explicitly models the interdependencies between processes and dynamically balances competing resource demands to minimize global average cycle time. We evaluate our method in five multi-process scenarios with different levels of resource contention, comparing it against state-of-the-art online resource allocation methods and three simple baselines. Our results show that MuProMAC is consistently among the top-performing methods in shared-resource environments. It achieves low cycle times and stable performance across different workload conditions, outperforming existing methods through its strong adaptability to evolving business processes and increasing complexity.

有效的资源分配对于优化业务流程至关重要。然而，大多数现有方法只关注单进程优化，忽略了多进程环境中存在的相互依赖性。这种限制导致资源分配效率低下，并对可伸缩性构成挑战。为了解决这一差距，我们提出了MuProMAC（多进程多代理协调），这是一种新的基于强化学习的方法，旨在优化多个相互依赖的业务流程之间的资源分配。与先前的方法不同，MuProMAC是第一个在线资源分配方法，它显式地建模进程之间的相互依赖关系，并动态平衡竞争资源需求，以最小化全局平均周期时间。我们在五个具有不同资源争用水平的多进程场景中评估了我们的方法，并将其与最先进的在线资源分配方法和三个简单的基线进行了比较。我们的结果表明，在共享资源环境中，MuProMAC始终是性能最好的方法之一。它在不同的工作负载条件下实现了低周期时间和稳定的性能，通过对不断发展的业务流程和不断增加的复杂性的强适应性，优于现有的方法。

{"title":"Efficient allocation of shared resources across multiple processes","authors":"Kiran Busch, Henrik Leopold","doi":"10.1016/j.is.2025.102663","DOIUrl":"10.1016/j.is.2025.102663","url":null,"abstract":"<div><div>Effective resource allocation is crucial for optimizing business processes. Yet, most existing methods focus solely on single-process optimization, overlooking the interdependencies present in multi-process environments. This limitation results in inefficient resource allocation, and scalability challenges. To address this gap, we propose MuProMAC (Multi-Process Multi-Agent Coordination), a novel reinforcement learning-based method designed to optimize resource allocation across multiple interdependent business processes. Unlike prior methods, MuProMAC is the first online resource allocation method that explicitly models the interdependencies between processes and dynamically balances competing resource demands to minimize global average cycle time. We evaluate our method in five multi-process scenarios with different levels of resource contention, comparing it against state-of-the-art online resource allocation methods and three simple baselines. Our results show that MuProMAC is consistently among the top-performing methods in shared-resource environments. It achieves low cycle times and stable performance across different workload conditions, outperforming existing methods through its strong adaptability to evolving business processes and increasing complexity.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"138 ","pages":"Article 102663"},"PeriodicalIF":3.4,"publicationDate":"2025-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145737934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0