Benedito Ribeiro Neto, Bianchi Meiguins, Tiago Araújo, Carlos dos Santos
Applications using fiducial markers have evolved across sectors such as industry, health, and education. Markers are effective because their highly distinguishable visual patterns and varied morphologies allow for high-accuracy pose estimation. However, designing a robust fiducial marker system is difficult and requires specific strategies to ensure reliability for applications such as photogrammetry and robot localization. This study aims to address this challenge through a systematic study of 88 articles selected using snowball methodology. This study focused on marker design characteristics to analyze different types of robustness. The goal of this study was to formally define fiducial markers, explore their intrinsic and extrinsic characteristics, and produce a taxonomy covering morphological and algorithmic aspects. The primary outcome is a comprehensive taxonomy and theoretical framework that provides best practices, guiding researchers in developing or employing robust fiducial markers tailored to their specific applications.
{"title":"Artificial Markers: A Comprehensive Systematic Review and Design Framework","authors":"Benedito Ribeiro Neto, Bianchi Meiguins, Tiago Araújo, Carlos dos Santos","doi":"10.1145/3793661","DOIUrl":"https://doi.org/10.1145/3793661","url":null,"abstract":"Applications using fiducial markers have evolved across sectors such as industry, health, and education. Markers are effective because their highly distinguishable visual patterns and varied morphologies allow for high-accuracy pose estimation. However, designing a robust fiducial marker system is difficult and requires specific strategies to ensure reliability for applications such as photogrammetry and robot localization. This study aims to address this challenge through a systematic study of 88 articles selected using snowball methodology. This study focused on marker design characteristics to analyze different types of robustness. The goal of this study was to formally define fiducial markers, explore their intrinsic and extrinsic characteristics, and produce a taxonomy covering morphological and algorithmic aspects. The primary outcome is a comprehensive taxonomy and theoretical framework that provides best practices, guiding researchers in developing or employing robust fiducial markers tailored to their specific applications.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"40 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146044844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hanh Tran, Matej Martinc, Jaya Caporusso, Julien Delaunay, Antoine Doucet, Senja Pollak
Automatic terminology or term extraction (ATE) is a Natural Language Processing (NLP) task intended to automatically identify specialized terms present in domain-specific corpora. As units of knowledge in a specific field of expertise, extracted terms are not only beneficial for several terminographical tasks, but also support and improve several complex downstream tasks, e.g., information retrieval, machine translation, topic detection, and sentiment analysis. ATE systems and datasets annotated for the task at hand have been studied and developed for decades, but more recent approaches have increasingly involved novel neural systems. Despite a large amount of new research on ATE tasks, systematic survey studies covering novel neural approaches are lacking, especially when it comes to the usage of large-scale language models (LLMs). We present a comprehensive survey of neural approaches to ATE, focusing on transformer-based neural models and the recent generative approaches based on LLMs. The study also compares these systems and previous ML-based approaches, which employed feature engineering and non-neural supervised learning algorithms.
{"title":"Recent Advances in Automatic Term Extraction: A Comprehensive Survey","authors":"Hanh Tran, Matej Martinc, Jaya Caporusso, Julien Delaunay, Antoine Doucet, Senja Pollak","doi":"10.1145/3787584","DOIUrl":"https://doi.org/10.1145/3787584","url":null,"abstract":"Automatic terminology or term extraction (ATE) is a Natural Language Processing (NLP) task intended to automatically identify specialized terms present in domain-specific corpora. As units of knowledge in a specific field of expertise, extracted terms are not only beneficial for several terminographical tasks, but also support and improve several complex downstream tasks, e.g., information retrieval, machine translation, topic detection, and sentiment analysis. ATE systems and datasets annotated for the task at hand have been studied and developed for decades, but more recent approaches have increasingly involved novel neural systems. Despite a large amount of new research on ATE tasks, systematic survey studies covering novel neural approaches are lacking, especially when it comes to the usage of large-scale language models (LLMs). We present a comprehensive survey of neural approaches to ATE, focusing on transformer-based neural models and the recent generative approaches based on LLMs. The study also compares these systems and previous ML-based approaches, which employed feature engineering and non-neural supervised learning algorithms.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"87 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146044924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shriyank Somvanshi, Md Monzurul Islam, Amir Rafe, Anannya Ghosh Tusti, Arka Chakraborty, Anika Baitullah, Tausif Islam Chowdhury, Nawaf Alnawmasi, Anandi Dutta, Subasish Das
Mechanistic interpretability seeks to reverse-engineer the internal logic of neural networks by uncovering human-understandable circuits, algorithms, and causal structures that drive model behavior. Unlike post hoc explanations that describe what models do, this paradigm focuses on why and how they compute, tracing information flow through neurons, attention heads, and activation pathways. This survey provides a high-level synthesis of the field-highlighting its motivation, conceptual foundations, and methodological taxonomy rather than enumerating individual techniques. We organize mechanistic interpretability across three abstraction layers- neurons , circuits , and algorithms -and three evaluation perspectives: behavioral , counterfactual , and causal . We further discuss representative approaches and toolchains that enable structural analysis of modern AI systems, outlining how mechanistic interpretability bridges theoretical insights with practical transparency. Despite rapid progress, challenges persist in scaling these analyses to frontier models, resolving polysemantic representations, and establishing standardized causal benchmarks. By connecting historical evolution, current methodologies, and emerging research directions, this survey aims to provide an integrative framework for understanding how mechanistic interpretability can support transparency, reliability, and governance in large-scale AI.
{"title":"Bridging the Black Box: A Survey on Mechanistic Interpretability in AI","authors":"Shriyank Somvanshi, Md Monzurul Islam, Amir Rafe, Anannya Ghosh Tusti, Arka Chakraborty, Anika Baitullah, Tausif Islam Chowdhury, Nawaf Alnawmasi, Anandi Dutta, Subasish Das","doi":"10.1145/3787104","DOIUrl":"https://doi.org/10.1145/3787104","url":null,"abstract":"Mechanistic interpretability seeks to reverse-engineer the internal logic of neural networks by uncovering human-understandable circuits, algorithms, and causal structures that drive model behavior. Unlike post hoc explanations that describe what models do, this paradigm focuses on why and how they compute, tracing information flow through neurons, attention heads, and activation pathways. This survey provides a high-level synthesis of the field-highlighting its motivation, conceptual foundations, and methodological taxonomy rather than enumerating individual techniques. We organize mechanistic interpretability across three abstraction layers- <jats:italic toggle=\"yes\">neurons</jats:italic> , <jats:italic toggle=\"yes\">circuits</jats:italic> , and <jats:italic toggle=\"yes\">algorithms</jats:italic> -and three evaluation perspectives: <jats:italic toggle=\"yes\">behavioral</jats:italic> , <jats:italic toggle=\"yes\">counterfactual</jats:italic> , and <jats:italic toggle=\"yes\">causal</jats:italic> . We further discuss representative approaches and toolchains that enable structural analysis of modern AI systems, outlining how mechanistic interpretability bridges theoretical insights with practical transparency. Despite rapid progress, challenges persist in scaling these analyses to frontier models, resolving polysemantic representations, and establishing standardized causal benchmarks. By connecting historical evolution, current methodologies, and emerging research directions, this survey aims to provide an integrative framework for understanding how mechanistic interpretability can support transparency, reliability, and governance in large-scale AI.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"1 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146042589","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Massimo Regona, Tan Yigitcanlar, Carol Hon, Melissa Teo
Artificial intelligence (AI) is reshaping industries by enhancing efficiency and accuracy, yet its adoption remains contingent on user trust, which is frequently undermined by concerns over privacy, algorithmic bias, and security vulnerabilities. Trust in AI depends on principles such as transparency, accountability, safety, privacy, robustness, and reliability, all of which are central to user confidence. However, existing studies often overlook the interdependencies among these factors and their collective influence on user engagement. Guided by Trust Theory and a systematic literature review employing the PRISMA protocol, this study examines the trust indicators most relevant to high-stakes applications. The review reveals that transparency and communication are consistently prioritised, while adaptability and affordability remain underexplored, highlighting gaps in current scholarship. Trust in AI evolves as users gain experience with these systems, with reliability, predictability, and ethical alignment emerging as critical determinants. Addressing persistent challenges such as bias, data protection, and fairness is essential for reinforcing trust and enabling broader adoption of AI across industries.
{"title":"Building Trust in Artificial Intelligence: A Systematic Review through the Lens of Trust Theory","authors":"Massimo Regona, Tan Yigitcanlar, Carol Hon, Melissa Teo","doi":"10.1145/3789256","DOIUrl":"https://doi.org/10.1145/3789256","url":null,"abstract":"Artificial intelligence (AI) is reshaping industries by enhancing efficiency and accuracy, yet its adoption remains contingent on user trust, which is frequently undermined by concerns over privacy, algorithmic bias, and security vulnerabilities. Trust in AI depends on principles such as transparency, accountability, safety, privacy, robustness, and reliability, all of which are central to user confidence. However, existing studies often overlook the interdependencies among these factors and their collective influence on user engagement. Guided by Trust Theory and a systematic literature review employing the PRISMA protocol, this study examines the trust indicators most relevant to high-stakes applications. The review reveals that transparency and communication are consistently prioritised, while adaptability and affordability remain underexplored, highlighting gaps in current scholarship. Trust in AI evolves as users gain experience with these systems, with reliability, predictability, and ethical alignment emerging as critical determinants. Addressing persistent challenges such as bias, data protection, and fairness is essential for reinforcing trust and enabling broader adoption of AI across industries.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"124 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145986540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
IoT data demands are growing, with Distributed Ledger Technologies (DLTs) offering secure data management, provided they can meet scaling and efficiency requirements that are more restrictive than in conventional application environments. This paper comprehensively surveys 27 DLTs of varying paradigms and implementation methods, proposes a scoring method for determining DLT-IoT integration suitability, and then applies that method to the surveyed DLTs. Six DLTs were shortlisted as the most promising, which were then subjected to in-depth analysis around three IoT use cases: health-IoT, e-commerce and automotive manufacturing. We discuss the viability of lightweight DLTs and identify crucial future research directions.
{"title":"Integration of IoT and Distributed Ledger Technologies: A Survey, Challenges, and Future Directions","authors":"Jusak Jusak, Steve Kerrison","doi":"10.1145/3789255","DOIUrl":"https://doi.org/10.1145/3789255","url":null,"abstract":"IoT data demands are growing, with Distributed Ledger Technologies (DLTs) offering secure data management, provided they can meet scaling and efficiency requirements that are more restrictive than in conventional application environments. This paper comprehensively surveys 27 DLTs of varying paradigms and implementation methods, proposes a scoring method for determining DLT-IoT integration suitability, and then applies that method to the surveyed DLTs. Six DLTs were shortlisted as the most promising, which were then subjected to in-depth analysis around three IoT use cases: health-IoT, e-commerce and automotive manufacturing. We discuss the viability of lightweight DLTs and identify crucial future research directions.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"20 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145986542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lianyu Hu, Mudi Jiang, Junjie Dong, Xinying Liu, Zengyou He
In recent years, much of the research on clustering algorithms has primarily focused on enhancing their accuracy and efficiency, frequently at the expense of interpretability. However, as these methods are increasingly being applied in high-stakes domains such as healthcare, finance, and autonomous systems, the need of transparent and interpretable clustering outcomes has become a critical concern. This is not only necessary for gaining user trust but also for satisfying the growing ethical and regulatory demands in these fields. Ensuring that decisions derived from clustering algorithms can be clearly understood and justified is now a fundamental requirement. To address this need, this paper provides a comprehensive and structured review of the current state of explainable clustering algorithms, identifying key criteria to distinguish between various methods. These insights can effectively assist researchers in making informed decisions about the most suitable explainable clustering methods for specific application contexts, while also promoting the development and adoption of clustering algorithms that are both efficient and transparent. For convenient access and reference, an open repository organizes representative and emerging interpretable clustering methods under the taxonomy proposed in this survey, available at https://github.com/hulianyu/Awesome-Interpretable-Clustering
{"title":"Interpretable Clustering: A Survey","authors":"Lianyu Hu, Mudi Jiang, Junjie Dong, Xinying Liu, Zengyou He","doi":"10.1145/3789495","DOIUrl":"https://doi.org/10.1145/3789495","url":null,"abstract":"In recent years, much of the research on clustering algorithms has primarily focused on enhancing their accuracy and efficiency, frequently at the expense of interpretability. However, as these methods are increasingly being applied in high-stakes domains such as healthcare, finance, and autonomous systems, the need of transparent and interpretable clustering outcomes has become a critical concern. This is not only necessary for gaining user trust but also for satisfying the growing ethical and regulatory demands in these fields. Ensuring that decisions derived from clustering algorithms can be clearly understood and justified is now a fundamental requirement. To address this need, this paper provides a comprehensive and structured review of the current state of explainable clustering algorithms, identifying key criteria to distinguish between various methods. These insights can effectively assist researchers in making informed decisions about the most suitable explainable clustering methods for specific application contexts, while also promoting the development and adoption of clustering algorithms that are both efficient and transparent. For convenient access and reference, an open repository organizes representative and emerging interpretable clustering methods under the taxonomy proposed in this survey, available at https://github.com/hulianyu/Awesome-Interpretable-Clustering","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"55 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145986496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kunal Chaturvedi, Nicholas Yang, Donald Dansereau, Christopher Lovejoy, Ali Braytee, Miriam Welgampola, Mukesh Prasad
Benign Positional Vertigo (BPV) is a common and correctable cause of dizziness worldwide, accompanied by unique nystagmus characteristics that can be recognized by trained healthcare workers. Nystagmus is an involuntary eye movement, consisting of an initial slow phase often followed by a subsequent quick phase, and is a key indicator of vestibular disorders including BPV. This review focuses on the application of machine learning Models for BPV diagnosis through the classification of nystagmus patterns. We examine the advancements in machine learning and deep learning techniques for nystagmus detection, highlighting the transition from traditional methods to more sophisticated approaches. We include a comprehensive analysis of recent studies, detailing the methodologies, datasets, and results of various models. We discuss the ongoing challenges and future directions in this domain, emphasizing the potential of these technologies to assist diagnosis of BPV by untrained clinicians and the promise of better patient outcomes. Through a systematic literature review process, this paper identifies gaps in current research and suggests areas for future exploration, aiming to support the application of artificial intelligence in the diagnosis of a common vertigo subtype.
{"title":"Diagnosis of Benign Positional Vertigo: A Systematic Review of Machine Learning and Deep Learning within Videonystagmography","authors":"Kunal Chaturvedi, Nicholas Yang, Donald Dansereau, Christopher Lovejoy, Ali Braytee, Miriam Welgampola, Mukesh Prasad","doi":"10.1145/3789494","DOIUrl":"https://doi.org/10.1145/3789494","url":null,"abstract":"Benign Positional Vertigo (BPV) is a common and correctable cause of dizziness worldwide, accompanied by unique nystagmus characteristics that can be recognized by trained healthcare workers. Nystagmus is an involuntary eye movement, consisting of an initial slow phase often followed by a subsequent quick phase, and is a key indicator of vestibular disorders including BPV. This review focuses on the application of machine learning Models for BPV diagnosis through the classification of nystagmus patterns. We examine the advancements in machine learning and deep learning techniques for nystagmus detection, highlighting the transition from traditional methods to more sophisticated approaches. We include a comprehensive analysis of recent studies, detailing the methodologies, datasets, and results of various models. We discuss the ongoing challenges and future directions in this domain, emphasizing the potential of these technologies to assist diagnosis of BPV by untrained clinicians and the promise of better patient outcomes. Through a systematic literature review process, this paper identifies gaps in current research and suggests areas for future exploration, aiming to support the application of artificial intelligence in the diagnosis of a common vertigo subtype.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"19 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145986544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Li Huang, Sophie Ebersold, Alexander Kogtenkov, Bertrand Meyer, Yinling Liu
The technology of formal software verification has made spectacular advances, but how much does it actually benefit the development of practical software? Considerable disagreement remains about the practicality of building systems with mechanically-checked proofs of correctness. Is this prospect confined to a few expensive, life-critical projects, or can the idea be applied to a wide segment of the software industry? To help answer this question, the present survey examines a range of projects, in various application areas, that have produced formally verified systems and deployed them for actual use. It considers the technologies used, the form of verification applied, the results obtained, and the lessons that the software industry should draw regarding its ability to benefit from formal verification techniques and tools. Note: this version is the extended article, covering all the systems identified as relevant. A shorter version, covering only a selection, is also available.
{"title":"Lessons from Formally Verified Deployed Software Systems","authors":"Li Huang, Sophie Ebersold, Alexander Kogtenkov, Bertrand Meyer, Yinling Liu","doi":"10.1145/3785652","DOIUrl":"https://doi.org/10.1145/3785652","url":null,"abstract":"The technology of formal software verification has made spectacular advances, but how much does it actually benefit the development of practical software? Considerable disagreement remains about the practicality of building systems with mechanically-checked proofs of correctness. Is this prospect confined to a few expensive, life-critical projects, or can the idea be applied to a wide segment of the software industry? To help answer this question, the present survey examines a range of projects, in various application areas, that have produced formally verified systems and deployed them for actual use. It considers the technologies used, the form of verification applied, the results obtained, and the lessons that the software industry should draw regarding its ability to benefit from formal verification techniques and tools. Note: this version is the extended article, covering all the systems identified as relevant. A shorter version, covering only a selection, is also available.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"56 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145986543","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The swift evolution of Large Language Models (LLMs) like the GPT family, LLaMA, ChatGLM, and Qwen represents significant progress in artificial intelligence research. Despite their remarkable capabilities in generating content, these models encounter substantial challenges when producing structured outputs and engaging in dynamic interactions, particularly when they need to retrieve external information in real time. To address these limitations, researchers have developed the ”Function Calling” paradigm. This approach enables language models to analyze user inquiries and engage with defined functions, thereby facilitating precise responses through connections to external sources, including databases, programming interfaces, and live data streams. This functionality has been successfully implemented across numerous sectors such as finance analytics, healthcare systems, and service operations. The implementation of function calling comprises three essential phases: preparation, execution, and processing. The preparation phase encompasses query analysis and function identification. During execution, the system evaluates whether a function is necessary, extracts relevant parameters, and oversees the operation. The processing phase concentrates on analyzing outcomes and crafting appropriate responses. Each phase presents unique difficulties, ranging from accurately selecting functions to managing complex parameter extraction and ensuring reliable execution. Researchers have established various evaluation frameworks and metrics to assess function calling performance, including success rates, computational efficiency, parameter extraction accuracy, and response quality indicators such as ROUGE-L evaluation scores. This survey systematically reviews the current landscape of function calling in LLMs, analyzing technical challenges, examining existing solutions, and discussing evaluation methodologies. We particularly focus on practical implementations and industrial applications, providing insights into both current achievements and future directions in this rapidly evolving field. For a comprehensive collection of related research papers and the Appendix file, please refer to our repository at GitHub.
{"title":"Function Calling in Large Language Models: Industrial Practices, Challenges, and Future Directions","authors":"Maolin Wang, Yingyi Zhang, Bowen Yu, Bingguang Hao, Cunyin Peng, Yicheng Chen, Wei Zhou, Jinjie Gu, Chenyi Zhuang, Ruocheng Guo, Wanyu Wang, Xiangyu Zhao","doi":"10.1145/3788284","DOIUrl":"https://doi.org/10.1145/3788284","url":null,"abstract":"The swift evolution of Large Language Models (LLMs) like the GPT family, LLaMA, ChatGLM, and Qwen represents significant progress in artificial intelligence research. Despite their remarkable capabilities in generating content, these models encounter substantial challenges when producing structured outputs and engaging in dynamic interactions, particularly when they need to retrieve external information in real time. To address these limitations, researchers have developed the ”Function Calling” paradigm. This approach enables language models to analyze user inquiries and engage with defined functions, thereby facilitating precise responses through connections to external sources, including databases, programming interfaces, and live data streams. This functionality has been successfully implemented across numerous sectors such as finance analytics, healthcare systems, and service operations. The implementation of function calling comprises three essential phases: preparation, execution, and processing. The preparation phase encompasses query analysis and function identification. During execution, the system evaluates whether a function is necessary, extracts relevant parameters, and oversees the operation. The processing phase concentrates on analyzing outcomes and crafting appropriate responses. Each phase presents unique difficulties, ranging from accurately selecting functions to managing complex parameter extraction and ensuring reliable execution. Researchers have established various evaluation frameworks and metrics to assess function calling performance, including success rates, computational efficiency, parameter extraction accuracy, and response quality indicators such as ROUGE-L evaluation scores. This survey systematically reviews the current landscape of function calling in LLMs, analyzing technical challenges, examining existing solutions, and discussing evaluation methodologies. We particularly focus on practical implementations and industrial applications, providing insights into both current achievements and future directions in this rapidly evolving field. For a comprehensive collection of related research papers and the Appendix file, please refer to our repository at GitHub.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"57 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145968629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas Baumer, Sascha Kern, Ludwig Fuchs, Günther Pernul
Identity and Access Management (IAM) challenges organizations, requiring carefully orchestrated processes, technologies, and authorizations. Despite its strategic relevance, we lack a consolidated scientific understanding of IAM metrics and their alignment with IAM goals, like security, compliance, and operational efficiency. This systematic review aims to identify and classify IAM metrics from the literature to support evidence-based IAM. It links collected metrics to IAM goals and audiences. The literature review followed the guidelines of Levy and Ellis. It includes publications from databases SpringerLink, AIS eLibrary, IEEE Explore, ScienceDirect, ACM Digital Library, and relevant cross-referenced publications. The search strategy used keyword combinations, like ”Identity and Access Management” and ”Metrics,” since 2000. We screened and included publications based on eligibility criteria for relevance, quality, and the explicit presentation of IAM metrics, resulting in sixty publications. The review identified 43 IAM metrics, categorized by seven perspectives derived from IAM goals and processes. Each metric was analyzed by its target, impact on IAM goals, and relevant audiences. The synthesis shows that the literature lacks unified terminology and frameworks for IAM metrics. Future research includes standardizing terminology, linking metrics and targets to maturity levels, and establishing IAM process metrics. The DEVISE project funded this work. It was not registered in PROSPERO.
身份和访问管理（Identity and Access Management， IAM）对组织提出了挑战，需要精心编排流程、技术和授权。尽管它具有战略相关性，但我们对IAM指标及其与IAM目标（如安全性、合规性和运营效率）的一致性缺乏统一的科学理解。本系统综述旨在从文献中识别和分类IAM指标，以支持基于证据的IAM。它将收集到的指标与IAM目标和受众联系起来。文献综述遵循Levy和Ellis的指导方针。它包括来自数据库SpringerLink、AIS Library、IEEE Explore、ScienceDirect、ACM Digital Library和相关交叉引用出版物的出版物。自2000年以来，搜索策略使用关键字组合，如“身份和访问管理”和“度量”。我们根据相关性、质量和IAM指标的明确呈现的资格标准筛选并纳入了出版物，共纳入了60篇出版物。该综述确定了43个IAM指标，并从IAM目标和流程的7个角度进行了分类。每个指标都根据其目标、对IAM目标的影响和相关受众进行了分析。综合表明，文献缺乏统一的术语和框架的IAM指标。未来的研究包括标准化术语，将指标和目标与成熟度级别联系起来，以及建立IAM流程指标。设计项目资助了这项工作。它没有在普洛斯彼罗登记。
{"title":"Identity and Access Management Metrics: A Systematic Review","authors":"Thomas Baumer, Sascha Kern, Ludwig Fuchs, Günther Pernul","doi":"10.1145/3788858","DOIUrl":"https://doi.org/10.1145/3788858","url":null,"abstract":"Identity and Access Management (IAM) challenges organizations, requiring carefully orchestrated processes, technologies, and authorizations. Despite its strategic relevance, we lack a consolidated scientific understanding of IAM metrics and their alignment with IAM goals, like security, compliance, and operational efficiency. This systematic review aims to identify and classify IAM metrics from the literature to support evidence-based IAM. It links collected metrics to IAM goals and audiences. The literature review followed the guidelines of Levy and Ellis. It includes publications from databases SpringerLink, AIS eLibrary, IEEE Explore, ScienceDirect, ACM Digital Library, and relevant cross-referenced publications. The search strategy used keyword combinations, like ”Identity and Access Management” and ”Metrics,” since 2000. We screened and included publications based on eligibility criteria for relevance, quality, and the explicit presentation of IAM metrics, resulting in sixty publications. The review identified 43 IAM metrics, categorized by seven perspectives derived from IAM goals and processes. Each metric was analyzed by its target, impact on IAM goals, and relevant audiences. The synthesis shows that the literature lacks unified terminology and frameworks for IAM metrics. Future research includes standardizing terminology, linking metrics and targets to maturity levels, and establishing IAM process metrics. The DEVISE project funded this work. It was not registered in PROSPERO.","PeriodicalId":50926,"journal":{"name":"ACM Computing Surveys","volume":"4 1","pages":""},"PeriodicalIF":16.6,"publicationDate":"2026-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145968627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：