Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31275
Peter Lindes, Steven Jones
Humans acquire language and related concepts in a trajectory over a lifetime. Concepts for simple interaction with the world are learned before language. Later, words are learned to name these concepts along with structures needed to represent larger meanings. Eventually, language advances to where it can drive the learning of new concepts. Throughout this trajectory a language processing capability uses architectural mechanisms to process language using the knowledge already acquired. We assume that this growing body of knowledge is made up of small units of form-meaning mapping that can be composed in many ways, suggesting that these units are learned incrementally from experience. In prior work we have built a system to comprehend human language within an autonomous robot using knowledge in such units developed by hand. Here we propose a research program to develop the ability of an artificial agent to acquire this knowledge incrementally and autonomously from its experience in a similar trajectory. We then propose a strategy for evaluating this human-like learning system using a large benchmark created as a tool for training deep learning systems. We expect that our human-like learning system will produce better task performance from training on only a small subset of this benchmark.
{"title":"Modeling Human-Like Acquisition of Language and Concepts","authors":"Peter Lindes, Steven Jones","doi":"10.1609/aaaiss.v3i1.31275","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31275","url":null,"abstract":"Humans acquire language and related concepts in a trajectory over a lifetime. Concepts for simple interaction with the world are learned before language. Later, words are learned to name these concepts along with structures needed to represent larger meanings. Eventually, language advances to where it can drive the learning of new concepts. Throughout this trajectory a language processing capability uses architectural mechanisms to process language using the knowledge already acquired. We assume that this growing body of knowledge is made up of small units of form-meaning mapping that can be composed in many ways, suggesting that these units are learned incrementally from experience. In prior work we have built a system to comprehend human language within an autonomous robot using knowledge in such units developed by hand. Here we propose a research program to develop the ability of an artificial agent to acquire this knowledge incrementally and autonomously from its experience in a similar trajectory. We then propose a strategy for evaluating this human-like learning system using a large benchmark created as a tool for training deep learning systems. We expect that our human-like learning system will produce better task performance from training on only a small subset of this benchmark.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"81 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141121330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31272
Kenneth D. Forbus
There is ample psychological evidence that analogy is ubiquitous in human learning, suggesting that computational models of analogy can play important roles in AI systems that learn in human-like ways. This talk will provide evidence for this, focusing mostly on recent advances in hierarchical analogical learning and working-memory analogical generalizations.
{"title":"Analogy as the Swiss Army Knife of Human-like Learning","authors":"Kenneth D. Forbus","doi":"10.1609/aaaiss.v3i1.31272","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31272","url":null,"abstract":"There is ample psychological evidence that analogy is ubiquitous in human learning, suggesting that computational models of analogy can play important roles in AI systems that learn in human-like ways. This talk will provide evidence for this, focusing mostly on recent advances in hierarchical analogical learning and working-memory analogical generalizations.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"73 11","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141123214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31181
Stephen Russell, Fabio Montes Suros, Ashwin Kumar
For a large healthcare system, ignoring costs associated with managing the patient encounter denial process (staffing, contracts, etc.), total denial-related amounts can be more than $1B annually in gross charges. Being able to predict a denial before it occurs has the potential for tremendous savings. Using machine learning to predict denial has the potential to allow denial-preventing interventions. However, challenges of data imbalance make creating a single generalized model difficult. We employ two biased models in a hybrid voting scheme to achieve results that exceed the state-of-the art and allow for incremental predictions as the encounter progresses. The model had the added benefit of monitoring the human-driven denial process that affect the underlying distribution, on which the models’ bias is based.
{"title":"Exploiting Machine Learning Bias: Predicting Medical Denials","authors":"Stephen Russell, Fabio Montes Suros, Ashwin Kumar","doi":"10.1609/aaaiss.v3i1.31181","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31181","url":null,"abstract":"For a large healthcare system, ignoring costs associated with managing the patient encounter denial process (staffing, contracts, etc.), total denial-related amounts can be more than $1B annually in gross charges. Being able to predict a denial before it occurs has the potential for tremendous savings. Using machine learning to predict denial has the potential to allow denial-preventing interventions. However, challenges of data imbalance make creating a single generalized model difficult. We employ two biased models in a hybrid voting scheme to achieve results that exceed the state-of-the art and allow for incremental predictions as the encounter progresses. The model had the added benefit of monitoring the human-driven denial process that affect the underlying distribution, on which the models’ bias is based.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"60 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141123298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31278
Sheikh Mannan, V. Vimal, Paul DiZio, Nikhil Krishnaswamy
In this paper, we explore how humans and AIs trained to perform a virtual inverted pendulum (VIP) balancing task converge and differ in their learning and performance strategies. We create a visual analogue of disoriented IP balancing, as may be experienced by pilots suffering from spatial disorientation, and train AI models on data from human subjects performing a real-world disoriented balancing task. We then place the trained AI models in a dyadic human-in-the-loop (HITL) training setting. Episodes in which human subjects disagreed with AI actions were logged and used to fine-tune the AI model. Human subjects then performed the task while being given guidance from pretrained and dyadically fine-tuned versions of an AI model. We examine the effects of HITL training on AI performance, AI guidance on human performance, and the behavior patterns of human subjects and AI models during task performance. We find that in many cases, HITL training improves AI performance, AI guidance improves human performance, and after dyadic training the two converge on similar behavior patterns.
在本文中，我们探讨了人类和人工智能在执行虚拟倒立摆（VIP）平衡任务时，在学习和执行策略上是如何趋同和差异的。我们创建了一个迷失方向的 IP 平衡视觉模拟（飞行员可能会经历空间迷失），并根据执行真实世界迷失方向平衡任务的人类受试者的数据训练人工智能模型。然后，我们将训练好的人工智能模型置于双人环内（HITL）训练环境中。我们记录了人类受试者与人工智能操作不一致的情况，并利用这些情况对人工智能模型进行微调。然后，人类受试者在人工智能模型的预训练和双向微调版本的指导下执行任务。我们研究了 HITL 训练对人工智能性能的影响、人工智能对人类性能的指导，以及人类受试者和人工智能模型在执行任务过程中的行为模式。我们发现，在许多情况下，HITL 训练提高了人工智能的性能，人工智能指导提高了人类的性能，而且在经过双向训练后，两者的行为模式趋于相似。
{"title":"Embodying Human-Like Modes of Balance Control Through Human-In-the-Loop Dyadic Learning","authors":"Sheikh Mannan, V. Vimal, Paul DiZio, Nikhil Krishnaswamy","doi":"10.1609/aaaiss.v3i1.31278","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31278","url":null,"abstract":"In this paper, we explore how humans and AIs trained to perform a virtual inverted pendulum (VIP) balancing task converge and differ in their learning and performance strategies. We create a visual analogue of disoriented IP balancing, as may be experienced by pilots suffering from spatial disorientation, and train AI models on data from human subjects performing a real-world disoriented balancing task. We then place the trained AI models in a dyadic human-in-the-loop (HITL) training setting. Episodes in which human subjects disagreed with AI actions were logged and used to fine-tune the AI model. Human subjects then performed the task while being given guidance from pretrained and dyadically fine-tuned versions of an AI model. We examine the effects of HITL training on AI performance, AI guidance on human performance, and the behavior patterns of human subjects and AI models during task performance. We find that in many cases, HITL training improves AI performance, AI guidance improves human performance, and after dyadic training the two converge on similar behavior patterns.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"48 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141118648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31178
Shih-Hong Huang, Ting-Hao 'Kenneth' Huang
It is easy to assume that Large Language Models (LLMs) will seamlessly take over applications, especially those that are largely automated. In the case of conversational voice assistants, commercial systems have been widely deployed and used over the past decade. However, are we indeed on the cusp of the future we envisioned? There exists a social-technical gap between what people want to accomplish and the actual capability of technology. In this paper, we present a case study comparing two voice assistants built on Amazon Alexa: one employing a human-in-the-loop workflow, the other utilizes LLM to engage in conversations with users. In our comparison, we discovered that the issues arising in current human-in-the-loop and LLM systems are not identical. However, the presence of a set of similar issues in both systems leads us to believe that focusing on the interaction between users and systems is crucial, perhaps even more so than focusing solely on the underlying technology itself. Merely enhancing the performance of the workers or the models may not adequately address these issues. This observation prompts our research question: What are the overlooked contributing factors in the effort to improve the capabilities of voice assistants, which might not have been emphasized in prior research?
{"title":"On Replacing Humans with Large Language Models in Voice-Based Human-in-the-Loop Systems","authors":"Shih-Hong Huang, Ting-Hao 'Kenneth' Huang","doi":"10.1609/aaaiss.v3i1.31178","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31178","url":null,"abstract":"It is easy to assume that Large Language Models (LLMs) will seamlessly take over applications, especially those that are largely automated. In the case of conversational voice assistants, commercial systems have been widely deployed and used over the past decade. However, are we indeed on the cusp of the future we envisioned? There exists a social-technical gap between what people want to accomplish and the actual capability of technology. In this paper, we present a case study comparing two voice assistants built on Amazon Alexa: one employing a human-in-the-loop workflow, the other utilizes LLM to engage in conversations with users. In our comparison, we discovered that the issues arising in current human-in-the-loop and LLM systems are not identical. However, the presence of a set of similar issues in both systems leads us to believe that focusing on the interaction between users and systems is crucial, perhaps even more so than focusing solely on the underlying technology itself. Merely enhancing the performance of the workers or the models may not adequately address these issues. This observation prompts our research question: What are the overlooked contributing factors in the effort to improve the capabilities of voice assistants, which might not have been emphasized in prior research?","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"45 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141118958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31284
Irina Rabkina
Neurotypical adult humans are impeccably good social reasoners. Despite the occasional faux pas, we know how to interact in most social settings and how to consider others' points of view. Young children, on the other hand, do not. Social reasoning, like many of our most important skills, is learned. Much like human children, AI agents are not good social reasoners. While some algorithms can perform some aspects of social reasoning, we are a ways off from AI that can interact naturally and appropriately in the broad range of settings that people can. In this talk, I will argue that learning social reasoning via the same processes used by people will help AI agents reason--and interact--more like people do. Specifically, I will argue that children learn social reasoning via analogy, and that AI agents should, too. I will present evidence from cognitive modeling experiments demonstrating the former and AI experiments demonstrating the latter. I will also propose future directions for social reasoning research that both demonstrate the need for robust, human-like social reasoning in AI and test the utility of common approaches.
{"title":"Human-Like Learning of Social Reasoning via Analogy","authors":"Irina Rabkina","doi":"10.1609/aaaiss.v3i1.31284","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31284","url":null,"abstract":"Neurotypical adult humans are impeccably good social reasoners. Despite the occasional faux pas, we know how to interact in most social settings and how to consider others' points of view. Young children, on the other hand, do not. Social reasoning, like many of our most important skills, is learned. \u0000\u0000Much like human children, AI agents are not good social reasoners. While some algorithms can perform some aspects of social reasoning, we are a ways off from AI that can interact naturally and appropriately in the broad range of settings that people can. In this talk, I will argue that learning social reasoning via the same processes used by people will help AI agents reason--and interact--more like people do. Specifically, I will argue that children learn social reasoning via analogy, and that AI agents should, too. I will present evidence from cognitive modeling experiments demonstrating the former and AI experiments demonstrating the latter. I will also propose future directions for social reasoning research that both demonstrate the need for robust, human-like social reasoning in AI and test the utility of common approaches.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"10 15","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141119950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31270
Brendan Conway-Smith, Robert L. West
Large Language Models (LLMs) lack robust metacognitive learning abilities and depend on human-provided algorithms and prompts for learning and output generation. Metacognition involves processes that monitor and enhance cognition. Learning how to learn - metacognitive learning - is crucial for adapting and optimizing learning strategies over time. Although LLMs possess limited metacognitive abilities, they cannot autonomously refine or optimize these strategies. Humans possess innate mechanisms for metacognitive learning that enable at least two unique abilities: discerning which metacognitive strategies are best and automatizing learning strategies. These processes have been effectively modeled in the ACT-R cognitive architecture, providing insights on a path toward greater learning autonomy in AI. Incorporating human-like metacognitive learning abilities into AI could potentially lead to the development of more autonomous and versatile learning mechanisms, as well as improved problem-solving capabilities and performance across diverse tasks.
{"title":"Toward Autonomy: Metacognitive Learning for Enhanced AI Performance","authors":"Brendan Conway-Smith, Robert L. West","doi":"10.1609/aaaiss.v3i1.31270","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31270","url":null,"abstract":"Large Language Models (LLMs) lack robust metacognitive learning abilities and depend on human-provided algorithms and prompts for learning and output generation. Metacognition involves processes that monitor and enhance cognition. Learning how to learn - metacognitive learning - is crucial for adapting and optimizing learning strategies over time. Although LLMs possess limited metacognitive abilities, they cannot autonomously refine or optimize these strategies. Humans possess innate mechanisms for metacognitive learning that enable at least two unique abilities: discerning which metacognitive strategies are best and automatizing learning strategies. These processes have been effectively modeled in the ACT-R cognitive architecture, providing insights on a path toward greater learning autonomy in AI. Incorporating human-like metacognitive learning abilities into AI could potentially lead to the development of more autonomous and versatile learning mechanisms, as well as improved problem-solving capabilities and performance across diverse tasks.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"35 13","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141120698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31262
Devang Jayachandran, P. Maldikar, Tyler S. Love, Jeremy Blum
Generative Artificial Intelligence (AI) was incorporated into a competitive programming event that targeted undergraduate students, including those with little programming experience. The competition incorporated a range of challenge design approaches that promoted meaningful interaction with generative AI system, even while keeping the challenge difficulty level to an appropriate level. An analysis of survey responses and competition data showed that this format lowered barriers to participation, successfully engaged students throughout the competition, and increased the likelihood that they would participate in a similar event. In an extension of this work, a professional development workshop for high school teachers is being developed, along with a contest for high school students. Participant surveys and logs of interaction with the contest and generative AI systems will be analyzed to measure the effect of generative AI on student self-efficacy and suggest ways to integrate generative AI instruction into computer science curriculum.
{"title":"Leveraging Generative Artificial Intelligence to Broaden Participation in Computer Science","authors":"Devang Jayachandran, P. Maldikar, Tyler S. Love, Jeremy Blum","doi":"10.1609/aaaiss.v3i1.31262","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31262","url":null,"abstract":"Generative Artificial Intelligence (AI) was incorporated into a competitive programming event that targeted undergraduate students, including those with little programming experience. The competition incorporated a range of challenge design approaches that promoted meaningful interaction with generative AI system, even while keeping the challenge difficulty level to an appropriate level. An analysis of survey responses and competition data showed that this format lowered barriers to participation, successfully engaged students throughout the competition, and increased the likelihood that they would participate in a similar event. In an extension of this work, a professional development workshop for high school teachers is being developed, along with a contest for high school students. Participant surveys and logs of interaction with the contest and generative AI systems will be analyzed to measure the effect of generative AI on student self-efficacy and suggest ways to integrate generative AI instruction into computer science curriculum.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"21 17","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141120792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31209
N. Tan, Niket Tandon, David Wadden, Oyvind Tafjord, M. Gahegan, Michael Witbrock
Claim verification in scientific domains requires models that faithfully incorporate relevant knowledge from the ever-growing, vast existing literature. Unfaithful claim verifications can lead to misinformation such as those observed during the COVID-19 pandemic. Fact-checking systems often fail to capture the complex relationship between claims and evidence, especially with ambiguous claims and implicit assumptions. Relying only on current LLMs poses challenges due to hallucinations and information traceability issues. To address these challenges, our approach considers multiple viewpoints onto the scientific literature, enabling the assessment of contradictory arguments and implicit assumptions. Our proposed inference method adds faithful reasoning to large language models by distilling information from diverse, relevant scientific abstracts. This method provides a verdict label that can be weighted by the reputation of the scientific articles and an explanation that can be traced back to sources. Our findings demonstrate that humans not only perceive our explanation to be significantly superior to the off-the-shelf model, but they also evaluate it as faithfully enabling the tracing of evidence back to its original sources.
{"title":"Faithful Reasoning over Scientific Claims","authors":"N. Tan, Niket Tandon, David Wadden, Oyvind Tafjord, M. Gahegan, Michael Witbrock","doi":"10.1609/aaaiss.v3i1.31209","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31209","url":null,"abstract":"Claim verification in scientific domains requires models that faithfully incorporate relevant knowledge from the ever-growing, vast existing literature. \u0000Unfaithful claim verifications can lead to misinformation such as those observed during the COVID-19 pandemic. Fact-checking systems often fail to capture the complex relationship between claims and evidence, especially with ambiguous claims and implicit assumptions. Relying only on current LLMs poses challenges due to hallucinations and information traceability issues. To address these challenges, our approach considers multiple viewpoints onto the scientific literature, enabling the assessment of contradictory arguments and implicit assumptions. Our proposed inference method adds faithful reasoning to large language models by distilling information from diverse, relevant scientific abstracts. This method provides a verdict label that can be weighted by the reputation of the scientific articles and an explanation that can be traced back to sources. Our findings demonstrate that humans not only perceive our explanation to be significantly superior to the off-the-shelf model, but they also evaluate it as faithfully enabling the tracing of evidence back to its original sources.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"77 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141120971","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31227
Mehreen Tahir, Muhammad Intizar Ali
Federated Learning (FL) has received tremendous attention as a decentralized machine learning (ML) framework that allows distributed data owners to collaboratively train a global model without sharing raw data. Since FL trains the model directly on edge devices, the heterogeneity of participating clients in terms of data distribution, hardware capabilities and network connectivity can significantly impact the overall performance of FL systems. Optimizing for model accuracy could extend the training time due to the diverse and resource-constrained nature of edge devices while minimizing training time could compromise the model's accuracy. Effective client selection thus becomes crucial to ensure that the training process is not only efficient but also capitalizes on the diverse data and computational capabilities of different devices. To this end, we propose FedPROM, a novel framework that tackles client selection in FL as a multi-criteria optimization problem. By leveraging the PROMETHEE method, FedPROM ranks clients based on their suitability for a given FL task, considering multiple criteria such as system resources, network conditions, and data quality. This approach allows FedPROM to dynamically select the most appropriate set of clients for each learning round, optimizing both model accuracy and training efficiency. Our evaluations on diverse datasets demonstrate that FedPROM outperforms several state-of-the-art FL client selection protocols in terms of convergence speed, and accuracy, highlighting the framework's effectiveness and the importance of multi-criteria client selection in FL.
{"title":"Multi-Criterion Client Selection for Efficient Federated Learning","authors":"Mehreen Tahir, Muhammad Intizar Ali","doi":"10.1609/aaaiss.v3i1.31227","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31227","url":null,"abstract":"Federated Learning (FL) has received tremendous attention as a decentralized machine learning (ML) framework that allows distributed data owners to collaboratively train a global model without sharing raw data. Since FL trains the model directly on edge devices, the heterogeneity of participating clients in terms of data distribution, hardware capabilities and network connectivity can significantly impact the overall performance of FL systems. Optimizing for model accuracy could extend the training time due to the diverse and resource-constrained nature of edge devices while minimizing training time could compromise the model's accuracy. Effective client selection thus becomes crucial to ensure that the training process is not only efficient but also capitalizes on the diverse data and computational capabilities of different devices. To this end, we propose FedPROM, a novel framework that tackles client selection in FL as a multi-criteria optimization problem. By leveraging the PROMETHEE method, FedPROM ranks clients based on their suitability for a given FL task, considering multiple criteria such as system resources, network conditions, and data quality. This approach allows FedPROM to dynamically select the most appropriate set of clients for each learning round, optimizing both model accuracy and training efficiency. Our evaluations on diverse datasets demonstrate that FedPROM outperforms several state-of-the-art FL client selection protocols in terms of convergence speed, and accuracy, highlighting the framework's effectiveness and the importance of multi-criteria client selection in FL.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":"30 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141119064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：