Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31197
Andreas Martin, Charuta Pande, Sandro Schwander, A. Ajuwon, Christoph Pimmer
FAQs are widely used to respond to users’ knowledge needs within knowledge domains. While LLM might be a promising way to address user questions, they are still prone to hallucinations i.e., inaccurate or wrong responses, which, can, inter alia, lead to massive problems, including, but not limited to, ethical issues. As a part of the healthcare coach chatbot for young Nigerian HIV clients, the need to meet their information needs through FAQs is one of the main coaching requirements. In this paper, we explore if domain knowledge in HIV FAQs can be represented as text embeddings to retrieve similar questions matching user queries, thus improving the understanding of the chatbot and the satisfaction of the users. Specifically, we describe our approach to developing an FAQ chatbot for the domain of HIV. We used a predefined FAQ question-answer knowledge base in English and Pidgin co-created by HIV clients and experts from Nigeria and Switzerland. The results of the post-engagement survey show that the chatbot mostly understood the user’s questions and could identify relevant matching questions and retrieve an appropriate response.
{"title":"Domain-specific Embeddings for Question-Answering Systems: FAQs for Health Coaching","authors":"Andreas Martin, Charuta Pande, Sandro Schwander, A. Ajuwon, Christoph Pimmer","doi":"10.1609/aaaiss.v3i1.31197","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31197","url":null,"abstract":"FAQs are widely used to respond to users’ knowledge needs within knowledge domains. While LLM might be a promising way to address user questions, they are still prone to hallucinations i.e., inaccurate or wrong responses, which, can, inter alia, lead to massive problems, including, but not limited to, ethical issues. As a part of the healthcare coach chatbot for young Nigerian HIV clients, the need to meet their information needs through FAQs is one of the main coaching requirements. In this paper, we explore if domain knowledge in HIV FAQs can be represented as text embeddings to retrieve similar questions matching user queries, thus improving the understanding of the chatbot and the satisfaction of the users. Specifically, we describe our approach to developing an FAQ chatbot for the domain of HIV. We used a predefined FAQ question-answer knowledge base in English and Pidgin co-created by HIV clients and experts from Nigeria and Switzerland. The results of the post-engagement survey show that the chatbot mostly understood the user’s questions and could identify relevant matching questions and retrieve an appropriate response.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141119441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31247
Andy Skumanich, Han Kyul Kim
A rapidly developing threat to societal well-being is from misinformation widely spread on social media. Even more concerning is ”mal-info” (malicious) which is amplified on certain social networks. Now there is an additional dimension to that threat, which is the use of Generative AI to deliberately augment the mis-info and mal-info. This paper highlights some of the ”fringe” social media channels which have a high level of mal-info as characterized by our AI/ML algorithms. We discuss various channels and focus on one in particular, ”GAB”, as representative of the potential negative impacts. We outline some of the current mal-info as an example. We capture elements, and observe the trends in time. We provide a set of AI/ML modes which can characterize the mal-info and allow for capture, tracking, and potentially for responding or for mitigation. We highlight the concern about malicious agents using GenAI for deliberate mal-info messaging specifically to disrupt societal well being. We suggest the characterizations presented as a methodology for initiating a more deliberate and quantitative approach to address these harmful aspects of social media which would adversely impact societal well being. The article highlights the potential for ”mal-info,” including disinfo, cyberbullying, and hate speech, to disrupt segments of society. The amplification of mal-info can result in serious real-world consequences such as mass shootings. Despite attempts to introduce moderation on major platforms like Facebook and to some extent on X/Twitter, there are now growing social networks such as Gab, Gettr, and Bitchute that offer completely unmoderated spaces. This paper presents an introduction to these platforms and the initial results of a semiquantitative analysis of Gab’s posts. The paper examines several characterization modes using text analysis. The paper emphasizes the developing dangerous use of generative AI algorithms by Gab and other fringe platforms, highlighting the risks to societal well being. This article aims to lay the foundation for capturing, monitoring, and mitigating these risks.
社交媒体上广泛传播的错误信息对社会福祉的威胁正在迅速发展。更令人担忧的是在某些社交网络上被放大的 "恶意信息"。现在，这种威胁又多了一个层面，那就是使用生成式人工智能来故意增强错误信息和恶意信息。本文重点介绍了一些 "边缘 "社交媒体渠道，根据我们的人工智能/ML 算法，这些渠道存在大量恶意信息。我们讨论了各种渠道，并重点讨论了 "GAB"，它是潜在负面影响的代表。我们以当前的一些恶意信息为例进行概述。我们捕捉要素，观察时间趋势。我们提供了一套人工智能/人工智能模式，可以描述恶意信息的特征，并允许捕获、跟踪和潜在的响应或缓解。我们强调了对恶意代理利用 GenAI 故意发送恶意信息以破坏社会福祉的担忧。我们建议将所提出的特征描述作为一种方法，以启动一种更深思熟虑的定量方法来解决社交媒体中这些会对社会福祉产生不利影响的有害方面。文章强调了 "恶意信息"（包括虚假信息、网络欺凌和仇恨言论）扰乱社会各阶层的可能性。恶意信息的放大可能导致严重的现实后果，如大规模枪击事件。尽管 Facebook 等主要平台试图引入节制，X/Twitter 也在一定程度上引入了节制，但现在，Gab、Gettr 和 Bitchute 等社交网络也在不断发展，它们提供了完全不受节制的空间。本文介绍了这些平台，以及对 Gab 帖子进行半定量分析的初步结果。本文通过文本分析研究了几种表征模式。本文强调了 Gab 和其他边缘平台正在危险地使用生成式人工智能算法，突出强调了对社会福祉的风险。本文旨在为捕捉、监控和降低这些风险奠定基础。
{"title":"Modes of Tracking Mal-Info in Social Media with AI/ML Tools to Help Mitigate Harmful GenAI for Improved Societal Well Being","authors":"Andy Skumanich, Han Kyul Kim","doi":"10.1609/aaaiss.v3i1.31247","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31247","url":null,"abstract":"A rapidly developing threat to societal well-being is from misinformation widely spread on social media. Even more concerning is ”mal-info” (malicious) which is amplified on certain social networks. Now there is an additional dimension to that threat, which is the use of Generative AI to deliberately augment the mis-info and mal-info. This paper highlights some of the ”fringe” social media channels which have a high level of mal-info as characterized by our AI/ML algorithms. We discuss various channels and focus on one in particular, ”GAB”, as representative of the potential negative impacts. We outline some of the current mal-info as an example. We capture elements, and observe the trends in time. We provide a set of AI/ML modes which can characterize the mal-info and allow for capture, tracking, and potentially for\u0000responding or for mitigation. We highlight the concern about malicious agents using GenAI for deliberate mal-info messaging specifically to disrupt societal well being. We suggest the characterizations presented as a methodology for initiating a more deliberate and quantitative approach to address these harmful aspects of social media which would adversely impact societal well being. \u0000\u0000The article highlights the potential for ”mal-info,” including disinfo, cyberbullying, and hate speech, to disrupt segments of society. The amplification of mal-info can result in serious real-world consequences such as mass shootings. Despite attempts to introduce moderation on major platforms like Facebook and to some extent on X/Twitter, there are now growing social networks such as Gab, Gettr, and Bitchute that offer completely unmoderated spaces. This paper presents an introduction\u0000to these platforms and the initial results of a semiquantitative analysis of Gab’s posts. The paper examines several characterization modes using text analysis. The paper emphasizes the developing dangerous use of generative AI algorithms by Gab and other fringe platforms, highlighting the risks to societal well being. This article aims to lay the foundation for capturing, monitoring, and mitigating these risks.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141118645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31225
Nojan Sheybani, F. Koushanfar
Inherently, federated learning (FL) robustness is very challenging to guarantee, especially when trying to maintain privacy. Compared to standard ML settings, FL's open training process allows for malicious clients to easily go under the radar. Alongside this, malicious clients can easily collude to attack the training process continuously, and without detection. FL models are also still susceptible to attacks on standard ML training procedures. This massive attack surface makes balancing the tradeoff between utility, practicality, robustness, and privacy extremely challenging. While there have been proposed defenses to attacks using popular privacy-preserving primitives, such as fully homomorphic encryption, they often face trouble balancing an all-important question that is present in all privacy-preserving systems: How much utility and practicality am I willing to give up to ensure privacy and robustness? In this work, we discuss a practical approach towards secure and robust FL and the challenges that face this field of emerging research.
从本质上讲，联合学习（FL）的稳健性很难保证，尤其是在试图维护隐私时。与标准 ML 设置相比，FL 的开放式训练过程可以让恶意客户端轻易地隐藏起来。与此同时，恶意客户端可以轻易地串通起来，在不被发现的情况下持续攻击训练过程。FL 模型仍然容易受到标准 ML 训练程序的攻击。这种巨大的攻击面使得平衡实用性、实用性、稳健性和隐私性之间的关系变得极具挑战性。虽然有人提出了使用流行的隐私保护原语（如全同态加密）来抵御攻击的方法，但这些方法在平衡所有隐私保护系统中都存在的一个重要问题上往往会遇到困难：为了确保隐私和稳健性，我愿意放弃多少实用性和实用性？在这项工作中，我们将讨论实现安全、稳健 FL 的实用方法，以及这一新兴研究领域所面临的挑战。
{"title":"You Can Have Your Cake and Eat It Too: Ensuring Practical Robustness and Privacy in Federated Learning","authors":"Nojan Sheybani, F. Koushanfar","doi":"10.1609/aaaiss.v3i1.31225","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31225","url":null,"abstract":"Inherently, federated learning (FL) robustness is very challenging to guarantee, especially when trying to maintain privacy. Compared to standard ML settings, FL's open training process allows for malicious clients to easily go under the radar. Alongside this, malicious clients can easily collude to attack the training process continuously, and without detection. FL models are also still susceptible to attacks on standard ML training procedures. This massive attack surface makes balancing the tradeoff between utility, practicality, robustness, and privacy extremely challenging. While there have been proposed defenses to attacks using popular privacy-preserving primitives, such as fully homomorphic encryption, they often face trouble balancing an all-important question that is present in all privacy-preserving systems: How much utility and practicality am I willing to give up to ensure privacy and robustness? In this work, we discuss a practical approach towards secure and robust FL and the challenges that face this field of emerging research.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141121426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31213
Nathalie Baracaldo
Federated learning (FL) has arisen as an alternative to collecting large amounts of data in a central place to train a machine learning (ML) model. FL is privacy-friendly, allowing multiple parties to collaboratively train an ML model without exchanging or transmitting their training data. For this purpose, an aggregator iteratively coordinates the training process among parties, and parties simply share with the aggregator model updates, which contain information pertinent to the model such as neural network weights. Besides privacy, generalization has been another key driver for FL: parties who do not have enough data to train a good performing model by themselves can now engage in FL to obtain an ML model suitable for their tasks. Products and real applications in the industry and consumer space have demonstrated the power of this learning paradigm. Recently, foundation models have taken the AI community by storm, promising to solve the shortage of labeled data. A foundation model is a powerful model that can be recycled for a variety of use cases by applying techniques such as zero-shot learning and full or parameter-efficient fine tuning. The premise is that the amount of data required to fine tune a foundation model for a new task is much smaller than fully training a traditional model from scratch. The reason why this is the case is that a good foundation model has already learned relevant general representations, and thus, adapting it to a new task only requires a minimal number of additional samples. This raises the question: Is FL still alive in the era of foundation models? In this talk, I will address this question. I will present some use cases where FL is very much alive. In these use cases, finding a foundation model with a desired representation is difficult if not impossible. With this pragmatic point of view, I hope to shed some light into a real use case where disparate private data is available in isolation at different parties and where labels may be located at a single party that doesn’t have any other information, making it impossible for a single party to train a model on its own. Furthermore, in some vertically-partitioned scenarios, cleaning data is not an option due to privacy-related reasons and it is not clear how to apply foundation models. Finally, I will also go over a few other requirements that are often overlooked, such as unlearning of data and its implications for the lifecycle management of FL and systems based on foundation models.
联邦学习（FL）是在一个中心位置收集大量数据以训练机器学习（ML）模型的一种替代方法。联合学习对隐私友好，允许多方协作训练 ML 模型，而无需交换或传输训练数据。为此，聚合器会反复协调各方的训练过程，各方只需与聚合器共享模型更新，其中包含神经网络权重等与模型相关的信息。除了隐私之外，通用化也是 FL 的另一个关键驱动因素：如果各方没有足够的数据来自行训练一个性能良好的模型，现在可以通过 FL 来获得适合其任务的 ML 模型。最近，基础模型在人工智能界掀起了一场风暴，有望解决标注数据短缺的问题。基础模型是一种功能强大的模型，通过应用零点学习和全面或参数高效微调等技术，可在各种用例中循环使用。前提是，针对新任务微调基础模型所需的数据量要比从头开始训练一个传统模型小得多。之所以会出现这种情况，是因为一个好的基础模型已经学会了相关的一般表征，因此，调整它以适应新任务只需要极少量的额外样本。这就提出了一个问题：在本讲座中，我将探讨这个问题。我将介绍一些 FL 非常活跃的用例。在这些用例中，找到一个具有所需表征的基础模型即使不是不可能，也是很困难的。在这种情况下，标签可能位于没有任何其他信息的单方，这使得单方无法独立训练模型。此外，在一些垂直分区的场景中，由于隐私相关的原因，清洗数据并不是一种选择，而且也不清楚如何应用基础模型。最后，我还将介绍一些经常被忽视的其他要求，如数据的非学习性及其对基于基础模型的 FL 和系统的生命周期管理的影响。
{"title":"Is Federated Learning Still Alive in the Foundation Model Era?","authors":"Nathalie Baracaldo","doi":"10.1609/aaaiss.v3i1.31213","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31213","url":null,"abstract":"Federated learning (FL) has arisen as an alternative to collecting large amounts of data in a central place to train a machine learning (ML) model. FL is privacy-friendly, allowing multiple parties to collaboratively train an ML model without exchanging or transmitting their training data. For this purpose, an aggregator iteratively coordinates the training process among parties, and parties simply share with the aggregator model updates, which contain information pertinent to the model such as neural network weights. Besides privacy, generalization has been another key driver for FL: parties who do not have enough data to train a good performing model by themselves can now engage in FL to obtain an ML model suitable for their tasks. Products and real applications in the industry and consumer space have demonstrated the power of this learning paradigm.\u0000\u0000\u0000Recently, foundation models have taken the AI community by storm, promising to solve the shortage of labeled data. A foundation model is a powerful model that can be recycled for a variety of use cases by applying techniques such as zero-shot learning and full or parameter-efficient fine tuning. The premise is that the amount of data required to fine tune a foundation model for a new task is much smaller than fully training a traditional model from scratch. The reason why this is the case is that a good foundation model has already learned relevant general representations, and thus, adapting it to a new task only requires a minimal number of additional samples. This raises the question: Is FL still alive in the era of foundation models?\u0000\u0000\u0000In this talk, I will address this question. I will present some use cases where FL is very much alive. In these use cases, finding a foundation model with a desired representation is difficult if not impossible. With this pragmatic point of view, I hope to shed some light into a real use case where disparate private data is available in isolation at different parties and where labels may be located at a single party that doesn’t have any other information, making it impossible for a single party to train a model on its own. Furthermore, in some vertically-partitioned scenarios, cleaning data is not an option due to privacy-related reasons and it is not clear how to apply foundation models. Finally, I will also go over a few other requirements that are often overlooked, such as unlearning of data and its implications for the lifecycle management of FL and systems based on foundation models.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141119158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31244
Helen Qin
This paper reports how generative AI can help children with specific language impairment (SLI) issues by developing an AI-assisted tool to support children with challenges in phonological development in English, especially children with English as the secondary language in the United States. Children from bilingual families often experience challenges in developing proficiency in English pronunciation and communication, which has been exacerbated by remote learning during the pandemic and led to learning loss. School-aged children with speech problems require timely intervention because children with language disorders find it difficult to communicate with others, leading to social isolation and academic difficulties. The needed intervention is often delayed due to the high cost of speech services and the shortage of Speech and Language Pathologists (SLPs). Individuals with a history of SLI have an increased risk of unemployment. An AI-assisted Phonological Development (AI-PD) tool was prototyped, aiming to alleviate these challenges by assisting caregivers in evaluating children's phonological development, assisting SLPs in lesson preparation, and mitigating the severe shortage of SLPs.
{"title":"Generative AI Applications in Helping Children with Speech Language Issues","authors":"Helen Qin","doi":"10.1609/aaaiss.v3i1.31244","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31244","url":null,"abstract":"This paper reports how generative AI can help children with specific language impairment (SLI) issues by developing an AI-assisted tool to support children with challenges in phonological development in English, especially children with English as the secondary language in the United States. Children from bilingual families often experience challenges in developing proficiency in English pronunciation and communication, which has been exacerbated by remote learning during the pandemic and led to learning loss. School-aged children with speech problems require timely intervention because children with language disorders find it difficult to communicate with others, leading to social isolation and academic difficulties. The needed intervention is often delayed due to the high cost of speech services and the shortage of Speech and Language Pathologists (SLPs). Individuals with a history of SLI have an increased risk of unemployment. An AI-assisted Phonological Development (AI-PD) tool was prototyped, aiming to alleviate these challenges by assisting caregivers in evaluating children's phonological development, assisting SLPs in lesson preparation, and mitigating the severe shortage of SLPs.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141119268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31264
Swati Mehrotra, Neelu Sinha
Artificial Intelligence (AI) has become pervasive in modern lives, with AI generative tools driving further transformation. However, a notable issue persists: the underrepresentation of females and individuals from ethnic and racial minorities in the tech industry. Despite generally positive attitudes toward technology among young students, this enthusiasm often does not extend to aspirations for careers in the field. To address this disparity, many schools in the United States are now offering computer science and AI courses at the high school level. Nevertheless, students from underrepresented groups often feel disconnected from these subjects, leading to low enrollment rates. Research underscores that students' career aspirations are solidified between the ages of 10-14 yrs, highlighting the importance of engaging them with computer science and computing skills during this formative period. Leveraging the Bourdieusian concept of social capital, this paper proposes educational interventions tailored for elementary schools. By nurturing students' technical social capital, these interventions aim to foster an inclusive ecosystem from an early age, when aspirations are taking shape. Ultimately, the goal is to enhance the accessibility of computer science education and related skills, empowering young students from underrepresented groups to pursue higher studies and careers in computer science and AI fields.
{"title":"A Human-Centric Approach towards Equity and Inclusion in AI Education","authors":"Swati Mehrotra, Neelu Sinha","doi":"10.1609/aaaiss.v3i1.31264","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31264","url":null,"abstract":"Artificial Intelligence (AI) has become pervasive in modern lives, with AI generative tools driving further transformation. However, a notable issue persists: the underrepresentation of females and individuals from ethnic and racial minorities in the tech industry. Despite generally positive attitudes toward technology among young students, this enthusiasm often does not extend to aspirations for careers in the field. To address this disparity, many schools in the United States are now offering computer science and AI courses at the high school level. Nevertheless, students from underrepresented groups often feel disconnected from these subjects, leading to low enrollment rates. Research underscores that students' career aspirations are solidified between the ages of 10-14 yrs, highlighting the importance of engaging them with computer science and computing skills during this formative period. Leveraging the Bourdieusian concept of social capital, this paper proposes educational interventions tailored for elementary schools. By nurturing students' technical social capital, these interventions aim to foster an inclusive ecosystem from an early age, when aspirations are taking shape. Ultimately, the goal is to enhance the accessibility of computer science education and related skills, empowering young students from underrepresented groups to pursue higher studies and careers in computer science and AI fields.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141123251","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31193
Milan Kostic, Hans Friedrich Witschel, Knut Hinkelmann, Maja Spahic-Bogdanovic
This study delves into the application of large language models (LLMs), such as ChatGPT-4, for the automated evaluation of student essays, with a focus on a case study conducted at the Swiss Institute of Business Administration. It explores the effectiveness of LLMs in assessing German-language student transfer assignments, and contrasts their performance with traditional evaluations by human lecturers. The primary findings highlight the challenges faced by LLMs in terms of accurately grading complex texts according to predefined categories and providing detailed feedback. This research illuminates the gap between the capabilities of LLMs and the nuanced requirements of student essay evaluation. The conclusion emphasizes the necessity for ongoing research and development in the area of LLM technology to improve the accuracy, reliability, and consistency of automated essay assessments in educational contexts.
{"title":"LLMs in Automated Essay Evaluation: A Case Study","authors":"Milan Kostic, Hans Friedrich Witschel, Knut Hinkelmann, Maja Spahic-Bogdanovic","doi":"10.1609/aaaiss.v3i1.31193","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31193","url":null,"abstract":"This study delves into the application of large language models (LLMs), such as ChatGPT-4, for the automated evaluation of student essays, with a focus on a case study conducted at the Swiss Institute of Business Administration. It explores the effectiveness of LLMs in assessing German-language student transfer assignments, and contrasts their performance with traditional evaluations by human lecturers. The primary findings highlight the challenges faced by LLMs in terms of accurately grading complex texts according to predefined categories and providing detailed feedback. This research illuminates the gap between the capabilities of LLMs and the nuanced requirements of student essay evaluation. The conclusion emphasizes the necessity for ongoing research and development in the area of LLM technology to improve the accuracy, reliability, and consistency of automated essay assessments in educational contexts.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141120466","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31256
Sharon Chee Yin Ho, Arisa Ema, Tanja Tajmel
The growing interest in Japan to implement text-to-image (T2I) generative artificial intelligence (GenAI) technologies in creative workflows has raised concern over what ethical and social implications these technologies will have on creative professionals. Our pilot study is the first to discuss what social and ethical oversights may emerge regarding such issues from prospective Japanese researchers – computer science (CS) graduate students studying in Japan. Given that these students are the primary demographic hired to work at research and development (R&D) labs at the forefront of such innovations in Japan, any social and ethical oversight on such issues may unequip them as future knowledge experts who will play a pivotal role in helping shape Japan’s policies regarding image generating AI technologies.
{"title":"The Impacts of Text-to-Image Generative AI on Creative Professionals According to Prospective Generative AI Researchers: Insights from Japan","authors":"Sharon Chee Yin Ho, Arisa Ema, Tanja Tajmel","doi":"10.1609/aaaiss.v3i1.31256","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31256","url":null,"abstract":"The growing interest in Japan to implement text-to-image (T2I) generative artificial intelligence (GenAI) technologies in creative workflows has raised concern over what ethical and social implications these technologies will have on creative professionals. Our pilot study is the first to discuss what social and ethical oversights may emerge regarding such issues from prospective Japanese researchers – computer science (CS) graduate students studying in Japan. Given that these students are the primary demographic hired to work at research and development (R&D) labs at the forefront of such innovations in Japan, any social and ethical oversight on such issues may unequip them as future knowledge experts who will play a pivotal role in helping shape Japan’s policies regarding image generating AI technologies.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141120500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31253
Kevin Vo
This study proposal combines the transformative potential of GPT-4 with an innovative approach to learning social and emotional skills, offering a novel conversational aid designed to enhance adolescents' social competence, and ultimately combat social disconnection in the digital era.
{"title":"Social Smarts with Tech Sparks: Harnessing LLMs for Youth Socioemotional Growth","authors":"Kevin Vo","doi":"10.1609/aaaiss.v3i1.31253","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31253","url":null,"abstract":"This study proposal combines the transformative potential of GPT-4 with an innovative approach to learning social and emotional skills, offering a novel conversational aid designed to enhance adolescents' social competence, and ultimately combat social disconnection in the digital era.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141122829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-20DOI: 10.1609/aaaiss.v3i1.31282
Edward M. Pogossian
Our model of cognizing roots in developmental psychology by Jean Piaget, follows researchers in modeling cognizing by solvers of combinatorial games, enriches object–oriented representatives of realities by input classifiers and relationships in English, while tends to be consistent with questioning the origination of cognizing in nature. Let us introduce the basics of the model, provide arguments for its adequacy, followed by those supporting the origination of cognizing.
{"title":"A Model of Cognizing Supporting the Origination of Cognizing in Nature","authors":"Edward M. Pogossian","doi":"10.1609/aaaiss.v3i1.31282","DOIUrl":"https://doi.org/10.1609/aaaiss.v3i1.31282","url":null,"abstract":"Our model of cognizing roots in developmental psychology by Jean Piaget, follows researchers in modeling cognizing by solvers of combinatorial games, enriches object–oriented representatives of realities by input classifiers and relationships in English, while tends to be consistent with questioning the origination of cognizing in nature. \u0000Let us introduce the basics of the model, provide arguments for its adequacy, followed by those supporting the origination of cognizing.","PeriodicalId":516827,"journal":{"name":"Proceedings of the AAAI Symposium Series","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141118979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：