Pub Date : 2024-11-19DOI: 10.1016/j.mlwa.2024.100606
Ari Vesalainen , Mikko Tolonen , Laura Ruotsalainen
Scholarly editions play a crucial role in humanities research, particularly in the study of literature and historical documents. The primary objective of these editions is to reconstruct the original text or provide insights into the author’s intentions. Traditionally, crafting a critical edition required a lifetime of dedication. However, thanks to recent advancements in deep learning and computer vision, modern text recognition tools can now be used to expedite this process. A key part of these tools is document layout analysis (DLA), where image segmentation methods are used to detect different text elements. Most existing DLA solutions have focused on evaluating the accuracy of these methods, often neglecting to study the practical consequences of method selection. In this study, we have developed a new metric, the Document Layout Error Rate (DLER), which evaluates the performance of fine-grained DLA methods within the overall pipeline. This metric helps identify the method with the lowest error rate, thereby minimizing the manual effort required for corrections. We applied this evaluation method to assess four different methods and their efficacy for the DLA task in the context of David Hume’s History of England.
学术版本在人文学科研究，尤其是文学和历史文献研究中发挥着至关重要的作用。这些版本的主要目的是重构原文或深入了解作者的意图。传统上，制作批判性版本需要一生的奉献。然而，得益于深度学习和计算机视觉领域的最新进展，现在可以使用现代文本识别工具来加快这一过程。这些工具的一个关键部分是文档排版分析（DLA），其中使用图像分割方法来检测不同的文本元素。大多数现有的 DLA 解决方案都侧重于评估这些方法的准确性，往往忽视了对方法选择的实际影响的研究。在本研究中，我们开发了一种新指标--文档布局错误率（DLER），用于评估整个管道中细粒度 DLA 方法的性能。该指标有助于确定错误率最低的方法，从而最大限度地减少人工修正所需的工作量。我们在大卫-休谟（David Hume）的《英国史》（History of England）中应用了这种评估方法来评估四种不同的方法及其在 DLA 任务中的功效。
{"title":"Document Layout Error Rate (DLER) metric to evaluate image segmentation methods","authors":"Ari Vesalainen , Mikko Tolonen , Laura Ruotsalainen","doi":"10.1016/j.mlwa.2024.100606","DOIUrl":"10.1016/j.mlwa.2024.100606","url":null,"abstract":"<div><div>Scholarly editions play a crucial role in humanities research, particularly in the study of literature and historical documents. The primary objective of these editions is to reconstruct the original text or provide insights into the author’s intentions. Traditionally, crafting a critical edition required a lifetime of dedication. However, thanks to recent advancements in deep learning and computer vision, modern text recognition tools can now be used to expedite this process. A key part of these tools is document layout analysis (DLA), where image segmentation methods are used to detect different text elements. Most existing DLA solutions have focused on evaluating the accuracy of these methods, often neglecting to study the practical consequences of method selection. In this study, we have developed a new metric, the Document Layout Error Rate (DLER), which evaluates the performance of fine-grained DLA methods within the overall pipeline. This metric helps identify the method with the lowest error rate, thereby minimizing the manual effort required for corrections. We applied this evaluation method to assess four different methods and their efficacy for the DLA task in the context of David Hume’s <em>History of England</em>.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100606"},"PeriodicalIF":0.0,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142698987","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Machine learning (ML) is poised to drive innovations in clinical microbiomics, such as in disease diagnostics and prognostics. However, the successful implementation of ML in these domains necessitates the development of reproducible, interpretable models that meet the rigorous performance standards set by regulatory agencies. This study aims to identify key areas in need of improvement in current ML practices within microbiomics, with a focus on bridging the gap between existing methodologies and the requirements for clinical application. To do so, we analyze 100 peer-reviewed articles from 2021 to 2022. Within this corpus, datasets have a median size of 161.5 samples, with over one-third containing fewer than 100 samples, signaling a high potential for overfitting. Limited demographic data further raises concerns about generalizability and fairness, with 24% of studies omitting participants' country of residence, and attributes like race/ethnicity, education, and income rarely reported (11%, 2%, and 0%, respectively). Methodological issues are also common; for instance, for 86% of studies we could not confidently rule out test set omission and data leakage, suggesting a strong potential for inflated performance estimates across the literature. Reproducibility is a concern, with 78% of studies abstaining from sharing their ML code publicly. Based on this analysis, we provide guidance to avoid common pitfalls that can hinder model performance, generalizability, and trustworthiness. An interactive tutorial on applying ML to microbiomics data accompanies the discussion, to help establish and reinforce best practices within the community.
机器学习（ML）有望推动临床微生物组学的创新，如疾病诊断和预后。然而，要在这些领域成功实施机器学习，就必须开发出可重复、可解释的模型，以满足监管机构制定的严格性能标准。本研究旨在确定微生物组学中当前 ML 实践中需要改进的关键领域，重点是缩小现有方法与临床应用要求之间的差距。为此，我们分析了 2021 年至 2022 年的 100 篇同行评审文章。在这一语料库中，数据集的中位数为 161.5 个样本，其中超过三分之一的数据集包含的样本少于 100 个，这表明过度拟合的可能性很大。有限的人口统计学数据进一步引发了对普遍性和公平性的担忧，24% 的研究遗漏了参与者的居住国，种族/民族、教育程度和收入等属性也很少被报告（分别为 11%、2% 和 0%）。方法论问题也很常见；例如，在 86% 的研究中，我们无法有把握地排除测试集遗漏和数据泄漏的可能性，这表明在所有文献中都很有可能出现夸大绩效估计值的情况。可重复性也是一个令人担忧的问题，有 78% 的研究放弃公开共享其 ML 代码。基于这一分析，我们提供了避免常见陷阱的指南，这些陷阱可能会妨碍模型的性能、可推广性和可信度。在讨论的同时，我们还提供了将 ML 应用于微生物组学数据的互动教程，以帮助在社区内建立和加强最佳实践。
{"title":"Supervised machine learning for microbiomics: Bridging the gap between current and best practices","authors":"Natasha Katherine Dudek , Mariami Chakhvadze , Saba Kobakhidze , Omar Kantidze , Yuriy Gankin","doi":"10.1016/j.mlwa.2024.100607","DOIUrl":"10.1016/j.mlwa.2024.100607","url":null,"abstract":"<div><div>Machine learning (ML) is poised to drive innovations in clinical microbiomics, such as in disease diagnostics and prognostics. However, the successful implementation of ML in these domains necessitates the development of reproducible, interpretable models that meet the rigorous performance standards set by regulatory agencies. This study aims to identify key areas in need of improvement in current ML practices within microbiomics, with a focus on bridging the gap between existing methodologies and the requirements for clinical application. To do so, we analyze 100 peer-reviewed articles from 2021 to 2022. Within this corpus, datasets have a median size of 161.5 samples, with over one-third containing fewer than 100 samples, signaling a high potential for overfitting. Limited demographic data further raises concerns about generalizability and fairness, with 24% of studies omitting participants' country of residence, and attributes like race/ethnicity, education, and income rarely reported (11%, 2%, and 0%, respectively). Methodological issues are also common; for instance, for 86% of studies we could not confidently rule out test set omission and data leakage, suggesting a strong potential for inflated performance estimates across the literature. Reproducibility is a concern, with 78% of studies abstaining from sharing their ML code publicly. Based on this analysis, we provide guidance to avoid common pitfalls that can hinder model performance, generalizability, and trustworthiness. An interactive tutorial on applying ML to microbiomics data accompanies the discussion, to help establish and reinforce best practices within the community.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100607"},"PeriodicalIF":0.0,"publicationDate":"2024-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142698988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-12DOI: 10.1016/j.mlwa.2024.100602
Pedro Reviriego , Javier Conde , Elena Merino-Gómez , Gonzalo Martínez , José Alberto Hernández
The introduction of Artificial Intelligence (AI) generative language models such as GPT (Generative Pre-trained Transformer) and conversational tools such as ChatGPT has triggered a revolution that can transform how text is generated. This has many implications, for example, as AI-generated text becomes a significant fraction of the text, would this affect the language capabilities of readers and also the training of newer AI tools? Would it affect the evolution of languages? Focusing on one specific aspect of the language: words; will the use of tools such as ChatGPT increase or reduce the vocabulary used or the lexical diversity? This has implications for words, as those not included in AI-generated content will tend to be less and less popular and may eventually be lost. In this work, we perform an initial comparison of the vocabulary and lexical diversity of ChatGPT and humans when performing the same tasks. In more detail, two datasets containing the answers to different types of questions answered by ChatGPT and humans, and a third dataset in which ChatGPT paraphrases sentences and questions are used. The analysis shows that ChatGPT-3.5 tends to use fewer distinct words and lower diversity than humans while ChatGPT-4 has a similar lexical diversity as humans and in some cases even larger. These results are very preliminary and additional datasets and ChatGPT configurations have to be evaluated to extract more general conclusions. Therefore, further research is needed to understand how the use of ChatGPT and more broadly generative AI tools will affect the vocabulary and lexical diversity in different types of text and languages.
{"title":"Playing with words: Comparing the vocabulary and lexical diversity of ChatGPT and humans","authors":"Pedro Reviriego , Javier Conde , Elena Merino-Gómez , Gonzalo Martínez , José Alberto Hernández","doi":"10.1016/j.mlwa.2024.100602","DOIUrl":"10.1016/j.mlwa.2024.100602","url":null,"abstract":"<div><div>The introduction of Artificial Intelligence (AI) generative language models such as GPT (Generative Pre-trained Transformer) and conversational tools such as ChatGPT has triggered a revolution that can transform how text is generated. This has many implications, for example, as AI-generated text becomes a significant fraction of the text, would this affect the language capabilities of readers and also the training of newer AI tools? Would it affect the evolution of languages? Focusing on one specific aspect of the language: words; will the use of tools such as ChatGPT increase or reduce the vocabulary used or the lexical diversity? This has implications for words, as those not included in AI-generated content will tend to be less and less popular and may eventually be lost. In this work, we perform an initial comparison of the vocabulary and lexical diversity of ChatGPT and humans when performing the same tasks. In more detail, two datasets containing the answers to different types of questions answered by ChatGPT and humans, and a third dataset in which ChatGPT paraphrases sentences and questions are used. The analysis shows that ChatGPT-3.5 tends to use fewer distinct words and lower diversity than humans while ChatGPT-4 has a similar lexical diversity as humans and in some cases even larger. These results are very preliminary and additional datasets and ChatGPT configurations have to be evaluated to extract more general conclusions. Therefore, further research is needed to understand how the use of ChatGPT and more broadly generative AI tools will affect the vocabulary and lexical diversity in different types of text and languages.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100602"},"PeriodicalIF":0.0,"publicationDate":"2024-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142698989","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-10DOI: 10.1016/j.mlwa.2024.100605
Amir Moslemi , Anna Briskina , Zubeka Dang , Jason Li
Deep learning has achieved notable success across academia, medicine, and industry. Its ability to identify complex patterns in large-scale data and to manage millions of parameters has made it highly advantageous. However, deploying deep learning models presents a significant challenge due to their high computational demands. Knowledge distillation (KD) has emerged as a key technique for model compression and efficient knowledge transfer, enabling the deployment of deep learning models on resource-limited devices without compromising performance. This survey examines recent advancements in KD, highlighting key innovations in architectures, training paradigms, and application domains. We categorize contemporary KD methods into traditional approaches, such as response-based, feature-based, and relation-based knowledge distillation, and novel advanced paradigms, including self-distillation, cross-modal distillation, and adversarial distillation strategies. Additionally, we discuss emerging challenges, particularly in the context of distillation under limited data scenarios, privacy-preserving KD, and the interplay with other model compression techniques like quantization. Our survey also explores applications across computer vision, natural language processing, and multimodal tasks, where KD has driven performance improvements and enhanced model compression. This review aims to provide researchers and practitioners with a comprehensive understanding of the state-of-the-art in knowledge distillation, bridging foundational concepts with the latest methodologies and practical implications.
{"title":"A survey on knowledge distillation: Recent advancements","authors":"Amir Moslemi , Anna Briskina , Zubeka Dang , Jason Li","doi":"10.1016/j.mlwa.2024.100605","DOIUrl":"10.1016/j.mlwa.2024.100605","url":null,"abstract":"<div><div>Deep learning has achieved notable success across academia, medicine, and industry. Its ability to identify complex patterns in large-scale data and to manage millions of parameters has made it highly advantageous. However, deploying deep learning models presents a significant challenge due to their high computational demands. Knowledge distillation (KD) has emerged as a key technique for model compression and efficient knowledge transfer, enabling the deployment of deep learning models on resource-limited devices without compromising performance. This survey examines recent advancements in KD, highlighting key innovations in architectures, training paradigms, and application domains. We categorize contemporary KD methods into traditional approaches, such as response-based, feature-based, and relation-based knowledge distillation, and novel advanced paradigms, including self-distillation, cross-modal distillation, and adversarial distillation strategies. Additionally, we discuss emerging challenges, particularly in the context of distillation under limited data scenarios, privacy-preserving KD, and the interplay with other model compression techniques like quantization. Our survey also explores applications across computer vision, natural language processing, and multimodal tasks, where KD has driven performance improvements and enhanced model compression. This review aims to provide researchers and practitioners with a comprehensive understanding of the state-of-the-art in knowledge distillation, bridging foundational concepts with the latest methodologies and practical implications.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100605"},"PeriodicalIF":0.0,"publicationDate":"2024-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142698990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-08DOI: 10.1016/j.mlwa.2024.100604
Tian Su , Senarath Dharmasena , David Leatham , Charles Gilliland
Texas rural land markets have several special features that makes it unique from other rural land markets in the United States. In 2021, Texas agricultural land, including buildings, is valued at $299.88 billion, which is almost 10% of the nation's total agricultural real estate value and 83% of the state's land is categorized as rural. In addition, due to its size and geologic features, Texas’ diverse landscape creates complex and widely divergent conditions affecting ownership and marketing of the land. Despite this complexity, lack of granular level and reliable transactional data on land sales has prevented thorough investigation into Texas land markets to uncover various interdependencies. Using quarterly transactional land value data from 1966 to 2017, this study uses cutting-edge machine learning algorithms and probabilistic graphical models to uncover causal interaction patterns of different land markets in Texas. The results reveal that Texas rural land markets are interdependent. Current and potential landholders and lenders can use the results from this work to aid strategic decision making. Financial institutions and investment groups could be made aware of the trend of one land market relative to other markets and adjust their holdings accordingly. Landowners may better understand changes in net wealth, which affect their ability to borrow capital and operate efficiently. Moreover, lenders may also benefit from the information to manage collateral and thus maintain the stability of their operation.
{"title":"Texas rural land market integration: A causal analysis using machine learning applications","authors":"Tian Su , Senarath Dharmasena , David Leatham , Charles Gilliland","doi":"10.1016/j.mlwa.2024.100604","DOIUrl":"10.1016/j.mlwa.2024.100604","url":null,"abstract":"<div><div>Texas rural land markets have several special features that makes it unique from other rural land markets in the United States. In 2021, Texas agricultural land, including buildings, is valued at $299.88 billion, which is almost 10% of the nation's total agricultural real estate value and 83% of the state's land is categorized as rural. In addition, due to its size and geologic features, Texas’ diverse landscape creates complex and widely divergent conditions affecting ownership and marketing of the land. Despite this complexity, lack of granular level and reliable transactional data on land sales has prevented thorough investigation into Texas land markets to uncover various interdependencies. Using quarterly transactional land value data from 1966 to 2017, this study uses cutting-edge machine learning algorithms and probabilistic graphical models to uncover causal interaction patterns of different land markets in Texas. The results reveal that Texas rural land markets are interdependent. Current and potential landholders and lenders can use the results from this work to aid strategic decision making. Financial institutions and investment groups could be made aware of the trend of one land market relative to other markets and adjust their holdings accordingly. Landowners may better understand changes in net wealth, which affect their ability to borrow capital and operate efficiently. Moreover, lenders may also benefit from the information to manage collateral and thus maintain the stability of their operation.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100604"},"PeriodicalIF":0.0,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142650979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-07DOI: 10.1016/j.mlwa.2024.100601
Liuyi Yang, Patrick Finnerty, Chikara Ohta
Transfer learning can address the issue of insufficient labels in machine learning. Using knowledge in a labeled domain (source domain) can assist in acquiring and learning knowledge in a domain (target domain) that lacks some or all labels. In this paper, we propose a new cluster-based semi-supervised transfer learning (CBSSTL) under a new assumption that samples in the target domain are unlabeled but contain cluster information. Furthermore, we propose a new transfer learning framework and a method for fine-tuning parameters. We tested and compared the proposed method with other unsupervised and semi-supervised transfer learning methods on well-known image datasets. The experimental results demonstrate the effectiveness of the proposed method. Additionally, we created a localization dataset for transfer learning. Finally, we tested and analyzed the proposed method on this dataset. Its particularly challenging nature makes it difficult for our method to work effectively.
{"title":"Applications of cluster-based transfer learning in image and localization tasks","authors":"Liuyi Yang, Patrick Finnerty, Chikara Ohta","doi":"10.1016/j.mlwa.2024.100601","DOIUrl":"10.1016/j.mlwa.2024.100601","url":null,"abstract":"<div><div>Transfer learning can address the issue of insufficient labels in machine learning. Using knowledge in a labeled domain (source domain) can assist in acquiring and learning knowledge in a domain (target domain) that lacks some or all labels. In this paper, we propose a new cluster-based semi-supervised transfer learning (CBSSTL) under a new assumption that samples in the target domain are unlabeled but contain cluster information. Furthermore, we propose a new transfer learning framework and a method for fine-tuning parameters. We tested and compared the proposed method with other unsupervised and semi-supervised transfer learning methods on well-known image datasets. The experimental results demonstrate the effectiveness of the proposed method. Additionally, we created a localization dataset for transfer learning. Finally, we tested and analyzed the proposed method on this dataset. Its particularly challenging nature makes it difficult for our method to work effectively.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100601"},"PeriodicalIF":0.0,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142650984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-02DOI: 10.1016/j.mlwa.2024.100598
Jean Haurogné , Nihala Basheer , Shareeful Islam
Vulnerabilities in the source code are one of the main causes of potential threats in software-intensive systems. There are a large number of vulnerabilities published each day, and effective vulnerability detection is critical to identifying and mitigating these vulnerabilities. AI has emerged as a promising solution to enhance vulnerability detection, offering the ability to analyse vast amounts of data and identify patterns indicative of potential threats. However, AI-based methods often face several challenges, specifically when dealing with large datasets and understanding the specific context of the problem. Large Language Model (LLM) is now widely considered to tackle more complex tasks and handle large datasets, which also exhibits limitations in terms of explaining the model outcome and existing works focus on providing overview of explainability and transparency. This research introduces a novel transparency obligation practice for vulnerability detection using BERT based LLMs. We address the black-box nature of LLMs by employing XAI techniques, unique combination of SHAP, LIME, heat map. We propose an architecture that combines the BERT model with transparency obligation practices, which ensures the assurance of transparency throughout the entire LLM life cycle. An experiment is performed with a large source code dataset to demonstrate the applicability of the proposed approach. The result shows higher accuracy of 91.8 % for the vulnerability detection and model explainability outcome is highly influenced by “vulnerable”, “function”, "mysql_tmpdir_list", “strmov” tokens using both SHAP and LIME framework. Heatmap of attention weights, highlights the local token interactions that aid in understanding the model's decision points.
{"title":"Vulnerability detection using BERT based LLM model with transparency obligation practice towards trustworthy AI","authors":"Jean Haurogné , Nihala Basheer , Shareeful Islam","doi":"10.1016/j.mlwa.2024.100598","DOIUrl":"10.1016/j.mlwa.2024.100598","url":null,"abstract":"<div><div>Vulnerabilities in the source code are one of the main causes of potential threats in software-intensive systems. There are a large number of vulnerabilities published each day, and effective vulnerability detection is critical to identifying and mitigating these vulnerabilities. AI has emerged as a promising solution to enhance vulnerability detection, offering the ability to analyse vast amounts of data and identify patterns indicative of potential threats. However, AI-based methods often face several challenges, specifically when dealing with large datasets and understanding the specific context of the problem. Large Language Model (LLM) is now widely considered to tackle more complex tasks and handle large datasets, which also exhibits limitations in terms of explaining the model outcome and existing works focus on providing overview of explainability and transparency. This research introduces a novel transparency obligation practice for vulnerability detection using BERT based LLMs. We address the black-box nature of LLMs by employing XAI techniques, unique combination of SHAP, LIME, heat map. We propose an architecture that combines the BERT model with transparency obligation practices, which ensures the assurance of transparency throughout the entire LLM life cycle. An experiment is performed with a large source code dataset to demonstrate the applicability of the proposed approach. The result shows higher accuracy of 91.8 % for the vulnerability detection and model explainability outcome is highly influenced by “vulnerable”, “function”, \"mysql_tmpdir_list\", “strmov” tokens using both SHAP and LIME framework. Heatmap of attention weights, highlights the local token interactions that aid in understanding the model's decision points.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100598"},"PeriodicalIF":0.0,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142650982","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-31DOI: 10.1016/j.mlwa.2024.100599
Amandeep Singh, Yovela Murzello, Hyowon Lee, Shene Abdalla, Siby Samuel
The intersection of Artificial Intelligence (AI) and moral philosophy presents unique challenges in the development of autonomous vehicles, particularly in scenarios requiring split-second ethical decisions. This study examines the relationship between working memory (WM) and moral judgments in simulated AV scenarios, quantifying the effects of varying cognitive load on utilitarian decision-making under different time constraints. We experimented with 336 participants, each completing 16 simulated driving trials presenting unique ethical dilemmas. Results reveal a complex interplay between cognitive load and ethical choices. Under high temporal pressure (1-second response window), utilitarian decisions decreased significantly from 92.77 % to 70.08 %. Extended time constraints led to increased utilitarian choices. Statistical analyses validated these findings across diverse ethical contexts. Chi-square tests revealed significant associations between WM load and utilitarian decisions in 1-second conditions, particularly for high-stakes scenarios. Logistic regression showed that WM significantly decreased the likelihood of utilitarian decisions in these scenarios. Six supervised machine learning models were employed, with Gaussian Naive Bayes achieving the highest predictive accuracy (82.2 % to 97.0 %) in distinguishing utilitarian decisions. Partial Dependence analysis revealed a strong negative correlation between WM and utilitarian decisions, especially in the 1-second interval. The 2-second interval emerged as potentially optimal for balancing time constraints and cognitive load. These findings contribute to the theoretical understanding of ethical decision-making under cognitive load and provide practical insights for developing ethically aligned autonomous systems, with implications for improving safety, optimizing takeover protocols, and enhancing the ethical reasoning capabilities of autonomous driving systems.
{"title":"Moral decision making: Explainable insights into the role of working memory in autonomous driving","authors":"Amandeep Singh, Yovela Murzello, Hyowon Lee, Shene Abdalla, Siby Samuel","doi":"10.1016/j.mlwa.2024.100599","DOIUrl":"10.1016/j.mlwa.2024.100599","url":null,"abstract":"<div><div>The intersection of Artificial Intelligence (AI) and moral philosophy presents unique challenges in the development of autonomous vehicles, particularly in scenarios requiring split-second ethical decisions. This study examines the relationship between working memory (WM) and moral judgments in simulated AV scenarios, quantifying the effects of varying cognitive load on utilitarian decision-making under different time constraints. We experimented with 336 participants, each completing 16 simulated driving trials presenting unique ethical dilemmas. Results reveal a complex interplay between cognitive load and ethical choices. Under high temporal pressure (1-second response window), utilitarian decisions decreased significantly from 92.77 % to 70.08 %. Extended time constraints led to increased utilitarian choices. Statistical analyses validated these findings across diverse ethical contexts. Chi-square tests revealed significant associations between WM load and utilitarian decisions in 1-second conditions, particularly for high-stakes scenarios. Logistic regression showed that WM significantly decreased the likelihood of utilitarian decisions in these scenarios. Six supervised machine learning models were employed, with Gaussian Naive Bayes achieving the highest predictive accuracy (82.2 % to 97.0 %) in distinguishing utilitarian decisions. Partial Dependence analysis revealed a strong negative correlation between WM and utilitarian decisions, especially in the 1-second interval. The 2-second interval emerged as potentially optimal for balancing time constraints and cognitive load. These findings contribute to the theoretical understanding of ethical decision-making under cognitive load and provide practical insights for developing ethically aligned autonomous systems, with implications for improving safety, optimizing takeover protocols, and enhancing the ethical reasoning capabilities of autonomous driving systems.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100599"},"PeriodicalIF":0.0,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142650983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-30DOI: 10.1016/j.mlwa.2024.100596
Ivan Oyege , Harriet Sibitenda , Maruthi Sridhar Balaji Bhaskar
The application of artificial intelligence for identifying Fall armyworm (Spodoptera frugiperda), African armyworm (Spodoptera exempta), and Maize stem borer (Busseola fusca) is critical due to the threats they pose to global food production. This study aims to evaluate and identify the most accurate and robust DL models in detecting and classifying these three significant agricultural pests. Seven traditional DL models: Convolutional Neural Network, Visual Geometry Group (VGG16), Residual Networks (ResNet50), MobileNetV2, InceptionV3, Deep Neural Network (DNN), and InceptionResNetV2 and the advanced You Look Only Once (YOLOv8) model were trained and tested using pest image datasets. The results showed that all traditional models except DNN had high accuracies ranging from 93.17% (InceptionResNetV2) to 99.43% (MobileNet) in training and testing, with losses ranging from 1.71% (MobileNetV2) to 24.99% (InceptionResNetV2). DNN had a slightly lower accuracy range of 55.27% to 56.39% and a loss range of 85.02% to 89.96% in training and testing. YOLOv8 emerged as the best and most robust model in the pest detection and classification tasks, achieving Precision and Recall scores ranging from 98.4% to 100% on single-class and multi-class classifications, making it highly suitable for real-world pest management applications. This research pioneers the use of DL for the classification and detection of maize stem borer, African armyworm and Fall armyworm, unique and separately addressing a critical gap in agricultural pest management in corn. With early and accurate pest identification, crop protection measures can be implemented efficiently. The findings lead to reduced crop damage and enhanced food security.
{"title":"Deep learning applications for real-time and early detection of fall armyworm, African armyworm, and maize stem borer","authors":"Ivan Oyege , Harriet Sibitenda , Maruthi Sridhar Balaji Bhaskar","doi":"10.1016/j.mlwa.2024.100596","DOIUrl":"10.1016/j.mlwa.2024.100596","url":null,"abstract":"<div><div>The application of artificial intelligence for identifying Fall armyworm (<em>Spodoptera frugiperda</em>), African armyworm (<em>Spodoptera exempta</em>), and Maize stem borer (<em>Busseola fusca</em>) is critical due to the threats they pose to global food production. This study aims to evaluate and identify the most accurate and robust DL models in detecting and classifying these three significant agricultural pests. Seven traditional DL models: Convolutional Neural Network, Visual Geometry Group (VGG16), Residual Networks (ResNet50), MobileNetV2, InceptionV3, Deep Neural Network (DNN), and InceptionResNetV2 and the advanced You Look Only Once (YOLOv8) model were trained and tested using pest image datasets. The results showed that all traditional models except DNN had high accuracies ranging from 93.17% (InceptionResNetV2) to 99.43% (MobileNet) in training and testing, with losses ranging from 1.71% (MobileNetV2) to 24.99% (InceptionResNetV2). DNN had a slightly lower accuracy range of 55.27% to 56.39% and a loss range of 85.02% to 89.96% in training and testing. YOLOv8 emerged as the best and most robust model in the pest detection and classification tasks, achieving Precision and Recall scores ranging from 98.4% to 100% on single-class and multi-class classifications, making it highly suitable for real-world pest management applications. This research pioneers the use of DL for the classification and detection of maize stem borer, African armyworm and Fall armyworm, unique and separately addressing a critical gap in agricultural pest management in corn. With early and accurate pest identification, crop protection measures can be implemented efficiently. The findings lead to reduced crop damage and enhanced food security.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100596"},"PeriodicalIF":0.0,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142572647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-29DOI: 10.1016/j.mlwa.2024.100597
Surani Matharaarachchi , Mike Domaratzki , Saman Muthukumarana
Imbalanced datasets are frequent in machine learning, where certain classes are markedly underrepresented compared to others. This imbalance often results in sub-optimal model performance, as classifiers tend to favour the majority class. A significant challenge arises when abnormal instances, such as outliers, exist within the minority class, diminishing the effectiveness of traditional re-sampling methods like the Synthetic Minority Over-sampling Technique (SMOTE). This manuscript addresses this critical issue by introducing four SMOTE extensions: Distance ExtSMOTE, Dirichlet ExtSMOTE, FCRP SMOTE, and BGMM SMOTE. These methods leverage a weighted average of neighbouring instances to enhance the quality of synthetic samples and mitigate the impact of outliers. Comprehensive experiments conducted on diverse simulated and real-world imbalanced datasets demonstrate that the proposed methods improve classification performance compared to the original SMOTE and its most competitive variants. Notably, we demonstrate that Dirichlet ExtSMOTE outperforms most other proposed and existing SMOTE variants in terms of achieving better F1 score, MCC, and PR-AUC. Our results underscore the effectiveness of these advanced SMOTE extensions in tackling class imbalance, particularly in the presence of abnormal instances, offering robust solutions for real-world applications.
{"title":"Enhancing SMOTE for imbalanced data with abnormal minority instances","authors":"Surani Matharaarachchi , Mike Domaratzki , Saman Muthukumarana","doi":"10.1016/j.mlwa.2024.100597","DOIUrl":"10.1016/j.mlwa.2024.100597","url":null,"abstract":"<div><div>Imbalanced datasets are frequent in machine learning, where certain classes are markedly underrepresented compared to others. This imbalance often results in sub-optimal model performance, as classifiers tend to favour the majority class. A significant challenge arises when abnormal instances, such as outliers, exist within the minority class, diminishing the effectiveness of traditional re-sampling methods like the Synthetic Minority Over-sampling Technique (SMOTE). This manuscript addresses this critical issue by introducing four SMOTE extensions: Distance ExtSMOTE, Dirichlet ExtSMOTE, FCRP SMOTE, and BGMM SMOTE. These methods leverage a weighted average of neighbouring instances to enhance the quality of synthetic samples and mitigate the impact of outliers. Comprehensive experiments conducted on diverse simulated and real-world imbalanced datasets demonstrate that the proposed methods improve classification performance compared to the original SMOTE and its most competitive variants. Notably, we demonstrate that Dirichlet ExtSMOTE outperforms most other proposed and existing SMOTE variants in terms of achieving better F1 score, MCC, and PR-AUC. Our results underscore the effectiveness of these advanced SMOTE extensions in tackling class imbalance, particularly in the presence of abnormal instances, offering robust solutions for real-world applications.</div></div>","PeriodicalId":74093,"journal":{"name":"Machine learning with applications","volume":"18 ","pages":"Article 100597"},"PeriodicalIF":0.0,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578485","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号