Jianhui Lv , Byung-Gyu Kim , Adam Slowik , B.D. Parameshachari , Saru Kumari , Chien-Ming Chen , Keqin Li
{"title":"ERLNEIL-MDP: Evolutionary reinforcement learning with novelty-driven exploration for medical data processing","authors":"Jianhui Lv , Byung-Gyu Kim , Adam Slowik , B.D. Parameshachari , Saru Kumari , Chien-Ming Chen , Keqin Li","doi":"10.1016/j.swevo.2024.101769","DOIUrl":null,"url":null,"abstract":"<div><div>The rapid growth of medical data presents opportunities and challenges for healthcare professionals and researchers. To effectively process and analyze this complex and heterogeneous data, we propose evolutionary reinforcement learning with novelty-driven exploration and imitation learning for medical data processing (ERLNEIL-MDP) algorithm, including a novelty computation mechanism, an adaptive novelty-fitness selection strategy, an imitation-guided experience fusion mechanism, and an adaptive stability preservation module. The novelty computation mechanism quantifies the novelty of each policy based on its dissimilarity to the population and historical data, promoting exploration and diversity. The adaptive novelty-fitness selection strategy balances exploration and exploitation by considering policies' novelty and fitness during selection. The imitation-guided experience fusion mechanism incorporates expert knowledge and demonstrations into the learning process, accelerating the discovery of effective solutions. The adaptive stability preservation module ensures the stability and reliability of the learning process by dynamically adjusting the algorithm's hyperparameters and preserving elite policies across generations. These components work together to enhance the exploration, diversity, and stability of the learning process. The significance of this work lies in its potential to revolutionize medical data analysis, leading to more accurate diagnoses and personalized treatments. Experiments on MIMIC-III and n2c2 datasets demonstrate ERLNEIL-MDP's superior performance, achieving F1 scores of 0.933 and 0.928, respectively, representing 6.0 % and 6.7 % improvements over state-of-the-art methods. The algorithm exhibits strong convergence, high population diversity, and robustness to noise and missing data.</div></div>","PeriodicalId":48682,"journal":{"name":"Swarm and Evolutionary Computation","volume":null,"pages":null},"PeriodicalIF":8.2000,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Swarm and Evolutionary Computation","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2210650224003079","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
The rapid growth of medical data presents opportunities and challenges for healthcare professionals and researchers. To effectively process and analyze this complex and heterogeneous data, we propose evolutionary reinforcement learning with novelty-driven exploration and imitation learning for medical data processing (ERLNEIL-MDP) algorithm, including a novelty computation mechanism, an adaptive novelty-fitness selection strategy, an imitation-guided experience fusion mechanism, and an adaptive stability preservation module. The novelty computation mechanism quantifies the novelty of each policy based on its dissimilarity to the population and historical data, promoting exploration and diversity. The adaptive novelty-fitness selection strategy balances exploration and exploitation by considering policies' novelty and fitness during selection. The imitation-guided experience fusion mechanism incorporates expert knowledge and demonstrations into the learning process, accelerating the discovery of effective solutions. The adaptive stability preservation module ensures the stability and reliability of the learning process by dynamically adjusting the algorithm's hyperparameters and preserving elite policies across generations. These components work together to enhance the exploration, diversity, and stability of the learning process. The significance of this work lies in its potential to revolutionize medical data analysis, leading to more accurate diagnoses and personalized treatments. Experiments on MIMIC-III and n2c2 datasets demonstrate ERLNEIL-MDP's superior performance, achieving F1 scores of 0.933 and 0.928, respectively, representing 6.0 % and 6.7 % improvements over state-of-the-art methods. The algorithm exhibits strong convergence, high population diversity, and robustness to noise and missing data.
期刊介绍:
Swarm and Evolutionary Computation is a pioneering peer-reviewed journal focused on the latest research and advancements in nature-inspired intelligent computation using swarm and evolutionary algorithms. It covers theoretical, experimental, and practical aspects of these paradigms and their hybrids, promoting interdisciplinary research. The journal prioritizes the publication of high-quality, original articles that push the boundaries of evolutionary computation and swarm intelligence. Additionally, it welcomes survey papers on current topics and novel applications. Topics of interest include but are not limited to: Genetic Algorithms, and Genetic Programming, Evolution Strategies, and Evolutionary Programming, Differential Evolution, Artificial Immune Systems, Particle Swarms, Ant Colony, Bacterial Foraging, Artificial Bees, Fireflies Algorithm, Harmony Search, Artificial Life, Digital Organisms, Estimation of Distribution Algorithms, Stochastic Diffusion Search, Quantum Computing, Nano Computing, Membrane Computing, Human-centric Computing, Hybridization of Algorithms, Memetic Computing, Autonomic Computing, Self-organizing systems, Combinatorial, Discrete, Binary, Constrained, Multi-objective, Multi-modal, Dynamic, and Large-scale Optimization.