An in-depth study to fine-tune the hyperparameters of pre-trained transfer learning models with state-of-the-art optimization methods: Osteoarthritis severity classification with optimized architectures
{"title":"An in-depth study to fine-tune the hyperparameters of pre-trained transfer learning models with state-of-the-art optimization methods: Osteoarthritis severity classification with optimized architectures","authors":"Aysun Öcal, Hasan Koyuncu","doi":"10.1016/j.swevo.2024.101640","DOIUrl":null,"url":null,"abstract":"<div><p>Discrete & continuous optimization constitutes a challenging task and generally rises as an NP-hard problem. In the literature, as a derivative of this type of optimization issue, hyperparameter optimization of transfer learning (TL) architectures is not efficiently analyzed as a detailed survey in the literature. In this paper, the optimized TL-based models are effectively examined to handle this issue which constitutes the main aim of our study. For evaluation, knee osteoarthritis (KOA – a chronic degenerative joint disorder) dataset is handled to perform two challenging classification tasks which reveal the second aim of our study, <em>i.e.</em> binary- and multi-categorizations on KOA X-ray images. To fine-tune the hyperparameters of TL models, state-of-the-art optimization methods are chosen and compared on this competitive – NP-hard problem. Sixteen optimized architectures are designed using four efficient optimization methods (ASPSO, CDW-PSO, CSA, MSGO) and four oft-used TL models (MobileNetV2, ResNet18, ResNet50, ShuffleNet) to classify the X-ray KOA images. Regarding the experiments on both categorization tasks, the MSGO algorithm arises as more robust to be considered for hyperparameter tuning of TL-based models by achieving reliable performance. In addition, it's seen that MobileNetV2 and ResNet-based models come to the forefront for X-ray imaging-based classification by achieving high accuracy rates due to the usage of residual blocks. Consequently, in terms of mean accuracy, ResNet50-MSGO and MobileNetV2-CSA respectively record 93.15 % and 93.29 % success rates on multiclass categorization, while ResNet18-CDW-PSO and MobileNetV2-MSGO provide the same highest score of 99.43 % on binary categorization.</p></div>","PeriodicalId":48682,"journal":{"name":"Swarm and Evolutionary Computation","volume":null,"pages":null},"PeriodicalIF":8.2000,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Swarm and Evolutionary Computation","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2210650224001780","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Discrete & continuous optimization constitutes a challenging task and generally rises as an NP-hard problem. In the literature, as a derivative of this type of optimization issue, hyperparameter optimization of transfer learning (TL) architectures is not efficiently analyzed as a detailed survey in the literature. In this paper, the optimized TL-based models are effectively examined to handle this issue which constitutes the main aim of our study. For evaluation, knee osteoarthritis (KOA – a chronic degenerative joint disorder) dataset is handled to perform two challenging classification tasks which reveal the second aim of our study, i.e. binary- and multi-categorizations on KOA X-ray images. To fine-tune the hyperparameters of TL models, state-of-the-art optimization methods are chosen and compared on this competitive – NP-hard problem. Sixteen optimized architectures are designed using four efficient optimization methods (ASPSO, CDW-PSO, CSA, MSGO) and four oft-used TL models (MobileNetV2, ResNet18, ResNet50, ShuffleNet) to classify the X-ray KOA images. Regarding the experiments on both categorization tasks, the MSGO algorithm arises as more robust to be considered for hyperparameter tuning of TL-based models by achieving reliable performance. In addition, it's seen that MobileNetV2 and ResNet-based models come to the forefront for X-ray imaging-based classification by achieving high accuracy rates due to the usage of residual blocks. Consequently, in terms of mean accuracy, ResNet50-MSGO and MobileNetV2-CSA respectively record 93.15 % and 93.29 % success rates on multiclass categorization, while ResNet18-CDW-PSO and MobileNetV2-MSGO provide the same highest score of 99.43 % on binary categorization.
期刊介绍:
Swarm and Evolutionary Computation is a pioneering peer-reviewed journal focused on the latest research and advancements in nature-inspired intelligent computation using swarm and evolutionary algorithms. It covers theoretical, experimental, and practical aspects of these paradigms and their hybrids, promoting interdisciplinary research. The journal prioritizes the publication of high-quality, original articles that push the boundaries of evolutionary computation and swarm intelligence. Additionally, it welcomes survey papers on current topics and novel applications. Topics of interest include but are not limited to: Genetic Algorithms, and Genetic Programming, Evolution Strategies, and Evolutionary Programming, Differential Evolution, Artificial Immune Systems, Particle Swarms, Ant Colony, Bacterial Foraging, Artificial Bees, Fireflies Algorithm, Harmony Search, Artificial Life, Digital Organisms, Estimation of Distribution Algorithms, Stochastic Diffusion Search, Quantum Computing, Nano Computing, Membrane Computing, Human-centric Computing, Hybridization of Algorithms, Memetic Computing, Autonomic Computing, Self-organizing systems, Combinatorial, Discrete, Binary, Constrained, Multi-objective, Multi-modal, Dynamic, and Large-scale Optimization.