首页 > 最新文献

2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)最新文献

英文 中文
Explainable Machine Learning Models for Pneumonia Mortality Risk Prediction Using MIMIC-III Data 使用MIMIC-III数据预测肺炎死亡率风险的可解释机器学习模型
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068438
James Sanii, Wai-Yip Chan
To gain trust, machine learning (ML) models used in high stake applications such as clinical decision support need to provide explainable behaviours and outputs. To assess whether interpretable explanations can be obtained without sacrificing prediction performance, we compare using “black box” versus “glass box” models for predicting the mortality risk of patients diagnosed with pneumonia, using data in the MIMIC-III dataset. We examine five types of black box models: random forest (RF), support vector machine (SVM), gradient boosting classifier (GBC), AdaBoost (ADA), and multilayer perceptron (MLP), and three types of glassbox models: K-nearest neighbor (KNN), explainable boosting machine (EBM), and generalized additive models (GAM). When trained using 417 features, a black box RF model performs best with AUC of 0.896. With the feature set size reduced to 19, an EBM model performs the best with AUC 0.872. Both models exceed the AUC of 0.661, the best previously reported for the task. Our results suggest that ML models with inbuilt explainability may provide prediction power as attractive as black box models.
为了获得信任,用于高风险应用(如临床决策支持)的机器学习(ML)模型需要提供可解释的行为和输出。为了评估是否可以在不牺牲预测性能的情况下获得可解释的解释,我们使用MIMIC-III数据集中的数据,比较了使用“黑盒”模型和“玻璃盒”模型来预测诊断为肺炎患者的死亡风险。我们研究了五种类型的黑盒模型:随机森林(RF)、支持向量机(SVM)、梯度增强分类器(GBC)、AdaBoost (ADA)和多层感知器(MLP),以及三种类型的玻璃盒模型:k近邻(KNN)、可解释增强机(EBM)和广义加性模型(GAM)。当使用417个特征训练时,黑盒RF模型的AUC为0.896,表现最佳。当特征集减小到19时,EBM模型的AUC为0.872,表现最佳。这两个模型的AUC都超过了0.661,这是该任务之前报告的最佳AUC。我们的研究结果表明,具有内置可解释性的机器学习模型可以提供与黑盒模型一样有吸引力的预测能力。
{"title":"Explainable Machine Learning Models for Pneumonia Mortality Risk Prediction Using MIMIC-III Data","authors":"James Sanii, Wai-Yip Chan","doi":"10.1109/ISCMI56532.2022.10068438","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068438","url":null,"abstract":"To gain trust, machine learning (ML) models used in high stake applications such as clinical decision support need to provide explainable behaviours and outputs. To assess whether interpretable explanations can be obtained without sacrificing prediction performance, we compare using “black box” versus “glass box” models for predicting the mortality risk of patients diagnosed with pneumonia, using data in the MIMIC-III dataset. We examine five types of black box models: random forest (RF), support vector machine (SVM), gradient boosting classifier (GBC), AdaBoost (ADA), and multilayer perceptron (MLP), and three types of glassbox models: K-nearest neighbor (KNN), explainable boosting machine (EBM), and generalized additive models (GAM). When trained using 417 features, a black box RF model performs best with AUC of 0.896. With the feature set size reduced to 19, an EBM model performs the best with AUC 0.872. Both models exceed the AUC of 0.661, the best previously reported for the task. Our results suggest that ML models with inbuilt explainability may provide prediction power as attractive as black box models.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"83 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130334679","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluating the performance of the Quantum Approximate Optimisation Algorithm to solve the Quadratic Assignment Problem 评价量子近似优化算法求解二次分配问题的性能
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068445
M. Khumalo, K. Prag, K. Nixon
The performance of the Quantum Approximate Optimisation Algorithm (QAOA) in solving the Quadratic Assignment Problem (QAP) is evaluated, with the Variational Quantum Eigensolver (VQE) as a benchmark. The QAP is directly revelant to numerous industry scenarios. The QAP, a Combinatorial Optimisation Problem (COP), is classified as $mathcal{NP}$ -Hard. This classification means CPU time increases exponentially as the problem size scales when solving the QAP using deterministic optimisation techniques. Therefore, this work investigates the QAOA in search of a non-deterministic optimisation technique to efficiently obtain solutions to the QAP. This research compares two warm start techniques to solve QAP instances of sizes 3 to 7. The metrics of comparison - that measure efficiency and solution quality - were introduced in previous work on this topic. For the QAOA, the impact of the p-value, a determination of circuit depth, is investigated. Of the two quantum hybrid heuristics, the VQE retrieves solutions in a shorter computational time with a smaller circuit size, which allows for solving instances with a larger problem size. Compared to the VQE, the QAOA performs better in terms of feasibility as the problem size scales. The quantum warm start method results implies that the QAOA may not maintain higher solution quality for instances larger than size 4. Still, further investigation should be conducted once quantum devices with more qubits and higher quantum volumes are available.
以变分量子特征求解器(VQE)为基准,评价了量子近似优化算法(QAOA)在求解二次分配问题(QAP)中的性能。QAP与许多行业场景直接相关。QAP是一个组合优化问题(COP),被分类为$mathcal{NP}$ -Hard。这种分类意味着在使用确定性优化技术解决QAP时,CPU时间随着问题规模的扩大呈指数增长。因此,本文研究了QAOA,以寻找一种非确定性优化技术来有效地获得QAP的解。本研究比较了两种热启动技术来解决大小为3到7的QAP实例。比较的度量标准——衡量效率和解决方案质量——在本主题的前面的工作中已经介绍过。对于QAOA,研究了决定电路深度的p值的影响。在这两种量子混合启发式方法中,VQE以更短的计算时间和更小的电路尺寸检索解决方案,这允许解决具有更大问题规模的实例。与VQE相比,随着问题规模的扩大,QAOA在可行性方面表现得更好。量子热启动方法的结果表明,对于尺寸大于4的实例,QAOA可能无法保持较高的解质量。然而,一旦拥有更多量子比特和更大量子体积的量子器件可用,就应该进行进一步的研究。
{"title":"Evaluating the performance of the Quantum Approximate Optimisation Algorithm to solve the Quadratic Assignment Problem","authors":"M. Khumalo, K. Prag, K. Nixon","doi":"10.1109/ISCMI56532.2022.10068445","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068445","url":null,"abstract":"The performance of the Quantum Approximate Optimisation Algorithm (QAOA) in solving the Quadratic Assignment Problem (QAP) is evaluated, with the Variational Quantum Eigensolver (VQE) as a benchmark. The QAP is directly revelant to numerous industry scenarios. The QAP, a Combinatorial Optimisation Problem (COP), is classified as $mathcal{NP}$ -Hard. This classification means CPU time increases exponentially as the problem size scales when solving the QAP using deterministic optimisation techniques. Therefore, this work investigates the QAOA in search of a non-deterministic optimisation technique to efficiently obtain solutions to the QAP. This research compares two warm start techniques to solve QAP instances of sizes 3 to 7. The metrics of comparison - that measure efficiency and solution quality - were introduced in previous work on this topic. For the QAOA, the impact of the p-value, a determination of circuit depth, is investigated. Of the two quantum hybrid heuristics, the VQE retrieves solutions in a shorter computational time with a smaller circuit size, which allows for solving instances with a larger problem size. Compared to the VQE, the QAOA performs better in terms of feasibility as the problem size scales. The quantum warm start method results implies that the QAOA may not maintain higher solution quality for instances larger than size 4. Still, further investigation should be conducted once quantum devices with more qubits and higher quantum volumes are available.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"60 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128013900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Exploring the Potential of a Genetic Algorithm on a Real-World Complex Scheduling Problem 探索遗传算法在现实世界复杂调度问题上的潜力
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068465
Szilvia Jáhn-Erdös, Bence Kövári
Genetic algorithms for NP-complete problems are widespread since it is easy to obtain a solution to the problem. However, its optimality gives the real issue and is not guaranteed to be achievable. In our research, we address a special subproblem of scheduling problems, the final exam scheduling, in which special requirements restrict the state space, which often contradicts each other. The task's difficulty is the massive size of the state space. Genetic algorithm-based solutions were considered since a MILP solver could not find a solution in a reasonable time. A model was built to solve this problem using the genetic algorithm. Most of the possibilities were seen in the different mutation procedures, so we investigated them in more detail. A question for genetic algorithms is what parameters and probabilities to run the model with since the more freedom we give to the run, the larger the runtime. Finding the threshold between the two is essential. Therefore, our experiments measured sizeable real data sets to find the optimal values for this complex problem. The resulting algorithm can significantly facilitate the lengthy manual scheduling processes carried out so far in our university.
np完全问题的遗传算法由于易于求解而得到广泛应用。然而,它的最优性给出了真正的问题,并不能保证可以实现。在我们的研究中,我们解决了调度问题的一个特殊的子问题,期末考试调度,其中特殊的需求限制了状态空间,往往是相互矛盾的。该任务的难点在于状态空间的巨大规模。由于MILP求解器无法在合理时间内找到解,因此考虑了基于遗传算法的解。利用遗传算法建立了求解该问题的模型。在不同的突变过程中可以看到大多数可能性,因此我们对它们进行了更详细的研究。遗传算法的一个问题是运行模型的参数和概率,因为我们给运行的自由度越大,运行时间就越长。找到两者之间的界限是至关重要的。因此,我们的实验测量了相当大的真实数据集,以找到这个复杂问题的最佳值。由此产生的算法可以大大简化目前在我校进行的冗长的人工调度过程。
{"title":"Exploring the Potential of a Genetic Algorithm on a Real-World Complex Scheduling Problem","authors":"Szilvia Jáhn-Erdös, Bence Kövári","doi":"10.1109/ISCMI56532.2022.10068465","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068465","url":null,"abstract":"Genetic algorithms for NP-complete problems are widespread since it is easy to obtain a solution to the problem. However, its optimality gives the real issue and is not guaranteed to be achievable. In our research, we address a special subproblem of scheduling problems, the final exam scheduling, in which special requirements restrict the state space, which often contradicts each other. The task's difficulty is the massive size of the state space. Genetic algorithm-based solutions were considered since a MILP solver could not find a solution in a reasonable time. A model was built to solve this problem using the genetic algorithm. Most of the possibilities were seen in the different mutation procedures, so we investigated them in more detail. A question for genetic algorithms is what parameters and probabilities to run the model with since the more freedom we give to the run, the larger the runtime. Finding the threshold between the two is essential. Therefore, our experiments measured sizeable real data sets to find the optimal values for this complex problem. The resulting algorithm can significantly facilitate the lengthy manual scheduling processes carried out so far in our university.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128649661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Non-fungible Token Price Prediction with Multivariate LSTM Neural Networks 基于多元LSTM神经网络的不可替代代币价格预测
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068442
Jerome Branny, Rolf Dornberger, T. Hanne
In this paper, we investigate how to forecast Non-Fungible Token (NFT) sale prices by using multiple multivariate time series datasets containing features related to the NFT market space. We examined eight recent studies regarding the forecasting and valuation of NFTs and compared their most important findings. This laid the fundamental work for two separate machine learning prototypes based on Long Short-Term Memory (LSTM) which are able to forecast the sale price history of an individual NFT asset. Root Mean Squared Errors (RMSE) of 0.2975 and 0.24 were obtained which appears to be promising.
在本文中,我们研究了如何通过使用包含与NFT市场空间相关特征的多个多元时间序列数据集来预测非可替换代币(NFT)的销售价格。我们研究了最近关于nft预测和评估的八项研究,并比较了它们最重要的发现。这为两个独立的基于长短期记忆(LSTM)的机器学习原型奠定了基础,它们能够预测单个NFT资产的销售价格历史。获得的均方根误差(RMSE)为0.2975和0.24,这似乎是有希望的。
{"title":"Non-fungible Token Price Prediction with Multivariate LSTM Neural Networks","authors":"Jerome Branny, Rolf Dornberger, T. Hanne","doi":"10.1109/ISCMI56532.2022.10068442","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068442","url":null,"abstract":"In this paper, we investigate how to forecast Non-Fungible Token (NFT) sale prices by using multiple multivariate time series datasets containing features related to the NFT market space. We examined eight recent studies regarding the forecasting and valuation of NFTs and compared their most important findings. This laid the fundamental work for two separate machine learning prototypes based on Long Short-Term Memory (LSTM) which are able to forecast the sale price history of an individual NFT asset. Root Mean Squared Errors (RMSE) of 0.2975 and 0.24 were obtained which appears to be promising.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128681183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Loan Repayment Prediction Using Logistic Regression Ensemble Learning With Machine Learning Algorithms 使用逻辑回归集成学习与机器学习算法的贷款还款预测
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068483
T. Dinh, Binh Pham Thanh
Lending activities are an important part of the credit activities of financial institutions and banks. This is an area that brings great potential for development as well as a sustainable source of profit for financial institutions and banks. However, lending to customers also brings high risks. Therefore, predicting the ability to repay on time and understanding the factors affecting the repayment ability of customers is extremely important and necessary, to help financial institutions and banks enhance their ability to pay debts. customers' ability to identify and pay debts on time, contributing to minimizing bad debts and enhancing credit risk management. In this study, Machine Learning models will be used: Proposing a method to combine Logistic Regression with Random Forest, Logistic Regression with K-Nearest Neighbor, Logistic Regression with Support Vector Machine, Logistic Regression with Artificial Neural Network, Logistic Regression with Long short-term memory and finally Logistic Regression with Decision Tree to predict customers' ability to repay on time and compare and evaluate the performance of Machine Learning models. As a result, the Logistic Regression with the Random Forest model ensemble is found as the optimal predictive model and it is expected that Fico Score and annual income significantly influence the forecast.
贷款活动是金融机构和银行信贷活动的重要组成部分。这是一个具有巨大发展潜力的领域,也是金融机构和银行的可持续利润来源。然而,贷款给客户也带来了高风险。因此,预测客户的按时还款能力,了解影响客户还款能力的因素,对于帮助金融机构和银行提高偿债能力是极其重要和必要的。客户及时识别和偿还债务的能力,有助于减少坏账,加强信用风险管理。本研究将使用机器学习模型:提出将Logistic回归与随机森林、Logistic回归与k近邻、Logistic回归与支持向量机、Logistic回归与人工神经网络、Logistic回归与长短期记忆、Logistic回归与决策树相结合的方法来预测客户的按时还款能力,并比较和评估机器学习模型的性能。结果表明,随机森林模型集合的Logistic回归是最优预测模型,Fico评分和年收入对预测结果有显著影响。
{"title":"Loan Repayment Prediction Using Logistic Regression Ensemble Learning With Machine Learning Algorithms","authors":"T. Dinh, Binh Pham Thanh","doi":"10.1109/ISCMI56532.2022.10068483","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068483","url":null,"abstract":"Lending activities are an important part of the credit activities of financial institutions and banks. This is an area that brings great potential for development as well as a sustainable source of profit for financial institutions and banks. However, lending to customers also brings high risks. Therefore, predicting the ability to repay on time and understanding the factors affecting the repayment ability of customers is extremely important and necessary, to help financial institutions and banks enhance their ability to pay debts. customers' ability to identify and pay debts on time, contributing to minimizing bad debts and enhancing credit risk management. In this study, Machine Learning models will be used: Proposing a method to combine Logistic Regression with Random Forest, Logistic Regression with K-Nearest Neighbor, Logistic Regression with Support Vector Machine, Logistic Regression with Artificial Neural Network, Logistic Regression with Long short-term memory and finally Logistic Regression with Decision Tree to predict customers' ability to repay on time and compare and evaluate the performance of Machine Learning models. As a result, the Logistic Regression with the Random Forest model ensemble is found as the optimal predictive model and it is expected that Fico Score and annual income significantly influence the forecast.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133938416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Population-Based Algorithm for Solving the Fuzzy Capacitated Maximal Covering Location Problem 一种求解模糊最大覆盖定位问题的种群算法
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068459
Méziane Aïder, Imene Dey, M. Hifi
In this paper, we investigate the use of a population-based algorithm for tackling the fuzzy capacitated maximal covering location problem. Such a problem is characterized by a set of customers with their distances and its goal is to determine a subset of locations positioned on customers such that a maximum coverage of customers, including the both fuzzy coverage degree of facilities and the distance between customers, should be optimized. The proposed method is based upon the grey wolf optimizer, which starts by generating an initial population using a greedy rule strategy that is able to achieve feasible solutions according to the current positions of wolves. In order to enhance the quality of solutions induced, a series of local searches are added for exploring the search space by exploiting some nice strategies. The behavior of the method is computationally analyzed on a set of instances of the literature. Encouraging results have been provided.
在本文中,我们研究了一种基于种群的算法来解决模糊最大覆盖定位问题。该问题的特征是一组客户及其距离,其目标是确定客户位置的子集,使客户的最大覆盖范围,包括设施的模糊覆盖程度和客户之间的距离。提出的方法基于灰狼优化器,首先使用贪婪规则策略生成初始种群,该策略能够根据狼的当前位置获得可行解。为了提高归纳出的解的质量,通过利用一些好的策略,增加了一系列局部搜索来探索搜索空间。在一组文献实例上对该方法的行为进行了计算分析。已经取得了令人鼓舞的成果。
{"title":"A Population-Based Algorithm for Solving the Fuzzy Capacitated Maximal Covering Location Problem","authors":"Méziane Aïder, Imene Dey, M. Hifi","doi":"10.1109/ISCMI56532.2022.10068459","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068459","url":null,"abstract":"In this paper, we investigate the use of a population-based algorithm for tackling the fuzzy capacitated maximal covering location problem. Such a problem is characterized by a set of customers with their distances and its goal is to determine a subset of locations positioned on customers such that a maximum coverage of customers, including the both fuzzy coverage degree of facilities and the distance between customers, should be optimized. The proposed method is based upon the grey wolf optimizer, which starts by generating an initial population using a greedy rule strategy that is able to achieve feasible solutions according to the current positions of wolves. In order to enhance the quality of solutions induced, a series of local searches are added for exploring the search space by exploiting some nice strategies. The behavior of the method is computationally analyzed on a set of instances of the literature. Encouraging results have been provided.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"171 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132182983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Hadamard Estimated Attention Transformer (HEAT): Fast Approximation of Dot Product Self-attention for Transformers Using Low-Rank Projection of Hadamard Product Hadamard估计注意变压器(HEAT):利用Hadamard积的低秩投影快速逼近变压器的点积自注意
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068484
Jasper Kyle Catapang
In this paper, the author proposes a new transformer model called Hadamard Estimated Attention Transformer or HEAT, that utilizes a low-rank projection of the Hadamard product to approximate the self-attention mechanism in standard transformer architectures and thus aiming to speedup transformer training, finetuning, and inference altogether. The study shows how it is significantly better than the original transformer that uses dot product self-attention by offering a faster way to compute the original self-attention mechanism while maintaining and ultimately surpassing the quality of the original transformer architecture. It also bests Linformer and Nyströmformer in several machine translation tasks while matching and even outperforming Nyströmformer's accuracy in various text classification tasks.
在本文中,作者提出了一个新的变压器模型,称为Hadamard估计注意变压器或HEAT,它利用Hadamard乘积的低秩投影来近似标准变压器架构中的自注意机制,从而旨在加速变压器的训练,微调和推理。该研究表明,通过提供一种更快的方法来计算原始自注意机制,同时保持并最终超越原始变压器架构的质量,它如何明显优于使用点积自注意的原始变压器。它还在几个机器翻译任务中优于Linformer和Nyströmformer,而在各种文本分类任务中匹配甚至超过Nyströmformer的准确性。
{"title":"Hadamard Estimated Attention Transformer (HEAT): Fast Approximation of Dot Product Self-attention for Transformers Using Low-Rank Projection of Hadamard Product","authors":"Jasper Kyle Catapang","doi":"10.1109/ISCMI56532.2022.10068484","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068484","url":null,"abstract":"In this paper, the author proposes a new transformer model called Hadamard Estimated Attention Transformer or HEAT, that utilizes a low-rank projection of the Hadamard product to approximate the self-attention mechanism in standard transformer architectures and thus aiming to speedup transformer training, finetuning, and inference altogether. The study shows how it is significantly better than the original transformer that uses dot product self-attention by offering a faster way to compute the original self-attention mechanism while maintaining and ultimately surpassing the quality of the original transformer architecture. It also bests Linformer and Nyströmformer in several machine translation tasks while matching and even outperforming Nyströmformer's accuracy in various text classification tasks.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"316 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134507700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Comparison of Trajectory and Population-Based Algorithms for Optimizing Constrained Open-Pit Mining Problem 基于轨迹和种群的约束露天开采优化算法比较
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068481
I. Rahimi, Theodore Picard, Andrew Morabito, Kiriakos Pampalis, Aiden Abignano, A. Gandomi
The problem of open-pit mining optimization is a complex task, often containing many variables. In this paper, we apply a trajectory-based algorithm known as simulated annealing together with a well-known population-based algorithm, genetic algorithm, used to generate solutions for a formulation of the constrained pit problem (CPIT). Three datasets were used to test this simulation, Newman1, zuck_small, and KD. The results show that simulated annealing as a trajectory algorithm possesses a slightly better performance in comparison with the genetic algorithm in terms of profit value.
露天矿开采优化问题是一个复杂的问题,往往包含许多变量。在本文中,我们应用了一种基于轨迹的算法,即模拟退火算法,以及一种众所周知的基于种群的算法,即遗传算法,用于生成约束坑问题(CPIT)公式的解。三个数据集用于测试该模拟,Newman1, zuck_small和KD。结果表明,模拟退火作为一种轨迹算法,在利润值方面略优于遗传算法。
{"title":"Comparison of Trajectory and Population-Based Algorithms for Optimizing Constrained Open-Pit Mining Problem","authors":"I. Rahimi, Theodore Picard, Andrew Morabito, Kiriakos Pampalis, Aiden Abignano, A. Gandomi","doi":"10.1109/ISCMI56532.2022.10068481","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068481","url":null,"abstract":"The problem of open-pit mining optimization is a complex task, often containing many variables. In this paper, we apply a trajectory-based algorithm known as simulated annealing together with a well-known population-based algorithm, genetic algorithm, used to generate solutions for a formulation of the constrained pit problem (CPIT). Three datasets were used to test this simulation, Newman1, zuck_small, and KD. The results show that simulated annealing as a trajectory algorithm possesses a slightly better performance in comparison with the genetic algorithm in terms of profit value.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124997707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
CILDI: Class Incremental Learning with Distilled Images 用蒸馏图像进行课堂增量学习
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068463
Abel S. Zacarias, L. Alexandre
Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previously learned tasks. Learning new tasks in most proposals, implies to keeping examples of previously learned tasks to retrain the model when learning new tasks, which has an impact in terms of storage capacity. In this paper, we present a method that adds new capabilities, in an incrementally way, to an existing model keeping examples from previously learned classes but avoiding the problem of running out of storage by using distilled images to condensate sets of images into a single image. The experimental results on four data sets confirmed the effectiveness of CILDI to learn new classes incrementally across different tasks and obtaining a performance close to the state-of-the-art algorithms for class incremental learning using only one distilled image per learned class and beating the state-of-the-art on the four data sets when using 10 distilled images per learned class, while using a smaller memory footprint than the competing approaches.
终身学习旨在开发能够学习新任务的机器学习系统,同时保持以前学习任务的性能。在大多数建议中,学习新任务意味着在学习新任务时保留以前学习过的任务的示例来重新训练模型,这在存储容量方面有影响。在本文中,我们提出了一种方法,该方法以增量的方式为现有模型添加新功能,该模型保留了以前学习过的类的示例,但通过使用蒸馏图像将图像集凝聚成单个图像来避免耗尽存储的问题。在四个数据集上的实验结果证实了CILDI在不同任务中增量学习新类的有效性,并且在每个学习类只使用一个蒸馏图像的情况下获得接近最先进的类增量学习算法的性能,并且在每个学习类使用10个蒸馏图像的情况下在四个数据集上击败了最先进的算法,同时使用比竞争方法更小的内存占用。
{"title":"CILDI: Class Incremental Learning with Distilled Images","authors":"Abel S. Zacarias, L. Alexandre","doi":"10.1109/ISCMI56532.2022.10068463","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068463","url":null,"abstract":"Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previously learned tasks. Learning new tasks in most proposals, implies to keeping examples of previously learned tasks to retrain the model when learning new tasks, which has an impact in terms of storage capacity. In this paper, we present a method that adds new capabilities, in an incrementally way, to an existing model keeping examples from previously learned classes but avoiding the problem of running out of storage by using distilled images to condensate sets of images into a single image. The experimental results on four data sets confirmed the effectiveness of CILDI to learn new classes incrementally across different tasks and obtaining a performance close to the state-of-the-art algorithms for class incremental learning using only one distilled image per learned class and beating the state-of-the-art on the four data sets when using 10 distilled images per learned class, while using a smaller memory footprint than the competing approaches.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125309795","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Comparison of Linear Rank and Tournament for Parent Selection in a Genetic Algorithm Solving a Dynamic Travelling Salesman Problem 求解动态旅行商问题的遗传算法双亲选择的线性秩和比武比较
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068458
Ramona Boeh, T. Hanne, Rolf Dornberger
We compare the two parent selection methods “linear rank” and “tournament” in a Genetic Algorithm applied to a dynamic Travelling Salesman Problem (TSP). The inherent dynamics of the problem is considered by temporarily doubling the costs between two randomly selected cities. In our experiments we take into account tournament selection with tournament sizes of 3, 5, and 10. A larger tournament size results in as good a performance as with linear rank selection in a small-scale dynamic TSP, whereas smaller tournament sizes better preserve the diversity of the population and avoid getting stuck in local optima. However, the assumption that tournament is superior to linear rank on a dynamic TSP could neither be confirmed nor falsified in the applied testcases.
本文比较了应用于动态旅行商问题(TSP)的遗传算法中“线性排序”和“竞赛”两种父代选择方法。将两个随机选择的城市之间的成本暂时翻倍来考虑这个问题的内在动态。在我们的实验中,我们考虑了比赛规模为3、5和10的比赛选择。在小规模动态TSP中,较大的比赛规模会产生与线性排名选择一样好的表现,而较小的比赛规模则可以更好地保持种群的多样性,避免陷入局部最优状态。然而,在应用的测试用例中,锦标赛优于动态TSP上的线性排名的假设既不能被证实也不能被证伪。
{"title":"A Comparison of Linear Rank and Tournament for Parent Selection in a Genetic Algorithm Solving a Dynamic Travelling Salesman Problem","authors":"Ramona Boeh, T. Hanne, Rolf Dornberger","doi":"10.1109/ISCMI56532.2022.10068458","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068458","url":null,"abstract":"We compare the two parent selection methods “linear rank” and “tournament” in a Genetic Algorithm applied to a dynamic Travelling Salesman Problem (TSP). The inherent dynamics of the problem is considered by temporarily doubling the costs between two randomly selected cities. In our experiments we take into account tournament selection with tournament sizes of 3, 5, and 10. A larger tournament size results in as good a performance as with linear rank selection in a small-scale dynamic TSP, whereas smaller tournament sizes better preserve the diversity of the population and avoid getting stuck in local optima. However, the assumption that tournament is superior to linear rank on a dynamic TSP could neither be confirmed nor falsified in the applied testcases.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123899392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1