首页 > 最新文献

2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)最新文献

英文 中文
The Analysis of the Generator Architectures and Loss Functions in Improving the Stability of GANs Training towards Efficient Intrusion Detection 生成器结构和损失函数在提高gan训练稳定性以实现高效入侵检测中的分析
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068468
Raha Soleymanzadeh, R. Kashef
Various research studies have been recently introduced in developing generative models, especially in computer vision and image classification. These models are inspired by a generator and discriminator network architecture in a min-max optimization game called Generative Adversarial Networks (GANs). However, GANs-based models suffer from training instability, which means high oscillations during the training, which provides inaccurate results. There are various causes beyond the instability behaviours, such as the adopted generator architecture, loss function, and distance metrics. In this paper, we focus on the impact of the generator architectures and the loss functions on the GANs training. We aim to provide a comparative assessment of various architectures focusing on ensemble and hybrid models and loss functions such as Focal loss, Binary Cross-Entropy and Mean Squared loss function. Experimental results on NSL-KDD and UNSW-NB15 datasets show that the ensemble models are more stable in terms of training and have higher intrusion detection rates. Additionally, the focal loss can improve the performance of detection minority classes. Using Mean squared loss improved the detection rate for discriminator, however with the Binary Cross entropy loss function, the deep features representation is improved and there is more stability in trends for all architectures.
近年来,人们对生成模型的发展进行了大量的研究,特别是在计算机视觉和图像分类方面。这些模型的灵感来自最小-最大优化博弈生成对抗网络(GANs)中的生成器和鉴别器网络架构。然而,基于高斯的模型存在训练不稳定性,这意味着训练过程中的高振荡,从而提供不准确的结果。除不稳定行为外,还有各种原因,如采用的发电机结构、损失函数和距离度量。在本文中,我们重点研究了生成器结构和损失函数对gan训练的影响。我们的目标是提供各种架构的比较评估,重点是集成和混合模型以及损失函数,如焦损失,二元交叉熵和均方损失函数。在NSL-KDD和UNSW-NB15数据集上的实验结果表明,集成模型在训练方面更稳定,入侵检测率更高。此外,焦损可以提高检测少数类的性能。使用均方损失提高了鉴别器的检测率,而使用二元交叉熵损失函数改进了深度特征表示,并且对所有体系结构都具有更大的趋势稳定性。
{"title":"The Analysis of the Generator Architectures and Loss Functions in Improving the Stability of GANs Training towards Efficient Intrusion Detection","authors":"Raha Soleymanzadeh, R. Kashef","doi":"10.1109/ISCMI56532.2022.10068468","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068468","url":null,"abstract":"Various research studies have been recently introduced in developing generative models, especially in computer vision and image classification. These models are inspired by a generator and discriminator network architecture in a min-max optimization game called Generative Adversarial Networks (GANs). However, GANs-based models suffer from training instability, which means high oscillations during the training, which provides inaccurate results. There are various causes beyond the instability behaviours, such as the adopted generator architecture, loss function, and distance metrics. In this paper, we focus on the impact of the generator architectures and the loss functions on the GANs training. We aim to provide a comparative assessment of various architectures focusing on ensemble and hybrid models and loss functions such as Focal loss, Binary Cross-Entropy and Mean Squared loss function. Experimental results on NSL-KDD and UNSW-NB15 datasets show that the ensemble models are more stable in terms of training and have higher intrusion detection rates. Additionally, the focal loss can improve the performance of detection minority classes. Using Mean squared loss improved the detection rate for discriminator, however with the Binary Cross entropy loss function, the deep features representation is improved and there is more stability in trends for all architectures.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"162 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126807916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Evaluating the performance of the Quantum Approximate Optimisation Algorithm to solve the Quadratic Assignment Problem 评价量子近似优化算法求解二次分配问题的性能
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068445
M. Khumalo, K. Prag, K. Nixon
The performance of the Quantum Approximate Optimisation Algorithm (QAOA) in solving the Quadratic Assignment Problem (QAP) is evaluated, with the Variational Quantum Eigensolver (VQE) as a benchmark. The QAP is directly revelant to numerous industry scenarios. The QAP, a Combinatorial Optimisation Problem (COP), is classified as $mathcal{NP}$ -Hard. This classification means CPU time increases exponentially as the problem size scales when solving the QAP using deterministic optimisation techniques. Therefore, this work investigates the QAOA in search of a non-deterministic optimisation technique to efficiently obtain solutions to the QAP. This research compares two warm start techniques to solve QAP instances of sizes 3 to 7. The metrics of comparison - that measure efficiency and solution quality - were introduced in previous work on this topic. For the QAOA, the impact of the p-value, a determination of circuit depth, is investigated. Of the two quantum hybrid heuristics, the VQE retrieves solutions in a shorter computational time with a smaller circuit size, which allows for solving instances with a larger problem size. Compared to the VQE, the QAOA performs better in terms of feasibility as the problem size scales. The quantum warm start method results implies that the QAOA may not maintain higher solution quality for instances larger than size 4. Still, further investigation should be conducted once quantum devices with more qubits and higher quantum volumes are available.
以变分量子特征求解器(VQE)为基准,评价了量子近似优化算法(QAOA)在求解二次分配问题(QAP)中的性能。QAP与许多行业场景直接相关。QAP是一个组合优化问题(COP),被分类为$mathcal{NP}$ -Hard。这种分类意味着在使用确定性优化技术解决QAP时,CPU时间随着问题规模的扩大呈指数增长。因此,本文研究了QAOA,以寻找一种非确定性优化技术来有效地获得QAP的解。本研究比较了两种热启动技术来解决大小为3到7的QAP实例。比较的度量标准——衡量效率和解决方案质量——在本主题的前面的工作中已经介绍过。对于QAOA,研究了决定电路深度的p值的影响。在这两种量子混合启发式方法中,VQE以更短的计算时间和更小的电路尺寸检索解决方案,这允许解决具有更大问题规模的实例。与VQE相比,随着问题规模的扩大,QAOA在可行性方面表现得更好。量子热启动方法的结果表明,对于尺寸大于4的实例,QAOA可能无法保持较高的解质量。然而,一旦拥有更多量子比特和更大量子体积的量子器件可用,就应该进行进一步的研究。
{"title":"Evaluating the performance of the Quantum Approximate Optimisation Algorithm to solve the Quadratic Assignment Problem","authors":"M. Khumalo, K. Prag, K. Nixon","doi":"10.1109/ISCMI56532.2022.10068445","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068445","url":null,"abstract":"The performance of the Quantum Approximate Optimisation Algorithm (QAOA) in solving the Quadratic Assignment Problem (QAP) is evaluated, with the Variational Quantum Eigensolver (VQE) as a benchmark. The QAP is directly revelant to numerous industry scenarios. The QAP, a Combinatorial Optimisation Problem (COP), is classified as $mathcal{NP}$ -Hard. This classification means CPU time increases exponentially as the problem size scales when solving the QAP using deterministic optimisation techniques. Therefore, this work investigates the QAOA in search of a non-deterministic optimisation technique to efficiently obtain solutions to the QAP. This research compares two warm start techniques to solve QAP instances of sizes 3 to 7. The metrics of comparison - that measure efficiency and solution quality - were introduced in previous work on this topic. For the QAOA, the impact of the p-value, a determination of circuit depth, is investigated. Of the two quantum hybrid heuristics, the VQE retrieves solutions in a shorter computational time with a smaller circuit size, which allows for solving instances with a larger problem size. Compared to the VQE, the QAOA performs better in terms of feasibility as the problem size scales. The quantum warm start method results implies that the QAOA may not maintain higher solution quality for instances larger than size 4. Still, further investigation should be conducted once quantum devices with more qubits and higher quantum volumes are available.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"60 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128013900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Exploring the Potential of a Genetic Algorithm on a Real-World Complex Scheduling Problem 探索遗传算法在现实世界复杂调度问题上的潜力
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068465
Szilvia Jáhn-Erdös, Bence Kövári
Genetic algorithms for NP-complete problems are widespread since it is easy to obtain a solution to the problem. However, its optimality gives the real issue and is not guaranteed to be achievable. In our research, we address a special subproblem of scheduling problems, the final exam scheduling, in which special requirements restrict the state space, which often contradicts each other. The task's difficulty is the massive size of the state space. Genetic algorithm-based solutions were considered since a MILP solver could not find a solution in a reasonable time. A model was built to solve this problem using the genetic algorithm. Most of the possibilities were seen in the different mutation procedures, so we investigated them in more detail. A question for genetic algorithms is what parameters and probabilities to run the model with since the more freedom we give to the run, the larger the runtime. Finding the threshold between the two is essential. Therefore, our experiments measured sizeable real data sets to find the optimal values for this complex problem. The resulting algorithm can significantly facilitate the lengthy manual scheduling processes carried out so far in our university.
np完全问题的遗传算法由于易于求解而得到广泛应用。然而,它的最优性给出了真正的问题,并不能保证可以实现。在我们的研究中,我们解决了调度问题的一个特殊的子问题,期末考试调度,其中特殊的需求限制了状态空间,往往是相互矛盾的。该任务的难点在于状态空间的巨大规模。由于MILP求解器无法在合理时间内找到解,因此考虑了基于遗传算法的解。利用遗传算法建立了求解该问题的模型。在不同的突变过程中可以看到大多数可能性,因此我们对它们进行了更详细的研究。遗传算法的一个问题是运行模型的参数和概率,因为我们给运行的自由度越大,运行时间就越长。找到两者之间的界限是至关重要的。因此,我们的实验测量了相当大的真实数据集,以找到这个复杂问题的最佳值。由此产生的算法可以大大简化目前在我校进行的冗长的人工调度过程。
{"title":"Exploring the Potential of a Genetic Algorithm on a Real-World Complex Scheduling Problem","authors":"Szilvia Jáhn-Erdös, Bence Kövári","doi":"10.1109/ISCMI56532.2022.10068465","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068465","url":null,"abstract":"Genetic algorithms for NP-complete problems are widespread since it is easy to obtain a solution to the problem. However, its optimality gives the real issue and is not guaranteed to be achievable. In our research, we address a special subproblem of scheduling problems, the final exam scheduling, in which special requirements restrict the state space, which often contradicts each other. The task's difficulty is the massive size of the state space. Genetic algorithm-based solutions were considered since a MILP solver could not find a solution in a reasonable time. A model was built to solve this problem using the genetic algorithm. Most of the possibilities were seen in the different mutation procedures, so we investigated them in more detail. A question for genetic algorithms is what parameters and probabilities to run the model with since the more freedom we give to the run, the larger the runtime. Finding the threshold between the two is essential. Therefore, our experiments measured sizeable real data sets to find the optimal values for this complex problem. The resulting algorithm can significantly facilitate the lengthy manual scheduling processes carried out so far in our university.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128649661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Non-fungible Token Price Prediction with Multivariate LSTM Neural Networks 基于多元LSTM神经网络的不可替代代币价格预测
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068442
Jerome Branny, Rolf Dornberger, T. Hanne
In this paper, we investigate how to forecast Non-Fungible Token (NFT) sale prices by using multiple multivariate time series datasets containing features related to the NFT market space. We examined eight recent studies regarding the forecasting and valuation of NFTs and compared their most important findings. This laid the fundamental work for two separate machine learning prototypes based on Long Short-Term Memory (LSTM) which are able to forecast the sale price history of an individual NFT asset. Root Mean Squared Errors (RMSE) of 0.2975 and 0.24 were obtained which appears to be promising.
在本文中,我们研究了如何通过使用包含与NFT市场空间相关特征的多个多元时间序列数据集来预测非可替换代币(NFT)的销售价格。我们研究了最近关于nft预测和评估的八项研究,并比较了它们最重要的发现。这为两个独立的基于长短期记忆(LSTM)的机器学习原型奠定了基础,它们能够预测单个NFT资产的销售价格历史。获得的均方根误差(RMSE)为0.2975和0.24,这似乎是有希望的。
{"title":"Non-fungible Token Price Prediction with Multivariate LSTM Neural Networks","authors":"Jerome Branny, Rolf Dornberger, T. Hanne","doi":"10.1109/ISCMI56532.2022.10068442","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068442","url":null,"abstract":"In this paper, we investigate how to forecast Non-Fungible Token (NFT) sale prices by using multiple multivariate time series datasets containing features related to the NFT market space. We examined eight recent studies regarding the forecasting and valuation of NFTs and compared their most important findings. This laid the fundamental work for two separate machine learning prototypes based on Long Short-Term Memory (LSTM) which are able to forecast the sale price history of an individual NFT asset. Root Mean Squared Errors (RMSE) of 0.2975 and 0.24 were obtained which appears to be promising.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128681183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Loan Repayment Prediction Using Logistic Regression Ensemble Learning With Machine Learning Algorithms 使用逻辑回归集成学习与机器学习算法的贷款还款预测
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068483
T. Dinh, Binh Pham Thanh
Lending activities are an important part of the credit activities of financial institutions and banks. This is an area that brings great potential for development as well as a sustainable source of profit for financial institutions and banks. However, lending to customers also brings high risks. Therefore, predicting the ability to repay on time and understanding the factors affecting the repayment ability of customers is extremely important and necessary, to help financial institutions and banks enhance their ability to pay debts. customers' ability to identify and pay debts on time, contributing to minimizing bad debts and enhancing credit risk management. In this study, Machine Learning models will be used: Proposing a method to combine Logistic Regression with Random Forest, Logistic Regression with K-Nearest Neighbor, Logistic Regression with Support Vector Machine, Logistic Regression with Artificial Neural Network, Logistic Regression with Long short-term memory and finally Logistic Regression with Decision Tree to predict customers' ability to repay on time and compare and evaluate the performance of Machine Learning models. As a result, the Logistic Regression with the Random Forest model ensemble is found as the optimal predictive model and it is expected that Fico Score and annual income significantly influence the forecast.
贷款活动是金融机构和银行信贷活动的重要组成部分。这是一个具有巨大发展潜力的领域,也是金融机构和银行的可持续利润来源。然而,贷款给客户也带来了高风险。因此,预测客户的按时还款能力,了解影响客户还款能力的因素,对于帮助金融机构和银行提高偿债能力是极其重要和必要的。客户及时识别和偿还债务的能力,有助于减少坏账,加强信用风险管理。本研究将使用机器学习模型:提出将Logistic回归与随机森林、Logistic回归与k近邻、Logistic回归与支持向量机、Logistic回归与人工神经网络、Logistic回归与长短期记忆、Logistic回归与决策树相结合的方法来预测客户的按时还款能力,并比较和评估机器学习模型的性能。结果表明,随机森林模型集合的Logistic回归是最优预测模型,Fico评分和年收入对预测结果有显著影响。
{"title":"Loan Repayment Prediction Using Logistic Regression Ensemble Learning With Machine Learning Algorithms","authors":"T. Dinh, Binh Pham Thanh","doi":"10.1109/ISCMI56532.2022.10068483","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068483","url":null,"abstract":"Lending activities are an important part of the credit activities of financial institutions and banks. This is an area that brings great potential for development as well as a sustainable source of profit for financial institutions and banks. However, lending to customers also brings high risks. Therefore, predicting the ability to repay on time and understanding the factors affecting the repayment ability of customers is extremely important and necessary, to help financial institutions and banks enhance their ability to pay debts. customers' ability to identify and pay debts on time, contributing to minimizing bad debts and enhancing credit risk management. In this study, Machine Learning models will be used: Proposing a method to combine Logistic Regression with Random Forest, Logistic Regression with K-Nearest Neighbor, Logistic Regression with Support Vector Machine, Logistic Regression with Artificial Neural Network, Logistic Regression with Long short-term memory and finally Logistic Regression with Decision Tree to predict customers' ability to repay on time and compare and evaluate the performance of Machine Learning models. As a result, the Logistic Regression with the Random Forest model ensemble is found as the optimal predictive model and it is expected that Fico Score and annual income significantly influence the forecast.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133938416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Population-Based Algorithm for Solving the Fuzzy Capacitated Maximal Covering Location Problem 一种求解模糊最大覆盖定位问题的种群算法
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068459
Méziane Aïder, Imene Dey, M. Hifi
In this paper, we investigate the use of a population-based algorithm for tackling the fuzzy capacitated maximal covering location problem. Such a problem is characterized by a set of customers with their distances and its goal is to determine a subset of locations positioned on customers such that a maximum coverage of customers, including the both fuzzy coverage degree of facilities and the distance between customers, should be optimized. The proposed method is based upon the grey wolf optimizer, which starts by generating an initial population using a greedy rule strategy that is able to achieve feasible solutions according to the current positions of wolves. In order to enhance the quality of solutions induced, a series of local searches are added for exploring the search space by exploiting some nice strategies. The behavior of the method is computationally analyzed on a set of instances of the literature. Encouraging results have been provided.
在本文中,我们研究了一种基于种群的算法来解决模糊最大覆盖定位问题。该问题的特征是一组客户及其距离,其目标是确定客户位置的子集,使客户的最大覆盖范围,包括设施的模糊覆盖程度和客户之间的距离。提出的方法基于灰狼优化器,首先使用贪婪规则策略生成初始种群,该策略能够根据狼的当前位置获得可行解。为了提高归纳出的解的质量,通过利用一些好的策略,增加了一系列局部搜索来探索搜索空间。在一组文献实例上对该方法的行为进行了计算分析。已经取得了令人鼓舞的成果。
{"title":"A Population-Based Algorithm for Solving the Fuzzy Capacitated Maximal Covering Location Problem","authors":"Méziane Aïder, Imene Dey, M. Hifi","doi":"10.1109/ISCMI56532.2022.10068459","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068459","url":null,"abstract":"In this paper, we investigate the use of a population-based algorithm for tackling the fuzzy capacitated maximal covering location problem. Such a problem is characterized by a set of customers with their distances and its goal is to determine a subset of locations positioned on customers such that a maximum coverage of customers, including the both fuzzy coverage degree of facilities and the distance between customers, should be optimized. The proposed method is based upon the grey wolf optimizer, which starts by generating an initial population using a greedy rule strategy that is able to achieve feasible solutions according to the current positions of wolves. In order to enhance the quality of solutions induced, a series of local searches are added for exploring the search space by exploiting some nice strategies. The behavior of the method is computationally analyzed on a set of instances of the literature. Encouraging results have been provided.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"171 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132182983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Hadamard Estimated Attention Transformer (HEAT): Fast Approximation of Dot Product Self-attention for Transformers Using Low-Rank Projection of Hadamard Product Hadamard估计注意变压器(HEAT):利用Hadamard积的低秩投影快速逼近变压器的点积自注意
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068484
Jasper Kyle Catapang
In this paper, the author proposes a new transformer model called Hadamard Estimated Attention Transformer or HEAT, that utilizes a low-rank projection of the Hadamard product to approximate the self-attention mechanism in standard transformer architectures and thus aiming to speedup transformer training, finetuning, and inference altogether. The study shows how it is significantly better than the original transformer that uses dot product self-attention by offering a faster way to compute the original self-attention mechanism while maintaining and ultimately surpassing the quality of the original transformer architecture. It also bests Linformer and Nyströmformer in several machine translation tasks while matching and even outperforming Nyströmformer's accuracy in various text classification tasks.
在本文中,作者提出了一个新的变压器模型,称为Hadamard估计注意变压器或HEAT,它利用Hadamard乘积的低秩投影来近似标准变压器架构中的自注意机制,从而旨在加速变压器的训练,微调和推理。该研究表明,通过提供一种更快的方法来计算原始自注意机制,同时保持并最终超越原始变压器架构的质量,它如何明显优于使用点积自注意的原始变压器。它还在几个机器翻译任务中优于Linformer和Nyströmformer,而在各种文本分类任务中匹配甚至超过Nyströmformer的准确性。
{"title":"Hadamard Estimated Attention Transformer (HEAT): Fast Approximation of Dot Product Self-attention for Transformers Using Low-Rank Projection of Hadamard Product","authors":"Jasper Kyle Catapang","doi":"10.1109/ISCMI56532.2022.10068484","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068484","url":null,"abstract":"In this paper, the author proposes a new transformer model called Hadamard Estimated Attention Transformer or HEAT, that utilizes a low-rank projection of the Hadamard product to approximate the self-attention mechanism in standard transformer architectures and thus aiming to speedup transformer training, finetuning, and inference altogether. The study shows how it is significantly better than the original transformer that uses dot product self-attention by offering a faster way to compute the original self-attention mechanism while maintaining and ultimately surpassing the quality of the original transformer architecture. It also bests Linformer and Nyströmformer in several machine translation tasks while matching and even outperforming Nyströmformer's accuracy in various text classification tasks.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"316 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134507700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Comparison of Trajectory and Population-Based Algorithms for Optimizing Constrained Open-Pit Mining Problem 基于轨迹和种群的约束露天开采优化算法比较
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068481
I. Rahimi, Theodore Picard, Andrew Morabito, Kiriakos Pampalis, Aiden Abignano, A. Gandomi
The problem of open-pit mining optimization is a complex task, often containing many variables. In this paper, we apply a trajectory-based algorithm known as simulated annealing together with a well-known population-based algorithm, genetic algorithm, used to generate solutions for a formulation of the constrained pit problem (CPIT). Three datasets were used to test this simulation, Newman1, zuck_small, and KD. The results show that simulated annealing as a trajectory algorithm possesses a slightly better performance in comparison with the genetic algorithm in terms of profit value.
露天矿开采优化问题是一个复杂的问题,往往包含许多变量。在本文中,我们应用了一种基于轨迹的算法,即模拟退火算法,以及一种众所周知的基于种群的算法,即遗传算法,用于生成约束坑问题(CPIT)公式的解。三个数据集用于测试该模拟,Newman1, zuck_small和KD。结果表明,模拟退火作为一种轨迹算法,在利润值方面略优于遗传算法。
{"title":"Comparison of Trajectory and Population-Based Algorithms for Optimizing Constrained Open-Pit Mining Problem","authors":"I. Rahimi, Theodore Picard, Andrew Morabito, Kiriakos Pampalis, Aiden Abignano, A. Gandomi","doi":"10.1109/ISCMI56532.2022.10068481","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068481","url":null,"abstract":"The problem of open-pit mining optimization is a complex task, often containing many variables. In this paper, we apply a trajectory-based algorithm known as simulated annealing together with a well-known population-based algorithm, genetic algorithm, used to generate solutions for a formulation of the constrained pit problem (CPIT). Three datasets were used to test this simulation, Newman1, zuck_small, and KD. The results show that simulated annealing as a trajectory algorithm possesses a slightly better performance in comparison with the genetic algorithm in terms of profit value.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124997707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
CILDI: Class Incremental Learning with Distilled Images 用蒸馏图像进行课堂增量学习
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068463
Abel S. Zacarias, L. Alexandre
Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previously learned tasks. Learning new tasks in most proposals, implies to keeping examples of previously learned tasks to retrain the model when learning new tasks, which has an impact in terms of storage capacity. In this paper, we present a method that adds new capabilities, in an incrementally way, to an existing model keeping examples from previously learned classes but avoiding the problem of running out of storage by using distilled images to condensate sets of images into a single image. The experimental results on four data sets confirmed the effectiveness of CILDI to learn new classes incrementally across different tasks and obtaining a performance close to the state-of-the-art algorithms for class incremental learning using only one distilled image per learned class and beating the state-of-the-art on the four data sets when using 10 distilled images per learned class, while using a smaller memory footprint than the competing approaches.
终身学习旨在开发能够学习新任务的机器学习系统,同时保持以前学习任务的性能。在大多数建议中,学习新任务意味着在学习新任务时保留以前学习过的任务的示例来重新训练模型,这在存储容量方面有影响。在本文中,我们提出了一种方法,该方法以增量的方式为现有模型添加新功能,该模型保留了以前学习过的类的示例,但通过使用蒸馏图像将图像集凝聚成单个图像来避免耗尽存储的问题。在四个数据集上的实验结果证实了CILDI在不同任务中增量学习新类的有效性,并且在每个学习类只使用一个蒸馏图像的情况下获得接近最先进的类增量学习算法的性能,并且在每个学习类使用10个蒸馏图像的情况下在四个数据集上击败了最先进的算法,同时使用比竞争方法更小的内存占用。
{"title":"CILDI: Class Incremental Learning with Distilled Images","authors":"Abel S. Zacarias, L. Alexandre","doi":"10.1109/ISCMI56532.2022.10068463","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068463","url":null,"abstract":"Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previously learned tasks. Learning new tasks in most proposals, implies to keeping examples of previously learned tasks to retrain the model when learning new tasks, which has an impact in terms of storage capacity. In this paper, we present a method that adds new capabilities, in an incrementally way, to an existing model keeping examples from previously learned classes but avoiding the problem of running out of storage by using distilled images to condensate sets of images into a single image. The experimental results on four data sets confirmed the effectiveness of CILDI to learn new classes incrementally across different tasks and obtaining a performance close to the state-of-the-art algorithms for class incremental learning using only one distilled image per learned class and beating the state-of-the-art on the four data sets when using 10 distilled images per learned class, while using a smaller memory footprint than the competing approaches.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125309795","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Comparison of Linear Rank and Tournament for Parent Selection in a Genetic Algorithm Solving a Dynamic Travelling Salesman Problem 求解动态旅行商问题的遗传算法双亲选择的线性秩和比武比较
Pub Date : 2022-11-26 DOI: 10.1109/ISCMI56532.2022.10068458
Ramona Boeh, T. Hanne, Rolf Dornberger
We compare the two parent selection methods “linear rank” and “tournament” in a Genetic Algorithm applied to a dynamic Travelling Salesman Problem (TSP). The inherent dynamics of the problem is considered by temporarily doubling the costs between two randomly selected cities. In our experiments we take into account tournament selection with tournament sizes of 3, 5, and 10. A larger tournament size results in as good a performance as with linear rank selection in a small-scale dynamic TSP, whereas smaller tournament sizes better preserve the diversity of the population and avoid getting stuck in local optima. However, the assumption that tournament is superior to linear rank on a dynamic TSP could neither be confirmed nor falsified in the applied testcases.
本文比较了应用于动态旅行商问题(TSP)的遗传算法中“线性排序”和“竞赛”两种父代选择方法。将两个随机选择的城市之间的成本暂时翻倍来考虑这个问题的内在动态。在我们的实验中,我们考虑了比赛规模为3、5和10的比赛选择。在小规模动态TSP中,较大的比赛规模会产生与线性排名选择一样好的表现,而较小的比赛规模则可以更好地保持种群的多样性,避免陷入局部最优状态。然而,在应用的测试用例中,锦标赛优于动态TSP上的线性排名的假设既不能被证实也不能被证伪。
{"title":"A Comparison of Linear Rank and Tournament for Parent Selection in a Genetic Algorithm Solving a Dynamic Travelling Salesman Problem","authors":"Ramona Boeh, T. Hanne, Rolf Dornberger","doi":"10.1109/ISCMI56532.2022.10068458","DOIUrl":"https://doi.org/10.1109/ISCMI56532.2022.10068458","url":null,"abstract":"We compare the two parent selection methods “linear rank” and “tournament” in a Genetic Algorithm applied to a dynamic Travelling Salesman Problem (TSP). The inherent dynamics of the problem is considered by temporarily doubling the costs between two randomly selected cities. In our experiments we take into account tournament selection with tournament sizes of 3, 5, and 10. A larger tournament size results in as good a performance as with linear rank selection in a small-scale dynamic TSP, whereas smaller tournament sizes better preserve the diversity of the population and avoid getting stuck in local optima. However, the assumption that tournament is superior to linear rank on a dynamic TSP could neither be confirmed nor falsified in the applied testcases.","PeriodicalId":340397,"journal":{"name":"2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123899392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
2022 9th International Conference on Soft Computing & Machine Intelligence (ISCMI)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1