首页 > 最新文献

计算机工程与设计最新文献

英文 中文
Developing a news classifier for Greek using BERT 使用BERT开发希腊语新闻分类器
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932996
George Gkolfopoulos, Iraklis Varlamis
Text categorization is a significant task in the re-search field of text mining, which has recently benefited from deep neural network algorithms and advanced learning techniques that extract language models from large textual corpora. These Pre-Trained Language Models are the main components of state-of-the-art solutions in many natural language processing and text-mining tasks can be very generic, trained in generic text corpora, or domain-specific when they employ large corpora from specific application domains (e.g. social media, news, sciences, etc.). When only generic language models are available the overall performance in the task can be improved by adapting or fine-tuning the model used for the task, e.g. the classifier. Although multilingual language models are reported in the literature, such models are usually language-specific. This work presents a news article classifier, which has been trained on a small corpus and employs a Greek version of BERT language model. Comparison with existing machine learning-based classifiers shows that the proposed method outperforms well-known methods in text classification. In addition, the proposed approach allows the continuous training of the classifier through user-provided feedback on falsely classified articles.
文本分类是文本挖掘研究领域的一项重要任务,近年来得益于深度神经网络算法和从大型文本语料库中提取语言模型的先进学习技术。这些预训练语言模型是许多自然语言处理和文本挖掘任务中最先进的解决方案的主要组成部分,可以非常通用,在通用文本语料库中训练,或者在使用来自特定应用领域(例如社交媒体,新闻,科学等)的大型语料库时特定于领域。当只有通用语言模型可用时,可以通过调整或微调用于任务的模型(例如分类器)来提高任务中的整体性能。尽管文献中报道了多语言模型,但这些模型通常是特定于语言的。这项工作提出了一个新闻文章分类器,它已经在一个小的语料库上进行了训练,并采用了希腊版本的BERT语言模型。与现有基于机器学习的分类器的比较表明,该方法在文本分类方面优于现有的分类方法。此外,所提出的方法允许通过用户提供对错误分类文章的反馈来持续训练分类器。
{"title":"Developing a news classifier for Greek using BERT","authors":"George Gkolfopoulos, Iraklis Varlamis","doi":"10.1109/SEEDA-CECNSM57760.2022.9932996","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932996","url":null,"abstract":"Text categorization is a significant task in the re-search field of text mining, which has recently benefited from deep neural network algorithms and advanced learning techniques that extract language models from large textual corpora. These Pre-Trained Language Models are the main components of state-of-the-art solutions in many natural language processing and text-mining tasks can be very generic, trained in generic text corpora, or domain-specific when they employ large corpora from specific application domains (e.g. social media, news, sciences, etc.). When only generic language models are available the overall performance in the task can be improved by adapting or fine-tuning the model used for the task, e.g. the classifier. Although multilingual language models are reported in the literature, such models are usually language-specific. This work presents a news article classifier, which has been trained on a small corpus and employs a Greek version of BERT language model. Comparison with existing machine learning-based classifiers shows that the proposed method outperforms well-known methods in text classification. In addition, the proposed approach allows the continuous training of the classifier through user-provided feedback on falsely classified articles.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"64 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77510273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Tracking individuals’ health using mobile applications and Machine Learning 使用移动应用程序和机器学习跟踪个人健康状况
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932927
Giannis Botilias, Lamprini Pappa, P. Karvelis, C. Stylios
The widespread availability of smartphones and their high processing power have made them powerful mobile tools able to host and run various apps. In addition, wearable devices with low cost and accurate sensors gathering various physiological data and information are now available. Meanwhile, automated activity recognition is a rapidly evolving research area directly related to the mobile Health (mHealth) field. Rapid advancements in the Human Activity Recognition (HAR) field are mainly based on combining smartphones and wearable devices to succeed in advancing health tracking. This paper presents a mobile app designed and developed for monitoring changes in variables related to the physiological health status of an individual when he is moving around. The app tracks the physiological status of a human along with machine learning algorithms able to recognize and identify human activity and produce automatic alerts warning of dangerous health situations.
智能手机的广泛使用及其强大的处理能力使其成为强大的移动工具,能够托管和运行各种应用程序。此外,低成本和精确的传感器收集各种生理数据和信息的可穿戴设备现在已经可用。同时,自动活动识别是与移动健康(mHealth)领域直接相关的一个快速发展的研究领域。人体活动识别(HAR)领域的快速发展主要是基于智能手机和可穿戴设备的结合,以成功推进健康跟踪。本文介绍了一个移动应用程序的设计和开发,用于监测与个人的生理健康状态有关的变量的变化,当他四处走动。该应用程序跟踪人类的生理状态,以及能够识别和识别人类活动的机器学习算法,并对危险的健康状况发出自动警报。
{"title":"Tracking individuals’ health using mobile applications and Machine Learning","authors":"Giannis Botilias, Lamprini Pappa, P. Karvelis, C. Stylios","doi":"10.1109/SEEDA-CECNSM57760.2022.9932927","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932927","url":null,"abstract":"The widespread availability of smartphones and their high processing power have made them powerful mobile tools able to host and run various apps. In addition, wearable devices with low cost and accurate sensors gathering various physiological data and information are now available. Meanwhile, automated activity recognition is a rapidly evolving research area directly related to the mobile Health (mHealth) field. Rapid advancements in the Human Activity Recognition (HAR) field are mainly based on combining smartphones and wearable devices to succeed in advancing health tracking. This paper presents a mobile app designed and developed for monitoring changes in variables related to the physiological health status of an individual when he is moving around. The app tracks the physiological status of a human along with machine learning algorithms able to recognize and identify human activity and produce automatic alerts warning of dangerous health situations.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"1936 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87751823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Parameter Tuning of Linear Programming Solvers 线性规划求解器的参数整定
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9933002
N. Ploskas
Linear programming solvers include various options that can be used to control algorithmic aspects and considerably impact the solver performance. As it is obvious, manually finding optimal parameters is a very difficult task and sometimes impossible. For this reason, it is necessary to implement smart techniques that will automate this process. Other works have utilized derivative-free optimization solvers to tune solver parameters. In this work, eight open-source derivative-free optimization solvers are utilized for finding (near) optimal tuning parameters of state-of-the-art linear programming solvers. We investigate how sensitive linear programming solvers are to a parameter tuning process. Extensive computational results are presented on tuning four linear programming solvers (CLP, CPLEX, GUROBI, and XPRESS) over a set of 70 benchmark problems. We find better parameters for all linear programming solvers, achieving a reduction in execution time over their default parameters up to 26%. We conclude that several derivative-free optimization solvers outperform others on finding optimal optimal tuning parameters for linear programming solvers.
线性规划求解器包括各种选项,可用于控制算法方面,并大大影响求解器的性能。很明显,手动找到最佳参数是一项非常困难的任务,有时甚至是不可能的。出于这个原因,有必要实现将此过程自动化的智能技术。其他研究利用无导数优化求解器来调整求解器参数。在这项工作中,利用八个开源的无导数优化求解器来寻找最先进的线性规划求解器的(接近)最优调谐参数。我们研究线性规划解算器对参数整定过程的敏感性。在一组70个基准问题上调优四个线性规划求解器(CLP、CPLEX、GUROBI和XPRESS),给出了大量的计算结果。我们为所有线性规划求解器找到了更好的参数,与默认参数相比,执行时间减少了26%。我们得出结论,一些无导数优化解在寻找线性规划解的最优优化参数方面优于其他解。
{"title":"Parameter Tuning of Linear Programming Solvers","authors":"N. Ploskas","doi":"10.1109/SEEDA-CECNSM57760.2022.9933002","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9933002","url":null,"abstract":"Linear programming solvers include various options that can be used to control algorithmic aspects and considerably impact the solver performance. As it is obvious, manually finding optimal parameters is a very difficult task and sometimes impossible. For this reason, it is necessary to implement smart techniques that will automate this process. Other works have utilized derivative-free optimization solvers to tune solver parameters. In this work, eight open-source derivative-free optimization solvers are utilized for finding (near) optimal tuning parameters of state-of-the-art linear programming solvers. We investigate how sensitive linear programming solvers are to a parameter tuning process. Extensive computational results are presented on tuning four linear programming solvers (CLP, CPLEX, GUROBI, and XPRESS) over a set of 70 benchmark problems. We find better parameters for all linear programming solvers, achieving a reduction in execution time over their default parameters up to 26%. We conclude that several derivative-free optimization solvers outperform others on finding optimal optimal tuning parameters for linear programming solvers.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"94 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74669836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
beHost: A web application for potential accommodation providers interested in participating in the sharing economy beHost:一个面向有兴趣参与共享经济的潜在住宿供应商的web应用程序
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932936
Foteini Tatsi, Fotios Tatsis, K. Karamanis
The motives and factors influencing an individual’s decision to participate in the accommodation sharing economy differ from person to person. Profit and social interaction are highlighted as the main motives driving an individual to become a host. The number of hosts who manage more than one listing is constantly increasing. Managing more listings makes hosts more experienced in serving guests and making more profit. This paper introduces “beHost”, a web application built to guide potential first time hosts to become more competitive and achieve their goals.
影响个人参与住宿共享经济决策的动机和因素因人而异。利润和社会互动被强调为个人成为主持人的主要动机。管理多个房源的主机数量在不断增加。管理更多的房源可以让房东在服务客人方面更有经验,也能赚取更多的利润。本文介绍了“beHost”,这是一个web应用程序,旨在指导潜在的首次主机变得更具竞争力并实现他们的目标。
{"title":"beHost: A web application for potential accommodation providers interested in participating in the sharing economy","authors":"Foteini Tatsi, Fotios Tatsis, K. Karamanis","doi":"10.1109/SEEDA-CECNSM57760.2022.9932936","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932936","url":null,"abstract":"The motives and factors influencing an individual’s decision to participate in the accommodation sharing economy differ from person to person. Profit and social interaction are highlighted as the main motives driving an individual to become a host. The number of hosts who manage more than one listing is constantly increasing. Managing more listings makes hosts more experienced in serving guests and making more profit. This paper introduces “beHost”, a web application built to guide potential first time hosts to become more competitive and achieve their goals.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"6 1","pages":"1-4"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79653929","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Security and Privacy Vulnerabilities in Human Activity Recognition systems 人体活动识别系统中的安全和隐私漏洞
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932957
V. Liagkou, S. Sakka, C. Stylios
Human activity recognition systems (HARS) should allow the secure and trustworthy exchange of sensitive data between several kinds of participating parties with different aims and claims, regarding security, data protection, and trust issues. Initially in this work, a security flaw has been identified in a complete medical IoT application using wearable devices and smart sensors. Then, we list the security vulnerabilities and attempt to make suggestions on the prevention of security flaws that may appear during the implementation of HARS and we analyze a specific attack, the Man in the Middle attack, where a third malicious entity interferes with communication between two entities and is associated with key exchange protocols. Moreover, we discuss various design considerations for protecting the data that is transmitted and stored from different sources like smart wearables, mobile phones, and cloud applications by using cryptographic and privacy-preserving techniques. Finally, we show how the use of the OAuth2.0 protocol can ensure that only authenticated users interact with the HARS.
人类活动识别系统(HARS)应该允许在具有不同目标和要求的几种参与方之间安全可靠地交换敏感数据,涉及安全性、数据保护和信任问题。在这项工作的最初阶段,在使用可穿戴设备和智能传感器的完整医疗物联网应用中发现了一个安全漏洞。然后,我们列出了安全漏洞,并试图对HARS实施过程中可能出现的安全漏洞的预防提出建议,我们分析了一个具体的攻击,中间人攻击,其中第三个恶意实体干扰两个实体之间的通信,并与密钥交换协议相关联。此外,我们还讨论了通过使用加密和隐私保护技术来保护从不同来源(如智能可穿戴设备、移动电话和云应用程序)传输和存储的数据的各种设计考虑。最后,我们将展示使用OAuth2.0协议如何确保只有经过身份验证的用户才能与HARS交互。
{"title":"Security and Privacy Vulnerabilities in Human Activity Recognition systems","authors":"V. Liagkou, S. Sakka, C. Stylios","doi":"10.1109/SEEDA-CECNSM57760.2022.9932957","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932957","url":null,"abstract":"Human activity recognition systems (HARS) should allow the secure and trustworthy exchange of sensitive data between several kinds of participating parties with different aims and claims, regarding security, data protection, and trust issues. Initially in this work, a security flaw has been identified in a complete medical IoT application using wearable devices and smart sensors. Then, we list the security vulnerabilities and attempt to make suggestions on the prevention of security flaws that may appear during the implementation of HARS and we analyze a specific attack, the Man in the Middle attack, where a third malicious entity interferes with communication between two entities and is associated with key exchange protocols. Moreover, we discuss various design considerations for protecting the data that is transmitted and stored from different sources like smart wearables, mobile phones, and cloud applications by using cryptographic and privacy-preserving techniques. Finally, we show how the use of the OAuth2.0 protocol can ensure that only authenticated users interact with the HARS.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"13 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75453317","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An experimental protocol for exploration of stress in an immersive VR scenario with EEG 在一个沉浸式虚拟现实场景与脑电图探索压力的实验协议
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932987
Andreas Miltiadous, Vasileios Aspiotis, Konstantinos Sakkas, N. Giannakeas, E. Glavas, A. Tzallas
Stress is a subject always relevant to scientific research due to the numerous implications in human life. Typical biomarkers used in the physiological evaluation of stress include Electrocardiography, cortisol levels, galvanic skin response and other. Recently, one less widely used instrument for the assessment of stress that has been re-emerged due to advancements in computational power and machine learning techniques, is Electroencephalography. Moreover, as Virtual Reality HMDs are being rapidly adopted by the research community it becomes apparent that leveraging the offered advantages of VR for the exploration of stress can lead to novel controlable and reproducable experimental procedures. In this paper we combine EEG, ECG and the Perceived Stress Scale with a Virtual Reality phobia induction setting, to propose a protocol for assessing stress. The suggested protocol can be used for functional brain connectivity investigation and thus the evaluation of stress while it and can be expanded via the incorporation of machine learning algorithms for automatic stress level classification.
压力一直是科学研究的主题,因为它对人类生活有许多影响。用于应激生理评估的典型生物标志物包括心电图、皮质醇水平、皮肤电反应等。最近,由于计算能力和机器学习技术的进步,一种不太广泛使用的压力评估工具重新出现,那就是脑电图。此外,随着虚拟现实头显被研究界迅速采用,利用虚拟现实的优势来探索压力可以带来新的可控和可重复的实验过程变得很明显。在本文中,我们将脑电图、心电图和感知压力量表与虚拟现实恐惧症诱导设置相结合,提出了一种评估压力的方案。建议的协议可用于功能性脑连接调查,从而评估压力,同时可以通过结合机器学习算法进行自动压力水平分类来扩展。
{"title":"An experimental protocol for exploration of stress in an immersive VR scenario with EEG","authors":"Andreas Miltiadous, Vasileios Aspiotis, Konstantinos Sakkas, N. Giannakeas, E. Glavas, A. Tzallas","doi":"10.1109/SEEDA-CECNSM57760.2022.9932987","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932987","url":null,"abstract":"Stress is a subject always relevant to scientific research due to the numerous implications in human life. Typical biomarkers used in the physiological evaluation of stress include Electrocardiography, cortisol levels, galvanic skin response and other. Recently, one less widely used instrument for the assessment of stress that has been re-emerged due to advancements in computational power and machine learning techniques, is Electroencephalography. Moreover, as Virtual Reality HMDs are being rapidly adopted by the research community it becomes apparent that leveraging the offered advantages of VR for the exploration of stress can lead to novel controlable and reproducable experimental procedures. In this paper we combine EEG, ECG and the Perceived Stress Scale with a Virtual Reality phobia induction setting, to propose a protocol for assessing stress. The suggested protocol can be used for functional brain connectivity investigation and thus the evaluation of stress while it and can be expanded via the incorporation of machine learning algorithms for automatic stress level classification.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"49 1","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86464586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Brain MRI based diagnosis of autoimmune diseases using deep learning 基于脑MRI的自身免疫性疾病深度学习诊断
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932959
D. Amanatidis, Georgios Chatzisavvas, Michael F. Dossis
The diagnosis of an autoimmune disease usually requires a careful examination of the patient’s health history and the evaluation of any possible occupation and environment related exposures. Frequently, autoimmune disorders have early symptoms such as joint and muscle pain, fatigue, weight loss or fever. These symptoms however are non-specific and imaging technology tools can be extremely valuable for precise diagnosis. In this paper, we deal with autoimmune diseases that result in brain damage and more specifically, multiple sclerosis. Classification of brain MRI images is performed leveraging a Convolutional Neural Network, showing excellent results.
自身免疫性疾病的诊断通常需要仔细检查患者的健康史,并评估任何可能的职业和环境相关暴露。通常,自身免疫性疾病有早期症状,如关节和肌肉疼痛、疲劳、体重减轻或发烧。然而,这些症状是非特异性的,成像技术工具对于精确诊断是非常有价值的。在本文中,我们处理自身免疫性疾病,导致脑损伤,更具体地说,多发性硬化症。利用卷积神经网络对脑MRI图像进行分类,结果非常好。
{"title":"Brain MRI based diagnosis of autoimmune diseases using deep learning","authors":"D. Amanatidis, Georgios Chatzisavvas, Michael F. Dossis","doi":"10.1109/SEEDA-CECNSM57760.2022.9932959","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932959","url":null,"abstract":"The diagnosis of an autoimmune disease usually requires a careful examination of the patient’s health history and the evaluation of any possible occupation and environment related exposures. Frequently, autoimmune disorders have early symptoms such as joint and muscle pain, fatigue, weight loss or fever. These symptoms however are non-specific and imaging technology tools can be extremely valuable for precise diagnosis. In this paper, we deal with autoimmune diseases that result in brain damage and more specifically, multiple sclerosis. Classification of brain MRI images is performed leveraging a Convolutional Neural Network, showing excellent results.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"23 1","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88982936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Distributed Gibbs Sampling and LDA Modelling for Large Scale Big Data Management on PySpark 基于PySpark的大规模大数据管理的分布式Gibbs采样和LDA建模
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932990
Christos N. Karras, Aristeidis Karras, D. Tsolis, K. Giotopoulos, S. Sioutas
Big data management methods are paramount in the modern era as applications tend to create massive amounts of data that comes from various sources. Therefore, there is an urge to create adaptive, speedy and robust frameworks that can effectively handle massive datasets. Distributed environments such as Apache Spark are of note, as they can handle such data by creating clusters where a portion of the data is stored locally and then the results are returned with the use of Resilient Distributed Datasets (RDDs). In this paper a method for distributed marginal Gibbs sampling for widely used latent Dirichlet allocation (LDA) model is implemented on PySpark along with a Metropolis Hastings Random Walker. The Distributed LDA (DLDA) algorithm distributes a given dataset into P partitions and performs local LDA on each partition, for each document independently. Every nth iteration, local LDA models, that were trained on distinct partitions, are combined to assure the model ability to converge. Experimental results are promising as the proposed system demonstrates comparable performance in the final model quality to the sequential LDA, and achieves significant speedup time-optimizations when utilized with massive datasets.
大数据管理方法在现代时代是至关重要的,因为应用程序往往会创建来自各种来源的大量数据。因此,迫切需要创建能够有效处理大量数据集的自适应、快速和健壮的框架。像Apache Spark这样的分布式环境是值得注意的,因为它们可以通过创建集群来处理这些数据,其中一部分数据存储在本地,然后使用弹性分布式数据集(rdd)返回结果。本文利用Metropolis Hastings Random Walker在PySpark上实现了广泛应用的潜在狄利克雷分配(latent Dirichlet allocation, LDA)模型的分布式边际Gibbs抽样方法。分布式LDA (Distributed LDA)算法将给定的数据集分布到P个分区中,并在每个分区上独立地对每个文档执行本地LDA。在每第n次迭代中,对在不同分区上训练的局部LDA模型进行组合,以确保模型的收敛能力。实验结果是有希望的,因为所提出的系统在最终模型质量方面表现出与顺序LDA相当的性能,并且在使用大量数据集时实现了显着的加速时间优化。
{"title":"Distributed Gibbs Sampling and LDA Modelling for Large Scale Big Data Management on PySpark","authors":"Christos N. Karras, Aristeidis Karras, D. Tsolis, K. Giotopoulos, S. Sioutas","doi":"10.1109/SEEDA-CECNSM57760.2022.9932990","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932990","url":null,"abstract":"Big data management methods are paramount in the modern era as applications tend to create massive amounts of data that comes from various sources. Therefore, there is an urge to create adaptive, speedy and robust frameworks that can effectively handle massive datasets. Distributed environments such as Apache Spark are of note, as they can handle such data by creating clusters where a portion of the data is stored locally and then the results are returned with the use of Resilient Distributed Datasets (RDDs). In this paper a method for distributed marginal Gibbs sampling for widely used latent Dirichlet allocation (LDA) model is implemented on PySpark along with a Metropolis Hastings Random Walker. The Distributed LDA (DLDA) algorithm distributes a given dataset into P partitions and performs local LDA on each partition, for each document independently. Every nth iteration, local LDA models, that were trained on distinct partitions, are combined to assure the model ability to converge. Experimental results are promising as the proposed system demonstrates comparable performance in the final model quality to the sequential LDA, and achieves significant speedup time-optimizations when utilized with massive datasets.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"119 1","pages":"1-8"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88663874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Open weather data evaluation for crop irrigation prediction mechanisms in the AUGEIAS project AUGEIAS项目作物灌溉预测机制的开放气象数据评价
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932913
Thomai Karamitsou, Dimitrios Seventekidis, Christos Karapiperis, Konstantina Banti, Ioanna Karampelia, Thomas S. Kyriakidis, M. Louta
Treated wastewater reuse is increasingly important for efficient and sustainable management of water resources due to increased water demands. Motivated by the above, AUGEIAS proposes an Internet of Things (IoT) approach for clean and treated wastewater usage in precision agriculture. In this context, real-time measurements for wastewater treatment plant and field are correlated with open data to improve crop water needs prediction mechanisms. This paper presents the open weather sources that are used and evaluates their reliability. After the open data is evaluated, it is integrated with the data collected by IoT sensors/devices. By using the mean absolute percentage error metric, we evaluate the forecasting performance of open weather sources. According to our study, OpenWeatherMap’s forecast data proved more accurate, with a success rate at 83.3%.
由于用水需求的增加,处理过的废水回用对水资源的有效和可持续管理越来越重要。基于上述动机,AUGEIAS提出了一种物联网(IoT)方法,用于精准农业中清洁和处理过的废水的使用。在这种情况下,污水处理厂和农田的实时测量与开放数据相关联,以改进作物需水量预测机制。本文介绍了所使用的开放天气源,并对其可靠性进行了评估。在评估开放数据后,将其与物联网传感器/设备收集的数据集成。通过使用平均绝对百分比误差度量,我们评估了开放天气源的预报性能。根据我们的研究,OpenWeatherMap的预报数据被证明更加准确,准确率为83.3%。
{"title":"Open weather data evaluation for crop irrigation prediction mechanisms in the AUGEIAS project","authors":"Thomai Karamitsou, Dimitrios Seventekidis, Christos Karapiperis, Konstantina Banti, Ioanna Karampelia, Thomas S. Kyriakidis, M. Louta","doi":"10.1109/SEEDA-CECNSM57760.2022.9932913","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932913","url":null,"abstract":"Treated wastewater reuse is increasingly important for efficient and sustainable management of water resources due to increased water demands. Motivated by the above, AUGEIAS proposes an Internet of Things (IoT) approach for clean and treated wastewater usage in precision agriculture. In this context, real-time measurements for wastewater treatment plant and field are correlated with open data to improve crop water needs prediction mechanisms. This paper presents the open weather sources that are used and evaluates their reliability. After the open data is evaluated, it is integrated with the data collected by IoT sensors/devices. By using the mean absolute percentage error metric, we evaluate the forecasting performance of open weather sources. According to our study, OpenWeatherMap’s forecast data proved more accurate, with a success rate at 83.3%.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":" 7","pages":"1-4"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72385158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Object Detection for Low Light Images 低光图像的目标检测
Pub Date : 2022-09-23 DOI: 10.1109/SEEDA-CECNSM57760.2022.9932921
Dimitrios Mpouziotas, Eleftherios Mastrapas, Nikos Dimokas, Petros Karvelis, E. Glavas
Object detection is a computer vision method for locating objects in images. Although, it has surpassed human performance and it has been considered practically solved, there are still considerable challenges, such as when photos are captured under suboptimal lighting conditions due to environmental and/or technical constraints. On the other hand, a variety of methods have been developed to enhance low light images, which can boost an object detector’s performance. In this work, we apply different image enhancement methods and study how they affect the efficacy of a well known detector (You Only Look Once, YOLO). A statistical analysis between YOLO’s performance for each enhancing algorithm, using a low light imaging dataset, is also presented, proving that for these kind of images, enhancement is a valuable step.
目标检测是一种定位图像中目标的计算机视觉方法。虽然,它已经超越了人类的表现,并被认为实际上已经解决了,但仍然存在相当大的挑战,例如由于环境和/或技术限制,当照片在次优照明条件下拍摄时。另一方面,已经开发了各种方法来增强低光图像,这可以提高目标检测器的性能。在这项工作中,我们应用了不同的图像增强方法,并研究了它们如何影响一个众所周知的检测器(You Only Look Once, YOLO)的有效性。在微光成像数据集上,对不同增强算法的YOLO性能进行了统计分析,证明了对这类图像进行增强是有价值的一步。
{"title":"Object Detection for Low Light Images","authors":"Dimitrios Mpouziotas, Eleftherios Mastrapas, Nikos Dimokas, Petros Karvelis, E. Glavas","doi":"10.1109/SEEDA-CECNSM57760.2022.9932921","DOIUrl":"https://doi.org/10.1109/SEEDA-CECNSM57760.2022.9932921","url":null,"abstract":"Object detection is a computer vision method for locating objects in images. Although, it has surpassed human performance and it has been considered practically solved, there are still considerable challenges, such as when photos are captured under suboptimal lighting conditions due to environmental and/or technical constraints. On the other hand, a variety of methods have been developed to enhance low light images, which can boost an object detector’s performance. In this work, we apply different image enhancement methods and study how they affect the efficacy of a well known detector (You Only Look Once, YOLO). A statistical analysis between YOLO’s performance for each enhancing algorithm, using a low light imaging dataset, is also presented, proving that for these kind of images, enhancement is a valuable step.","PeriodicalId":68279,"journal":{"name":"计算机工程与设计","volume":"1 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2022-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83490922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
计算机工程与设计
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1