The identification of hepatitis C virus (HCV) virus-human protein interactions will not only help us understand the molecular mechanisms of related diseases but also be conductive to discovering new drug targets. An increasing number of clinically and experimentally validated interactions between HCV and human proteins have been documented in public databases, facilitating studies based on computational methods. In this study, we proposed a new computational approach, rotation forest position-specific scoring matrix (RF-PSSM), to predict the interactions among HCV and human proteins. In particular, PSSM was used to characterize each protein, two-dimensional principal component analysis (2DPCA) was then adopted for feature extraction of PSSM. Finally, rotation forest (RF) was used to implement classification. The results of various ablation experiments show that on independent datasets, the accuracy and area under curve (AUC) value of RF-PSSM can reach 93.74 % and 94.29%, respectively, outperforming almost all cutting-edge research. In addition, we used RF-PSSM to predict 9 human proteins that may interact with HCV protein E1, which can provide theoretical guidance for future experimental studies.
{"title":"RF-PSSM: A Combination of Rotation Forest Algorithm and Position-Specific Scoring Matrix for Improved Prediction of Protein-Protein Interactions Between Hepatitis C Virus and Human","authors":"Xin Liu;Yaping Lu;Liang Wang;Wei Geng;Xinyi Shi;Xiao Zhang","doi":"10.26599/BDMA.2022.9020031","DOIUrl":"https://doi.org/10.26599/BDMA.2022.9020031","url":null,"abstract":"The identification of hepatitis C virus (HCV) virus-human protein interactions will not only help us understand the molecular mechanisms of related diseases but also be conductive to discovering new drug targets. An increasing number of clinically and experimentally validated interactions between HCV and human proteins have been documented in public databases, facilitating studies based on computational methods. In this study, we proposed a new computational approach, rotation forest position-specific scoring matrix (RF-PSSM), to predict the interactions among HCV and human proteins. In particular, PSSM was used to characterize each protein, two-dimensional principal component analysis (2DPCA) was then adopted for feature extraction of PSSM. Finally, rotation forest (RF) was used to implement classification. The results of various ablation experiments show that on independent datasets, the accuracy and area under curve (AUC) value of RF-PSSM can reach 93.74\u0000<sup>%</sup>\u0000 and 94.29%, respectively, outperforming almost all cutting-edge research. In addition, we used RF-PSSM to predict 9 human proteins that may interact with HCV protein E1, which can provide theoretical guidance for future experimental studies.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"6 1","pages":"21-31"},"PeriodicalIF":13.6,"publicationDate":"2022-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9962810/09962955.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68007734","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.26599/BDMA.2022.9020012
Jianjun Sun;Yu Zhang
The massive growth of online commercial data has raised the request for an automatic recommender system to benefit both users and merchants. One of the most frequently used recommendation methods is collaborative filtering, but its accuracy is limited by the sparsity of the rating dataset. Most existing collaborative filtering methods consider all features when calculating user/item similarity and ignore much local information. In collaborative filtering, selecting neighbors and determining users' similarities are the most important parts. For the selection of better neighbors, this study proposes a novel biclustering method based on modified fuzzy adaptive resonance theory. To reflect the similarity between users, a new measure that considers the effect of the number of users' common items is proposed. Specifically, the proposed novel biclustering method is first adopted to obtain local similarity and local prediction. Second, item-based collaborative filtering is used to generate global predictions. Finally, the two resultant predictions are fused to obtain a final one. Experiment results demonstrate that the proposed method outperforms state-of-the-art models in terms of several aspects on three benchmark datasets.
{"title":"Recommendation System with Biclustering","authors":"Jianjun Sun;Yu Zhang","doi":"10.26599/BDMA.2022.9020012","DOIUrl":"https://doi.org/10.26599/BDMA.2022.9020012","url":null,"abstract":"The massive growth of online commercial data has raised the request for an automatic recommender system to benefit both users and merchants. One of the most frequently used recommendation methods is collaborative filtering, but its accuracy is limited by the sparsity of the rating dataset. Most existing collaborative filtering methods consider all features when calculating user/item similarity and ignore much local information. In collaborative filtering, selecting neighbors and determining users' similarities are the most important parts. For the selection of better neighbors, this study proposes a novel biclustering method based on modified fuzzy adaptive resonance theory. To reflect the similarity between users, a new measure that considers the effect of the number of users' common items is proposed. Specifically, the proposed novel biclustering method is first adopted to obtain local similarity and local prediction. Second, item-based collaborative filtering is used to generate global predictions. Finally, the two resultant predictions are fused to obtain a final one. Experiment results demonstrate that the proposed method outperforms state-of-the-art models in terms of several aspects on three benchmark datasets.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 4","pages":"282-293"},"PeriodicalIF":13.6,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9832761/09832768.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68067882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.26599/BDMA.2022.9020003
Mohamed Khalifa Boutahir;Yousef Farhaoui;Mourade Azrour;Imad Zeroual;Ahmad El Allaoui
Solar radiation is capable of producing heat, causing chemical reactions, or generating electricity. Thus, the amount of solar radiation at different times of the day must be determined to design and equip all solar systems. Moreover, it is necessary to have a thorough understanding of different solar radiation components, such as Direct Normal Irradiance (DNI), Diffuse Horizontal Irradiance (DHI), and Global Horizontal Irradiance (GHI). Unfortunately, measurements of solar radiation are not easily accessible for the majority of regions on the globe. This paper aims to develop a set of deep learning models through feature importance algorithms to predict the DNI data. The proposed models are based on historical data of meteorological parameters and solar radiation properties in a specific location of the region of Errachidia, Morocco, from January 1, 2017, to December 31, 2019, with an interval of 60 minutes. The findings demonstrated that feature selection approaches play a crucial role in forecasting of solar radiation accurately when compared with the available data.
{"title":"Effect of Feature Selection on the Prediction of Direct Normal Irradiance","authors":"Mohamed Khalifa Boutahir;Yousef Farhaoui;Mourade Azrour;Imad Zeroual;Ahmad El Allaoui","doi":"10.26599/BDMA.2022.9020003","DOIUrl":"https://doi.org/10.26599/BDMA.2022.9020003","url":null,"abstract":"Solar radiation is capable of producing heat, causing chemical reactions, or generating electricity. Thus, the amount of solar radiation at different times of the day must be determined to design and equip all solar systems. Moreover, it is necessary to have a thorough understanding of different solar radiation components, such as Direct Normal Irradiance (DNI), Diffuse Horizontal Irradiance (DHI), and Global Horizontal Irradiance (GHI). Unfortunately, measurements of solar radiation are not easily accessible for the majority of regions on the globe. This paper aims to develop a set of deep learning models through feature importance algorithms to predict the DNI data. The proposed models are based on historical data of meteorological parameters and solar radiation properties in a specific location of the region of Errachidia, Morocco, from January 1, 2017, to December 31, 2019, with an interval of 60 minutes. The findings demonstrated that feature selection approaches play a crucial role in forecasting of solar radiation accurately when compared with the available data.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 4","pages":"309-317"},"PeriodicalIF":13.6,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9832761/09832772.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68067554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Total Contents","authors":"","doi":"","DOIUrl":"https://doi.org/","url":null,"abstract":"","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 4","pages":"I-II"},"PeriodicalIF":13.6,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9832761/09832764.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68067884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.26599/BDMA.2021.9020019
Zouhaier Brahmia;Fabio Grandi;Rafik Bouaziz
Nowadays, ontologies, which are defined under the OWL 2 Web Ontology Language (OWL 2), are being used in several fields like artificial intelligence, knowledge engineering, and Semantic Web environments to access data, answer queries, or infer new knowledge. In particular, ontologies can be used to model the semantics of big data as an enabling factor for the deployment of intelligent analytics. Big data are being widely stored and exchanged in JavaScript Object Notation (JSON) format, in particular by Web applications. However, JSON data collections lack explicit semantics as they are in general schema-less, which does not allow to efficiently leverage the benefits of big data. Furthermore, several applications require bookkeeping of the entire history of big data changes, for which no support is provided by mainstream Big Data management systems, including Not only SQL (NoSQL) database systems. In this paper, we propose an approach, named $tau text{JOWL}$ (temporal OWL 2 from temporal JSON), which allows users (i) to automatically build a temporal OWL 2 ontology of data, following the Closed World Assumption (CWA), from temporal JSON-based big data, and (ii) to manage its incremental maintenance accommodating the evolution of these data, in a temporal and multi-schema environment.
如今，在OWL2 Web本体语言（OWL2）下定义的本体正被用于人工智能、知识工程和语义Web环境等多个领域，以访问数据、回答查询或推断新知识。特别是，本体可以用于对大数据的语义进行建模，作为部署智能分析的一个有利因素。大数据以JavaScript Object Notation（JSON）格式被广泛存储和交换，尤其是通过Web应用程序。然而，JSON数据集合缺乏明确的语义，因为它们通常没有模式，这不允许有效地利用大数据的好处。此外，一些应用程序需要对大数据变化的整个历史进行记账，而主流大数据管理系统（包括Not only SQL（NoSQL）数据库系统）对此没有提供支持。在本文中，我们提出了一种名为$tautext｛JOWL｝$（时态JSON中的时态OWL2）的方法，该方法允许用户（i）根据封闭世界假设（CWA），从基于时态JSON的大数据中自动构建时态OWL2数据本体，以及（ii）在时态和多模式环境中管理其增量维护，以适应这些数据的演变。
{"title":"$tautext{JOWL}$: A Systematic Approach to Build and Evolve a Temporal OWL 2 Ontology Based on Temporal JSON Big Data","authors":"Zouhaier Brahmia;Fabio Grandi;Rafik Bouaziz","doi":"10.26599/BDMA.2021.9020019","DOIUrl":"https://doi.org/10.26599/BDMA.2021.9020019","url":null,"abstract":"Nowadays, ontologies, which are defined under the OWL 2 Web Ontology Language (OWL 2), are being used in several fields like artificial intelligence, knowledge engineering, and Semantic Web environments to access data, answer queries, or infer new knowledge. In particular, ontologies can be used to model the semantics of big data as an enabling factor for the deployment of intelligent analytics. Big data are being widely stored and exchanged in JavaScript Object Notation (JSON) format, in particular by Web applications. However, JSON data collections lack explicit semantics as they are in general schema-less, which does not allow to efficiently leverage the benefits of big data. Furthermore, several applications require bookkeeping of the entire history of big data changes, for which no support is provided by mainstream Big Data management systems, including Not only SQL (NoSQL) database systems. In this paper, we propose an approach, named \u0000<tex>$tau text{JOWL}$</tex>\u0000 (temporal OWL 2 from temporal JSON), which allows users (i) to automatically build a temporal OWL 2 ontology of data, following the Closed World Assumption (CWA), from temporal JSON-based big data, and (ii) to manage its incremental maintenance accommodating the evolution of these data, in a temporal and multi-schema environment.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 4","pages":"271-281"},"PeriodicalIF":13.6,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9832761/09832769.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68067551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.26599/BDMA.2021.9020031
Mohammed Moutaib;Tarik Ahajjam;Mohammed Fattah;Yousef Farhaoui;Badraddine Aghoutane;Moulhime El Bekkali
The Internet of Things (IoT) is currently reflected in the increase in the number of connected objects, that is, devices with their own identity and computing and communication capacities. IoT is recognized as one of the most critical areas for future technologies, gaining worldwide attention. It applies to many areas, where it has achieved success, such as healthcare, where a patient is monitored using nodes and lightweight sensors. However, the powerful functions of IoT in the medical field are based on communication, analysis, processing, and management of data autonomously without any manual intervention, which presents many difficulties, such as energy consumption. However, these issues significantly slow down the development and rapid deployment of this technology. The main causes of wasted energy from connected objects include collisions that occur when two or more nodes send data simultaneously and the leading cause of data retransmission that occurs when a collision occurs or when data are not received correctly due to channel fading. The distance between nodes is one of the factors influencing energy consumption. In this article, we have proposed direct communication between nodes to avoid collision domains, which will help reduce data retransmission. The results show that the distribution can ensure the performance of the system under general conditions compared to the centralization and to the existing works.
{"title":"Application of Internet of Things in the Health Sector: Toward Minimizing Energy Consumption","authors":"Mohammed Moutaib;Tarik Ahajjam;Mohammed Fattah;Yousef Farhaoui;Badraddine Aghoutane;Moulhime El Bekkali","doi":"10.26599/BDMA.2021.9020031","DOIUrl":"https://doi.org/10.26599/BDMA.2021.9020031","url":null,"abstract":"The Internet of Things (IoT) is currently reflected in the increase in the number of connected objects, that is, devices with their own identity and computing and communication capacities. IoT is recognized as one of the most critical areas for future technologies, gaining worldwide attention. It applies to many areas, where it has achieved success, such as healthcare, where a patient is monitored using nodes and lightweight sensors. However, the powerful functions of IoT in the medical field are based on communication, analysis, processing, and management of data autonomously without any manual intervention, which presents many difficulties, such as energy consumption. However, these issues significantly slow down the development and rapid deployment of this technology. The main causes of wasted energy from connected objects include collisions that occur when two or more nodes send data simultaneously and the leading cause of data retransmission that occurs when a collision occurs or when data are not received correctly due to channel fading. The distance between nodes is one of the factors influencing energy consumption. In this article, we have proposed direct communication between nodes to avoid collision domains, which will help reduce data retransmission. The results show that the distribution can ensure the performance of the system under general conditions compared to the centralization and to the existing works.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 4","pages":"302-308"},"PeriodicalIF":13.6,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9832761/09832765.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68067553","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Artificial Intelligence (AI) is based on algorithms that allow machines to make decisions for humans. This technology enhances the users' experience in various ways. Several studies have been conducted in the field of education to solve the problem of student orientation and performance using various Machine Learning (ML) algorithms. The main goal of this article is to predict Moroccan students' performance in the region of Guelmim Oued Noun using an intelligent system based on neural networks, one of the best data mining techniques that provided us with the best results.
{"title":"Predicting Students' Final Performance Using Artificial Neural Networks","authors":"Tarik Ahajjam;Mohammed Moutaib;Haidar Aissa;Mourad Azrour;Yousef Farhaoui;Mohammed Fattah","doi":"10.26599/BDMA.2021.9020030","DOIUrl":"https://doi.org/10.26599/BDMA.2021.9020030","url":null,"abstract":"Artificial Intelligence (AI) is based on algorithms that allow machines to make decisions for humans. This technology enhances the users' experience in various ways. Several studies have been conducted in the field of education to solve the problem of student orientation and performance using various Machine Learning (ML) algorithms. The main goal of this article is to predict Moroccan students' performance in the region of Guelmim Oued Noun using an intelligent system based on neural networks, one of the best data mining techniques that provided us with the best results.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 4","pages":"294-301"},"PeriodicalIF":13.6,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9832761/09832763.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68067883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.26599/BDMA.2022.9020010
Tianlong Zheng;Chunli Zhang;Yueting Shi;Debao Chen;Sheng Liu
The unprecedented coronavirus disease 2019 (COVID-19) pandemic is still raging (in year 2021) in many countries worldwide. Various response strategies to study the characteristics and distributions of the virus in various regions of the world have been developed to assist in the prevention and control of this epidemic. Descriptive statistics and regression analysis on COVID-19 data from different countries were conducted in this study to compare and evaluate various regression models. Results showed that the extreme random forest regression (ERFR) model had the best performance, and factors such as population density, ozone, median age, life expectancy, and Human Development Index (HDI) were relatively influential on the spread and diffusion of COVID-19 in the ERFR model. In addition, the epidemic clustering characteristics were analyzed through the spectral clustering algorithm. The visualization results of spectral clustering showed that the geographical distribution of global COVID-19 pandemic spread formation was highly clustered, and its clustering characteristics and influencing factors also exhibited some consistency in distribution. This study aims to deepen the understanding of the international community regarding the global COVID-19 pandemic to develop measures for countries worldwide to mitigate potential large-scale outbreaks and improve the ability to respond to such public health emergencies.
{"title":"Influencing Factors and Clustering Characteristics of COVID-19: A Global Analysis","authors":"Tianlong Zheng;Chunli Zhang;Yueting Shi;Debao Chen;Sheng Liu","doi":"10.26599/BDMA.2022.9020010","DOIUrl":"https://doi.org/10.26599/BDMA.2022.9020010","url":null,"abstract":"The unprecedented coronavirus disease 2019 (COVID-19) pandemic is still raging (in year 2021) in many countries worldwide. Various response strategies to study the characteristics and distributions of the virus in various regions of the world have been developed to assist in the prevention and control of this epidemic. Descriptive statistics and regression analysis on COVID-19 data from different countries were conducted in this study to compare and evaluate various regression models. Results showed that the extreme random forest regression (ERFR) model had the best performance, and factors such as population density, ozone, median age, life expectancy, and Human Development Index (HDI) were relatively influential on the spread and diffusion of COVID-19 in the ERFR model. In addition, the epidemic clustering characteristics were analyzed through the spectral clustering algorithm. The visualization results of spectral clustering showed that the geographical distribution of global COVID-19 pandemic spread formation was highly clustered, and its clustering characteristics and influencing factors also exhibited some consistency in distribution. This study aims to deepen the understanding of the international community regarding the global COVID-19 pandemic to develop measures for countries worldwide to mitigate potential large-scale outbreaks and improve the ability to respond to such public health emergencies.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 4","pages":"318-338"},"PeriodicalIF":13.6,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9832761/09832767.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68068250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-09DOI: 10.26599/BDMA.2021.9020032
Samin Poudel;Marwan Bikdash
Dropping fractions of users or items judiciously can reduce the computational cost of Collaborative Filtering (CF) algorithms. The effect of this subsampling on the computing time and accuracy of CF is not fully understood, and clear guidelines for selecting optimal or even appropriate subsampling levels are not available. In this paper, we present a Density-based Random Stratified Subsampling using Clustering (DRSC) algorithm in which the desired Fraction of Users Dropped (FUD) and Fraction of Items Dropped (FID) are specified, and the overall density during subsampling is maintained. Subsequently, we develop simple models of the Training Time Improvement (TTI) and the Accuracy Loss (AL) as functions of FUD and FID, based on extensive simulations of seven standard CF algorithms as applied to various primary matrices from MovieLens, Yahoo Music Rating, and Amazon Automotive data. Simulations show that both TTI and a scaled AL are bi-linear in FID and FUD for all seven methods. The TTI linear regression of a CF method appears to be same for all datasets. Extensive simulations illustrate that TTI can be estimated reliably with FUD and FID only, but AL requires considering additional dataset characteristics. The derived models are then used to optimize the levels of subsampling addressing the tradeoff between TTI and AL. A simple sub-optimal approximation was found, in which the optimal AL is proportional to the optimal Training Time Reduction Factor (TTRF) for higher values of TTRF, and the optimal subsampling levels, like optimal FID/(1–FID), are proportional to the square root of TTRF.
明智地丢弃部分用户或项目可以降低协同过滤（CF）算法的计算成本。这种二次采样对CF的计算时间和精度的影响尚不完全清楚，也没有选择最佳甚至适当的二次采样水平的明确指南。在本文中，我们提出了一种使用聚类的基于密度的随机分层子采样（DRSC）算法，其中指定了所需的丢弃用户分数（FUD）和丢弃项目分数（FID），并保持子采样期间的总体密度。随后，我们基于对应用于MovieLens、Yahoo Music Rating和Amazon Automotive数据中的各种主矩阵的七种标准CF算法的广泛模拟，开发了作为FUD和FID函数的训练时间改进（TTI）和精度损失（AL）的简单模型。仿真表明，对于所有七种方法，在FID和FUD中，TTI和缩放的AL都是双线性的。CF方法的TTI线性回归似乎对所有数据集都是相同的。大量模拟表明，仅使用FUD和FID就可以可靠地估计TTI，但AL需要考虑额外的数据集特征。然后，将导出的模型用于优化子采样水平，以解决TTI和AL之间的折衷问题。找到了一个简单的次优近似，其中，对于较高的TTRF值，最佳AL与最佳训练时间缩减因子（TTRF）成比例，而最佳子采样水平（如最佳FID/（1–FID））与TTRF的平方根成比例。
{"title":"Optimal Dependence of Performance and Efficiency of Collaborative Filtering on Random Stratified Subsampling","authors":"Samin Poudel;Marwan Bikdash","doi":"10.26599/BDMA.2021.9020032","DOIUrl":"https://doi.org/10.26599/BDMA.2021.9020032","url":null,"abstract":"Dropping fractions of users or items judiciously can reduce the computational cost of Collaborative Filtering (CF) algorithms. The effect of this subsampling on the computing time and accuracy of CF is not fully understood, and clear guidelines for selecting optimal or even appropriate subsampling levels are not available. In this paper, we present a Density-based Random Stratified Subsampling using Clustering (DRSC) algorithm in which the desired Fraction of Users Dropped (FUD) and Fraction of Items Dropped (FID) are specified, and the overall density during subsampling is maintained. Subsequently, we develop simple models of the Training Time Improvement (TTI) and the Accuracy Loss (AL) as functions of FUD and FID, based on extensive simulations of seven standard CF algorithms as applied to various primary matrices from MovieLens, Yahoo Music Rating, and Amazon Automotive data. Simulations show that both TTI and a scaled AL are bi-linear in FID and FUD for all seven methods. The TTI linear regression of a CF method appears to be same for all datasets. Extensive simulations illustrate that TTI can be estimated reliably with FUD and FID only, but AL requires considering additional dataset characteristics. The derived models are then used to optimize the levels of subsampling addressing the tradeoff between TTI and AL. A simple sub-optimal approximation was found, in which the optimal AL is proportional to the optimal Training Time Reduction Factor (TTRF) for higher values of TTRF, and the optimal subsampling levels, like optimal FID/(1–FID), are proportional to the square root of TTRF.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 3","pages":"192-205"},"PeriodicalIF":13.6,"publicationDate":"2022-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9793354/09793360.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67848636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we consider the problem of “end-to-end” digital camera identification by considering sequence of images obtained from the cameras. The problem of digital camera identification is harder than the problem of identifying its analog counterpart since the process of analog to digital conversion smooths out the intrinsic noise in the analog signal. However it is known that identifying a digital camera is possible by analyzing the camera's intrinsic sensor artifacts that are introduced into the images/videos during the process of photo/video capture. It is known that such methods are computationally intensive requiring expensive pre-processing steps. In this paper we propose an end-to-end deep feature learning framework for identifying cameras using images obtained from them. We conduct experiments using three custom datasets: the first containing two cameras in an indoor environment where each camera may observe different scenes having no overlapping features, the second containing images from four cameras in an outdoor setting but where each camera observes scenes having overlapping features and the third containing images from two cameras observing the same checkerboard pattern in an indoor setting. Our results show that it is possible to capture the intrinsic hardware signature of the cameras using deep feature representations in an end-to-end framework. These deep feature maps can in turn be used to disambiguate the cameras from each another. Our system is end-to-end, requires no complicated pre-processing steps and the trained model is computationally efficient during testing, paving a way to have near instantaneous decisions for the problem of digital camera identification in production environments. Finally we present comparisons against the current state-of-the-art in digital camera identification which clearly establishes the superiority of the end-to-end solution.
{"title":"Deep Feature Learning for Intrinsic Signature Based Camera Discrimination","authors":"Chaity Banerjee;Tharun Kumar Doppalapudi;Eduardo Pasiliao;Tathagata Mukherjee","doi":"10.26599/BDMA.2022.9020006","DOIUrl":"https://doi.org/10.26599/BDMA.2022.9020006","url":null,"abstract":"In this paper we consider the problem of “end-to-end” digital camera identification by considering sequence of images obtained from the cameras. The problem of digital camera identification is harder than the problem of identifying its analog counterpart since the process of analog to digital conversion smooths out the intrinsic noise in the analog signal. However it is known that identifying a digital camera is possible by analyzing the camera's intrinsic sensor artifacts that are introduced into the images/videos during the process of photo/video capture. It is known that such methods are computationally intensive requiring expensive pre-processing steps. In this paper we propose an end-to-end deep feature learning framework for identifying cameras using images obtained from them. We conduct experiments using three custom datasets: the first containing two cameras in an indoor environment where each camera may observe different scenes having no overlapping features, the second containing images from four cameras in an outdoor setting but where each camera observes scenes having overlapping features and the third containing images from two cameras observing the same checkerboard pattern in an indoor setting. Our results show that it is possible to capture the intrinsic hardware signature of the cameras using deep feature representations in an end-to-end framework. These deep feature maps can in turn be used to disambiguate the cameras from each another. Our system is end-to-end, requires no complicated pre-processing steps and the trained model is computationally efficient during testing, paving a way to have near instantaneous decisions for the problem of digital camera identification in production environments. Finally we present comparisons against the current state-of-the-art in digital camera identification which clearly establishes the superiority of the end-to-end solution.","PeriodicalId":52355,"journal":{"name":"Big Data Mining and Analytics","volume":"5 3","pages":"206-227"},"PeriodicalIF":13.6,"publicationDate":"2022-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/iel7/8254253/9793354/09793358.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68010339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：