Fernando Camarena, M. González-Mendoza, Leonardo Chang, N. Hernández-Gress
Deep learning architectures lead the state-of-the-art in several computer vision, natural language processing, and reinforcement learning tasks due to their ability to extract multi-level representations without human engineering. The model’s performance is affected by the amount of labeled data used in training. Hence, novel approaches like self-supervised learning (SSL) extract the supervisory signal using unlabeled data. Although SSL reduces the dependency on human annotations, there are still two main drawbacks. First, high-computational resources are required to train a large-scale model from scratch. Second, knowledge from an SSL model is commonly finetuning to a target model, which forces them to share the same parameters and architecture and make it task-dependent. This paper explores how SSL benefits from knowledge distillation in constructing an efficient and non-task-dependent training framework. The experimental design compared the training process of an SSL algorithm trained from scratch and boosted by knowledge distillation in a teacher-student paradigm using the video-based human action recognition dataset UCF101. Results show that knowledge distillation accelerates the convergence of a network and removes the reliance on model architectures.
{"title":"Boosting Self-supervised Video-based Human Action Recognition Through Knowledge Distillation","authors":"Fernando Camarena, M. González-Mendoza, Leonardo Chang, N. Hernández-Gress","doi":"10.52591/lxai202211286","DOIUrl":"https://doi.org/10.52591/lxai202211286","url":null,"abstract":"Deep learning architectures lead the state-of-the-art in several computer vision, natural language processing, and reinforcement learning tasks due to their ability to extract multi-level representations without human engineering. The model’s performance is affected by the amount of labeled data used in training. Hence, novel approaches like self-supervised learning (SSL) extract the supervisory signal using unlabeled data. Although SSL reduces the dependency on human annotations, there are still two main drawbacks. First, high-computational resources are required to train a large-scale model from scratch. Second, knowledge from an SSL model is commonly finetuning to a target model, which forces them to share the same parameters and architecture and make it task-dependent. This paper explores how SSL benefits from knowledge distillation in constructing an efficient and non-task-dependent training framework. The experimental design compared the training process of an SSL algorithm trained from scratch and boosted by knowledge distillation in a teacher-student paradigm using the video-based human action recognition dataset UCF101. Results show that knowledge distillation accelerates the convergence of a network and removes the reliance on model architectures.","PeriodicalId":266286,"journal":{"name":"LatinX in AI at Neural Information Processing Systems Conference 2022","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129554714","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The prediction of Electronic Stopping Power for general ions and targets is a problem that lacks a closed-form solution. While full approximate solutions from first principles exist for certain cases, the most general model in use is a pseudo-empirical model. This paper presents our advances towards creating predictive models that leverage state-of-the-art Machine Learning techniques. A key component of our approach is the training data selection. We show results that outperform or are on par with the current best pseudo-empirical Stopping Power model as quantified by the Mean Absolute Percentage Error metric.
{"title":"Towards a Machine Learning Prediction of Electronic Stopping Power","authors":"Felipe Bivort","doi":"10.52591/lxai202211281","DOIUrl":"https://doi.org/10.52591/lxai202211281","url":null,"abstract":"The prediction of Electronic Stopping Power for general ions and targets is a problem that lacks a closed-form solution. While full approximate solutions from first principles exist for certain cases, the most general model in use is a pseudo-empirical model. This paper presents our advances towards creating predictive models that leverage state-of-the-art Machine Learning techniques. A key component of our approach is the training data selection. We show results that outperform or are on par with the current best pseudo-empirical Stopping Power model as quantified by the Mean Absolute Percentage Error metric.","PeriodicalId":266286,"journal":{"name":"LatinX in AI at Neural Information Processing Systems Conference 2022","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121325852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. H. Victor, C. Barberan, Richard Baraniuk, Jane Grande-Allen
Heart valves consist of leaflets that can degrade due to a range of disease processes. To better design prostheses, it is critical to study leaflet mechanics. Although mechanical testing of heart valve leaflets (HVLs) is the standard for evaluating mechanical behavior, imaging and deep learning (DL) networks, such as convolutional neural networks (CNNs), are more readily available and cost-effective. In this work, we determined the influence that a dataset that we curated had on the ability of a CNN to predict the stress-strain response of the leaflets. Our findings indicate that CNNs can be used to predict the polynomial coefficients needed for reconstructing the toe and linear regions of typically observed mechanical behavior, which lie near the physiological strain, 10% strain.
{"title":"Using Deep Learning and Macroscopic Imaging of Porcine Heart Valve Leaflets to Predict Uniaxial Stress-Strain Responses","authors":"L. H. Victor, C. Barberan, Richard Baraniuk, Jane Grande-Allen","doi":"10.52591/lxai2022112812","DOIUrl":"https://doi.org/10.52591/lxai2022112812","url":null,"abstract":"Heart valves consist of leaflets that can degrade due to a range of disease processes. To better design prostheses, it is critical to study leaflet mechanics. Although mechanical testing of heart valve leaflets (HVLs) is the standard for evaluating mechanical behavior, imaging and deep learning (DL) networks, such as convolutional neural networks (CNNs), are more readily available and cost-effective. In this work, we determined the influence that a dataset that we curated had on the ability of a CNN to predict the stress-strain response of the leaflets. Our findings indicate that CNNs can be used to predict the polynomial coefficients needed for reconstructing the toe and linear regions of typically observed mechanical behavior, which lie near the physiological strain, 10% strain.","PeriodicalId":266286,"journal":{"name":"LatinX in AI at Neural Information Processing Systems Conference 2022","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125233241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Oscar Bustos-Brinez, Joseph A. Gallego-Mejia, Fabio Gonzalez
This paper presents a novel anomaly detection method, called AD-DMKDE, based on the use of Kernel Density Estimation (KDE) along with density matrices (a powerful mathematical formalism from quantum mechanics) and Fourier features. The proposed method was systematically compared with eleven state-of-the-art anomaly detection methods on various data sets, and AD-DMKDE shows competitive performance. The method uses neural-network optimization to find the parameters of data embedding, and the prediction phase complexity of the proposed algorithm is constant relative to the training data size.
{"title":"Anomaly Detection through Density Matrices and Kernel Density Estimation (AD-DMKDE)","authors":"Oscar Bustos-Brinez, Joseph A. Gallego-Mejia, Fabio Gonzalez","doi":"10.52591/lxai2022112810","DOIUrl":"https://doi.org/10.52591/lxai2022112810","url":null,"abstract":"This paper presents a novel anomaly detection method, called AD-DMKDE, based on the use of Kernel Density Estimation (KDE) along with density matrices (a powerful mathematical formalism from quantum mechanics) and Fourier features. The proposed method was systematically compared with eleven state-of-the-art anomaly detection methods on various data sets, and AD-DMKDE shows competitive performance. The method uses neural-network optimization to find the parameters of data embedding, and the prediction phase complexity of the proposed algorithm is constant relative to the training data size.","PeriodicalId":266286,"journal":{"name":"LatinX in AI at Neural Information Processing Systems Conference 2022","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131711996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Low Surface Brightness Galaxies (LSBGs) constitute an important segment of the galaxy population, however, due to their diffuse nature, their search is challenging. The detection of LSBGs is usually done with a combination of parametric methods and visual inspection, which becomes unfeasible for future astronomical surveys that will collect petabytes of data. Thus, in this work we explore the usage of Locality-Sensitive Hashing for the approximate similarity search of LSBGs in large astronomical catalogs. We use 11670190 objects from the Dark Energy Survey Y3 Gold coadd catalog to create an approximate k nearest neighbors model based on the properties of the objects, developing a tool able to find new LSBG candidates while using only one known LSBG. From just one labeled examplewe are able to find various known LSBGs and many objects visually similar to LSBGs but not yet catalogued. Also, due to the generality of similarity search models, we are able to search for and recover other rare astronomical objects without the need of retraining or generating a large sample. Our code is available on https://github.com/zysymu/lsh-astro.
{"title":"Similarity Search of Low Surface Brightness Galaxies in Large Astronomical Catalogs","authors":"Marcos Tidball, C. Furlanetto","doi":"10.52591/lxai202211282","DOIUrl":"https://doi.org/10.52591/lxai202211282","url":null,"abstract":"Low Surface Brightness Galaxies (LSBGs) constitute an important segment of the galaxy population, however, due to their diffuse nature, their search is challenging. The detection of LSBGs is usually done with a combination of parametric methods and visual inspection, which becomes unfeasible for future astronomical surveys that will collect petabytes of data. Thus, in this work we explore the usage of Locality-Sensitive Hashing for the approximate similarity search of LSBGs in large astronomical catalogs. We use 11670190 objects from the Dark Energy Survey Y3 Gold coadd catalog to create an approximate k nearest neighbors model based on the properties of the objects, developing a tool able to find new LSBG candidates while using only one known LSBG. From just one labeled examplewe are able to find various known LSBGs and many objects visually similar to LSBGs but not yet catalogued. Also, due to the generality of similarity search models, we are able to search for and recover other rare astronomical objects without the need of retraining or generating a large sample. Our code is available on https://github.com/zysymu/lsh-astro.","PeriodicalId":266286,"journal":{"name":"LatinX in AI at Neural Information Processing Systems Conference 2022","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114768453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recently, with the advancement of deep learning, several applications in text classification have advanced significantly. However, this improvement comes with a cost because deep learning is vulnerable to adversarial examples. This weakness indicates that deep learning is not very robust. Fortunately, the input of a text classifier is discrete. Hence, it can prevent the classifier from state-of-the-art attacks. Nonetheless, previous works have generated black-box attacks that successfully manipulate the discrete values of the input to find adversarial examples. Therefore, instead of changing the discrete values, we transform the input into its embedding vector containing real values to perform the state-of-the-art white-box attacks. Then, we convert the perturbed embedding vector back into a text and name it an adversarial example. In summary, we create a framework that measures the robustness of a text classifier by using the gradients of the classifier.
{"title":"On Adversarial Examples for Text Classification By Perturbing Latent Representations","authors":"Korn Sooksatra, Pablo Rivas, Bikram Khanal","doi":"10.52591/lxai202211284","DOIUrl":"https://doi.org/10.52591/lxai202211284","url":null,"abstract":"Recently, with the advancement of deep learning, several applications in text classification have advanced significantly. However, this improvement comes with a cost because deep learning is vulnerable to adversarial examples. This weakness indicates that deep learning is not very robust. Fortunately, the input of a text classifier is discrete. Hence, it can prevent the classifier from state-of-the-art attacks. Nonetheless, previous works have generated black-box attacks that successfully manipulate the discrete values of the input to find adversarial examples. Therefore, instead of changing the discrete values, we transform the input into its embedding vector containing real values to perform the state-of-the-art white-box attacks. Then, we convert the perturbed embedding vector back into a text and name it an adversarial example. In summary, we create a framework that measures the robustness of a text classifier by using the gradients of the classifier.","PeriodicalId":266286,"journal":{"name":"LatinX in AI at Neural Information Processing Systems Conference 2022","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130945844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Discriminating fine movements within the same limb using electroencephalography (EEG) signals is a current challenge to non-invasive BCIs systems due to the close spatial representations on the motor cortex area of the brain, the signal-to-noise ratio, and the stochastic nature of this kind of signals. This research presents the performance evaluation of different strategies of classification using Linear Discriminant Analysis (LDA) method and power spectral density (PSD) features for three tasks: make a fist, open the hand, and keep the anatomical position of the hand. For this, EEG signals were collected from 10 healthy subjects and evaluated with different cross-validation methods: Monte Carlo, to implement an Offline Analysis And Leave-one-out for a pseudo-online implementation. The results show that the average accuracy for classifying the start of each task is approximately 76% for offline and Pseudo-online Analysis, classifying just the start of movement is 54% and 62% respectively for same both methods and 45% for and 32% classifying between classes. Based on these results, it can be said that the implementation of a BCI based on PSD features and LDA method could work to detect the start of one of the proposed tasks, but to discriminate the movement it is necessary to implement a different strategy for increase accuracy in the classification problem.
{"title":"Classification of fine hand movements of the same limb through EEG signals.","authors":"J. Sánchez","doi":"10.52591/lxai202211285","DOIUrl":"https://doi.org/10.52591/lxai202211285","url":null,"abstract":"Discriminating fine movements within the same limb using electroencephalography (EEG) signals is a current challenge to non-invasive BCIs systems due to the close spatial representations on the motor cortex area of the brain, the signal-to-noise ratio, and the stochastic nature of this kind of signals. This research presents the performance evaluation of different strategies of classification using Linear Discriminant Analysis (LDA) method and power spectral density (PSD) features for three tasks: make a fist, open the hand, and keep the anatomical position of the hand. For this, EEG signals were collected from 10 healthy subjects and evaluated with different cross-validation methods: Monte Carlo, to implement an Offline Analysis And Leave-one-out for a pseudo-online implementation. The results show that the average accuracy for classifying the start of each task is approximately 76% for offline and Pseudo-online Analysis, classifying just the start of movement is 54% and 62% respectively for same both methods and 45% for and 32% classifying between classes. Based on these results, it can be said that the implementation of a BCI based on PSD features and LDA method could work to detect the start of one of the proposed tasks, but to discriminate the movement it is necessary to implement a different strategy for increase accuracy in the classification problem.","PeriodicalId":266286,"journal":{"name":"LatinX in AI at Neural Information Processing Systems Conference 2022","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115016561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：