Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00165
Debanjan Banerjee, Gyan Prabhat, Riyanka Bhowal
Information in the form of text can be found in abundance in the web today, which can be mined to solve multifarious problems. Customer reviews, for instance, flow in across multiple sources in thousands per day which can be leveraged to obtain several insights. Our goal is to extract cases of a rare event e.g., recall of products, allegations of ethics or, legal concerns or, threats to product-safety, etc. from this enormous amount of data. Manual identification of such cases to be reported is extremely labour-intensive as well as time-sensitive, but failure to do so can have fatal impact on the industry's overall health and dependability; missing out on even a single case may lead to huge penalties in terms of customer experience, product liability and industry reputation. In this paper, we will discuss classification through Positive and Unlabeled data, PU classification, where the only class, for which instances are available, is a rare event. In iCASSTLE, we propose a two-staged approach where Stage I leverages three unique components of text mining to procure representative training data containing instances of both classes in the right proportion, and Stage II uses results from Stage I to run a semi-supervised classification. We applied this to multiple datasets differing in nature of Product Safety as well as nature of imbalance and iCASSTLE is proven to perform better than the state-of-the-art methods for the relevant use-cases.
{"title":"iCASSTLE : Imbalanced Classification Algorithm for Semi Supervised Text Learning","authors":"Debanjan Banerjee, Gyan Prabhat, Riyanka Bhowal","doi":"10.1109/ICMLA.2018.00165","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00165","url":null,"abstract":"Information in the form of text can be found in abundance in the web today, which can be mined to solve multifarious problems. Customer reviews, for instance, flow in across multiple sources in thousands per day which can be leveraged to obtain several insights. Our goal is to extract cases of a rare event e.g., recall of products, allegations of ethics or, legal concerns or, threats to product-safety, etc. from this enormous amount of data. Manual identification of such cases to be reported is extremely labour-intensive as well as time-sensitive, but failure to do so can have fatal impact on the industry's overall health and dependability; missing out on even a single case may lead to huge penalties in terms of customer experience, product liability and industry reputation. In this paper, we will discuss classification through Positive and Unlabeled data, PU classification, where the only class, for which instances are available, is a rare event. In iCASSTLE, we propose a two-staged approach where Stage I leverages three unique components of text mining to procure representative training data containing instances of both classes in the right proportion, and Stage II uses results from Stage I to run a semi-supervised classification. We applied this to multiple datasets differing in nature of Product Safety as well as nature of imbalance and iCASSTLE is proven to perform better than the state-of-the-art methods for the relevant use-cases.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"12 1","pages":"1012-1016"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75821059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00134
Malak Abdullah, M. Hadzikadic, Samira Shaikh
Social media is growing as a communication medium where people can express online their feelings and opinions on a variety of topics in ways they rarely do in person. Detecting sentiments and emotions in text have gained considerable amount of attention in the last few years. The significant role of the Arab region in international politics and in the global economy have led to the investigation of sentiments and emotions in Arabic. This paper describes our system - SEDAT, to detect sentiments and emotions in Arabic tweets. We use word and document embeddings and a set of semantic features and apply CNN-LSTM and a fully connected neural network architectures to obtain performance results that show substantial improvements in Spearman correlation scores over the baseline models.
{"title":"SEDAT: Sentiment and Emotion Detection in Arabic Text Using CNN-LSTM Deep Learning","authors":"Malak Abdullah, M. Hadzikadic, Samira Shaikh","doi":"10.1109/ICMLA.2018.00134","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00134","url":null,"abstract":"Social media is growing as a communication medium where people can express online their feelings and opinions on a variety of topics in ways they rarely do in person. Detecting sentiments and emotions in text have gained considerable amount of attention in the last few years. The significant role of the Arab region in international politics and in the global economy have led to the investigation of sentiments and emotions in Arabic. This paper describes our system - SEDAT, to detect sentiments and emotions in Arabic tweets. We use word and document embeddings and a set of semantic features and apply CNN-LSTM and a fully connected neural network architectures to obtain performance results that show substantial improvements in Spearman correlation scores over the baseline models.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"106 1","pages":"835-840"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72927172","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Air pollution is threatening human's health since the industrial revolution, but there are not efficient ways to solve air pollution, so forecasting air quality has become an efficient measure to prevent citizens from hurting of heavy air pollution. In this paper, we proposed an advanced Seq2Seq (Sequence to Sequence) model called attention-based air quality forecasting model (ABAFM) whose RNN encoder is replaced by pure attention mechanism with position embedding. This improvement not only reduces the training time of Seq2Seq model with attention but also enhances the robustness of Seq2Seq models. We implemented ABAFM in Olympic center and Dongsi monitoring stations in Beijing to forecast PM2.5 in future 24 hours. The experimental results showed that the proposed model outperformed the related arts, especially in sudden changes.
自工业革命以来，空气污染一直威胁着人类的健康，但没有有效的方法来解决空气污染，因此空气质量预测成为防止市民遭受重污染伤害的有效措施。本文提出了一种改进的Seq2Seq (Sequence to Sequence)模型，即基于注意力的空气质量预测模型(ABAFM)，该模型将RNN编码器替换为具有位置嵌入的纯注意力机制。这种改进不仅减少了Seq2Seq模型的训练时间，而且增强了Seq2Seq模型的鲁棒性。在北京奥运中心和东四监测站实施ABAFM，预测未来24小时PM2.5。实验结果表明，该模型在突发性变化情况下的表现优于相关算法。
{"title":"An Attention-Based Air Quality Forecasting Method","authors":"Bo Liu, Shuo Yan, Jianqiang Li, Guangzhi Qu, Yong Li, Jianlei Lang, Rentao Gu","doi":"10.1109/ICMLA.2018.00115","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00115","url":null,"abstract":"Air pollution is threatening human's health since the industrial revolution, but there are not efficient ways to solve air pollution, so forecasting air quality has become an efficient measure to prevent citizens from hurting of heavy air pollution. In this paper, we proposed an advanced Seq2Seq (Sequence to Sequence) model called attention-based air quality forecasting model (ABAFM) whose RNN encoder is replaced by pure attention mechanism with position embedding. This improvement not only reduces the training time of Seq2Seq model with attention but also enhances the robustness of Seq2Seq models. We implemented ABAFM in Olympic center and Dongsi monitoring stations in Beijing to forecast PM2.5 in future 24 hours. The experimental results showed that the proposed model outperformed the related arts, especially in sudden changes.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"25 1","pages":"728-733"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74248053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00018
P. Pulkkinen, Neetesh Tiwari, Akhil Kumar, Christopher Jones
Managing risk is important to any E-commerce merchant. Various machine learning (ML) models combined with a rule set as the decision layer is a common practice to manage the risks. Unlike the ML models that can be automatically refreshed periodically based on new risk patterns, rules are generally static and rely on manual updates. To tackle that, this paper presents a data-driven and automated rule optimization method that generates multiple Pareto-optimal rule sets representing different trade-offs between business objectives. This enables business owners to make informed decisions when choosing between optimized rule sets for changing business needs and risks. Furthermore, manual work in rule management is greatly reduced. For scalability this method leverages Apache Spark and runs either on a single host or in a distributed environment in the cloud. This allows us to perform the optimization in a distributed fashion using millions of transactions, hundreds of variables and hundreds of rules during the training. The proposed method is general but we used it for optimizing real-world E-commerce (Amazon) risk rule sets. It could also be used in other fields such as finance and medicine.
{"title":"A Multi-objective Rule Optimizer with an Application to Risk Management","authors":"P. Pulkkinen, Neetesh Tiwari, Akhil Kumar, Christopher Jones","doi":"10.1109/ICMLA.2018.00018","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00018","url":null,"abstract":"Managing risk is important to any E-commerce merchant. Various machine learning (ML) models combined with a rule set as the decision layer is a common practice to manage the risks. Unlike the ML models that can be automatically refreshed periodically based on new risk patterns, rules are generally static and rely on manual updates. To tackle that, this paper presents a data-driven and automated rule optimization method that generates multiple Pareto-optimal rule sets representing different trade-offs between business objectives. This enables business owners to make informed decisions when choosing between optimized rule sets for changing business needs and risks. Furthermore, manual work in rule management is greatly reduced. For scalability this method leverages Apache Spark and runs either on a single host or in a distributed environment in the cloud. This allows us to perform the optimization in a distributed fashion using millions of transactions, hundreds of variables and hundreds of rules during the training. The proposed method is general but we used it for optimizing real-world E-commerce (Amazon) risk rule sets. It could also be used in other fields such as finance and medicine.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"22 1","pages":"66-72"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78914130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00185
S. Indrapriyadarsini, Shahrzad Mahboubi, H. Ninomiya, H. Asai
Recent studies incorporate Nesterov's accelerated gradient method for the acceleration of gradient based training. The Nesterov's Accelerated Quasi-Newton (NAQ) method has shown to drastically improve the convergence speed compared to the conventional quasi-Newton method. This paper implements NAQ for non-convex optimization on Tensorflow. Two modifications have been proposed to the original NAQ algorithm to ensure global convergence and eliminate linesearch. The performance of the proposed algorithm - mNAQ is evaluated on standard non-convex function approximation benchmark problems and microwave circuit modelling problems. The results show that the improved algorithm converges better and faster compared to first order optimizers such as AdaGrad, RMSProp, Adam, and the second order methods such as the quasi-Newton method.
{"title":"Implementation of a Modified Nesterov's Accelerated Quasi-Newton Method on Tensorflow","authors":"S. Indrapriyadarsini, Shahrzad Mahboubi, H. Ninomiya, H. Asai","doi":"10.1109/ICMLA.2018.00185","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00185","url":null,"abstract":"Recent studies incorporate Nesterov's accelerated gradient method for the acceleration of gradient based training. The Nesterov's Accelerated Quasi-Newton (NAQ) method has shown to drastically improve the convergence speed compared to the conventional quasi-Newton method. This paper implements NAQ for non-convex optimization on Tensorflow. Two modifications have been proposed to the original NAQ algorithm to ensure global convergence and eliminate linesearch. The performance of the proposed algorithm - mNAQ is evaluated on standard non-convex function approximation benchmark problems and microwave circuit modelling problems. The results show that the improved algorithm converges better and faster compared to first order optimizers such as AdaGrad, RMSProp, Adam, and the second order methods such as the quasi-Newton method.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"29 1","pages":"1147-1154"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79317093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00106
Saeed S. Alahmari, Dmitry Goldgof, L. Hall, P. Dave, H. A. Phoulady, P. Mouton
Lack of enough labeled data is a major problem in building machine learning based models when the manual annotation (labeling) is error-prone, expensive, tedious, and time-consuming. In this paper, we introduce an iterative deep learning based method to improve segmentation and counting of cells based on unbiased stereology applied to regions of interest of extended depth of field (EDF) images. This method uses an existing machine learning algorithm called the adaptive segmentation algorithm (ASA) to generate masks (verified by a user) for EDF images to train deep learning models. Then an iterative deep learning approach is used to feed newly predicted and accepted deep learning masks/images (verified by a user) to the training set of the deep learning model. The error rate in unbiased stereology count of cells on an unseen test set reduced from about 3 % to less than 1 % after 5 iterations of the iterative deep learning based unbiased stereology process.
{"title":"Iterative Deep Learning Based Unbiased Stereology with Human-in-the-Loop","authors":"Saeed S. Alahmari, Dmitry Goldgof, L. Hall, P. Dave, H. A. Phoulady, P. Mouton","doi":"10.1109/ICMLA.2018.00106","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00106","url":null,"abstract":"Lack of enough labeled data is a major problem in building machine learning based models when the manual annotation (labeling) is error-prone, expensive, tedious, and time-consuming. In this paper, we introduce an iterative deep learning based method to improve segmentation and counting of cells based on unbiased stereology applied to regions of interest of extended depth of field (EDF) images. This method uses an existing machine learning algorithm called the adaptive segmentation algorithm (ASA) to generate masks (verified by a user) for EDF images to train deep learning models. Then an iterative deep learning approach is used to feed newly predicted and accepted deep learning masks/images (verified by a user) to the training set of the deep learning model. The error rate in unbiased stereology count of cells on an unseen test set reduced from about 3 % to less than 1 % after 5 iterations of the iterative deep learning based unbiased stereology process.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"52 1","pages":"665-670"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84017968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00187
Y. Shehu, Ariel Ruiz-Garcia, V. Palade, Anne James
Fingerprints, as one of the most widely used biometric modalities, can be used to identify and distinguish between genders. Gender classification is very important in reducing the time when investigating criminal offenders and gender impersonation. In this work, we use deep Convolutional Neural Networks (CNNs) to not only classify fingerprints by gender, but also identify individual hands and fingers. Transfer learning is employed to speed up the training of the CNN. The CNN achieves an accuracy of 75.2%, 93.5%, and 76.72% for the classification of gender, hand, and fingers, respectively. These results obtained using our publicly available Sokoto Coventry Fingerprint Dataset (SOCOFing) serve as benchmark classification results on this dataset.
{"title":"Detailed Identification of Fingerprints Using Convolutional Neural Networks","authors":"Y. Shehu, Ariel Ruiz-Garcia, V. Palade, Anne James","doi":"10.1109/ICMLA.2018.00187","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00187","url":null,"abstract":"Fingerprints, as one of the most widely used biometric modalities, can be used to identify and distinguish between genders. Gender classification is very important in reducing the time when investigating criminal offenders and gender impersonation. In this work, we use deep Convolutional Neural Networks (CNNs) to not only classify fingerprints by gender, but also identify individual hands and fingers. Transfer learning is employed to speed up the training of the CNN. The CNN achieves an accuracy of 75.2%, 93.5%, and 76.72% for the classification of gender, hand, and fingers, respectively. These results obtained using our publicly available Sokoto Coventry Fingerprint Dataset (SOCOFing) serve as benchmark classification results on this dataset.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"35 1","pages":"1161-1165"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84572530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00223
Omneya Attallah, Heba Gadelkarim, M. Sharkas
Detecting and classifying fetal brain abnormalities from magnetic resonance imaging (MRI) is important, as approximately 3 in 1000 women are pregnant with a fetal of abnormal brain. Early detection of fetal brain abnormalities using machine learning techniques can improve the quality of diagnosis and treatment planning. The literature has shown that most of the work made to classify brain abnormalities in very early age is for preterm infants and neonates not fetal. However, research papers that studied fetal brain MRI images have mapped these images with the neonates MRI images to classify an abnormal behavior in newborns not fetal. In this paper, a pipeline process is proposed for fetal brain classification (FBC) which uses machine learning techniques. The main contribution of this paper is the classification of fetal brain abnormalities in early stage, before the fetal is born. The proposed algorithm is capable of detecting and classifying a variety of abnormalities from MRI images with a wide range of fetal gestational age (GA) (from 16 to 39 weeks) using a flexible and simple method with low computational cost. The novel proposed method consists of four phases; segmentation, enhancement, feature extraction and classification. The results have shown that the proposed method has an area under ROC curve (AUC) of 84%, 86%, 80% and 84.5% for, Linear discriminate analysis (LDA), support vector machine (SVM), K-nearest neighbor (KNN), and Ensemble Subspace Discriminates classifiers respectively. This shows that our proposed has successfully classified fetal brain abnormalities with images of different fetal GA. The results are promising. Future work will be done to improve classification results and increase the dataset.
{"title":"Detecting and Classifying Fetal Brain Abnormalities Using Machine Learning Techniques","authors":"Omneya Attallah, Heba Gadelkarim, M. Sharkas","doi":"10.1109/ICMLA.2018.00223","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00223","url":null,"abstract":"Detecting and classifying fetal brain abnormalities from magnetic resonance imaging (MRI) is important, as approximately 3 in 1000 women are pregnant with a fetal of abnormal brain. Early detection of fetal brain abnormalities using machine learning techniques can improve the quality of diagnosis and treatment planning. The literature has shown that most of the work made to classify brain abnormalities in very early age is for preterm infants and neonates not fetal. However, research papers that studied fetal brain MRI images have mapped these images with the neonates MRI images to classify an abnormal behavior in newborns not fetal. In this paper, a pipeline process is proposed for fetal brain classification (FBC) which uses machine learning techniques. The main contribution of this paper is the classification of fetal brain abnormalities in early stage, before the fetal is born. The proposed algorithm is capable of detecting and classifying a variety of abnormalities from MRI images with a wide range of fetal gestational age (GA) (from 16 to 39 weeks) using a flexible and simple method with low computational cost. The novel proposed method consists of four phases; segmentation, enhancement, feature extraction and classification. The results have shown that the proposed method has an area under ROC curve (AUC) of 84%, 86%, 80% and 84.5% for, Linear discriminate analysis (LDA), support vector machine (SVM), K-nearest neighbor (KNN), and Ensemble Subspace Discriminates classifiers respectively. This shows that our proposed has successfully classified fetal brain abnormalities with images of different fetal GA. The results are promising. Future work will be done to improve classification results and increase the dataset.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"1 1","pages":"1371-1376"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73147683","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00054
A. Yaguchi, Taiji Suzuki, Wataru Asano, Shuhei Nitta, Y. Sakata, A. Tanizawa
In recent years, deep neural networks (DNNs) have been applied to various machine leaning tasks, including image recognition, speech recognition, and machine translation. However, large DNN models are needed to achieve state-of-the-art performance, exceeding the capabilities of edge devices. Model reduction is thus needed for practical use. In this paper, we point out that deep learning automatically induces group sparsity of weights, in which all weights connected to an output channel (node) are zero, when training DNNs under the following three conditions: (1) rectified-linear-unit (ReLU) activations, (2) an L2-regularized objective function, and (3) the Adam optimizer. Next, we analyze this behavior both theoretically and experimentally, and propose a simple model reduction method: eliminate the zero weights after training the DNN. In experiments on MNIST and CIFAR-10 datasets, we demonstrate the sparsity with various training setups. Finally, we show that our method can efficiently reduce the model size and performs well relative to methods that use a sparsity-inducing regularizer.
{"title":"Adam Induces Implicit Weight Sparsity in Rectifier Neural Networks","authors":"A. Yaguchi, Taiji Suzuki, Wataru Asano, Shuhei Nitta, Y. Sakata, A. Tanizawa","doi":"10.1109/ICMLA.2018.00054","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00054","url":null,"abstract":"In recent years, deep neural networks (DNNs) have been applied to various machine leaning tasks, including image recognition, speech recognition, and machine translation. However, large DNN models are needed to achieve state-of-the-art performance, exceeding the capabilities of edge devices. Model reduction is thus needed for practical use. In this paper, we point out that deep learning automatically induces group sparsity of weights, in which all weights connected to an output channel (node) are zero, when training DNNs under the following three conditions: (1) rectified-linear-unit (ReLU) activations, (2) an L2-regularized objective function, and (3) the Adam optimizer. Next, we analyze this behavior both theoretically and experimentally, and propose a simple model reduction method: eliminate the zero weights after training the DNN. In experiments on MNIST and CIFAR-10 datasets, we demonstrate the sparsity with various training setups. Finally, we show that our method can efficiently reduce the model size and performs well relative to methods that use a sparsity-inducing regularizer.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"37 1","pages":"318-325"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86364857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-12-01DOI: 10.1109/ICMLA.2018.00203
M. Ogihara, Daniel Galarraga, Gang Ren, T. Tavares
Popular music lyrics are usually brief in length yet sophisticated in narrative content, emotional expression, and structural aesthetics. In this paper, we propose a graph-based analysis and interpretation framework for popular music lyrics using the sematic word embedding representation. This framework explores the temporal and structural information in music lyrics, such as word sequential pattern, lyric format pattern, and predominate song forms, to enhance the understanding of the interaction between the semantic and structural properties of music lyrics. Our proposed analysis and interpretation framework provides extensive tools for representing various properties of music lyrics as graph structural elements and then we implemented feature extraction tools for a comprehensive characterization of the lyric graph using graph analysis or complex network methodologies. The empirical studies based on contrasting music genres are then presented to illustrate the usage of the proposed tools and to demonstrate its modeling and analysis capabilities.
{"title":"The Semantic Shapes of Popular Music Lyrics: Graph-Based Representation, Analysis, and Interpretation of Popular Music Lyrics in Semantic Natural Language Embedding Space","authors":"M. Ogihara, Daniel Galarraga, Gang Ren, T. Tavares","doi":"10.1109/ICMLA.2018.00203","DOIUrl":"https://doi.org/10.1109/ICMLA.2018.00203","url":null,"abstract":"Popular music lyrics are usually brief in length yet sophisticated in narrative content, emotional expression, and structural aesthetics. In this paper, we propose a graph-based analysis and interpretation framework for popular music lyrics using the sematic word embedding representation. This framework explores the temporal and structural information in music lyrics, such as word sequential pattern, lyric format pattern, and predominate song forms, to enhance the understanding of the interaction between the semantic and structural properties of music lyrics. Our proposed analysis and interpretation framework provides extensive tools for representing various properties of music lyrics as graph structural elements and then we implemented feature extraction tools for a comprehensive characterization of the lyric graph using graph analysis or complex network methodologies. The empirical studies based on contrasting music genres are then presented to illustrate the usage of the proposed tools and to demonstrate its modeling and analysis capabilities.","PeriodicalId":6533,"journal":{"name":"2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)","volume":"132 1","pages":"1249-1254"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86605275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：