首页 > 最新文献

2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)最新文献

英文 中文
Human fall detection using segment-level cnn features and sparse dictionary learning 基于片段级cnn特征和稀疏字典学习的人体跌倒检测
C. Ge, I. Gu, Jie Yang
This paper addresses issues in human fall detection from videos. Unlike using handcrafted features in the conventional machine learning, we extract features from Convolutional Neural Networks (CNNs) for human fall detection. Similar to many existing work using two stream inputs, we use a spatial CNN stream with raw image difference and a temporal CNN stream with optical flow as the inputs of CNN. Different from conventional two stream action recognition work, we exploit sparse representation with residual-based pooling on the CNN extracted features, for obtaining more discriminative feature codes. For characterizing the sequential information in video activity, we use the code vector from long-range dynamic feature representation by concatenating codes in segment-levels as the input to a SVM classifier. Experiments have been conducted on two public video databases for fall detection. Comparisons with six existing methods show the effectiveness of the proposed method.
本文讨论了从视频中检测人体跌倒的问题。与在传统机器学习中使用手工制作的特征不同,我们从卷积神经网络(cnn)中提取特征用于人体跌倒检测。与许多使用两种流输入的现有工作类似,我们使用具有原始图像差分的空间CNN流和光流的时间CNN流作为CNN输入。与传统的两流动作识别工作不同,我们在CNN提取的特征上利用基于残差池化的稀疏表示,以获得更具判别性的特征代码。为了表征视频活动中的顺序信息,我们使用远程动态特征表示的代码向量,通过在段级别上连接代码作为支持向量机分类器的输入。在两个公共视频数据库上进行了跌落检测实验。与已有的六种方法进行了比较,结果表明了该方法的有效性。
{"title":"Human fall detection using segment-level cnn features and sparse dictionary learning","authors":"C. Ge, I. Gu, Jie Yang","doi":"10.1109/MLSP.2017.8168185","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168185","url":null,"abstract":"This paper addresses issues in human fall detection from videos. Unlike using handcrafted features in the conventional machine learning, we extract features from Convolutional Neural Networks (CNNs) for human fall detection. Similar to many existing work using two stream inputs, we use a spatial CNN stream with raw image difference and a temporal CNN stream with optical flow as the inputs of CNN. Different from conventional two stream action recognition work, we exploit sparse representation with residual-based pooling on the CNN extracted features, for obtaining more discriminative feature codes. For characterizing the sequential information in video activity, we use the code vector from long-range dynamic feature representation by concatenating codes in segment-levels as the input to a SVM classifier. Experiments have been conducted on two public video databases for fall detection. Comparisons with six existing methods show the effectiveness of the proposed method.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"17 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77713093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
Blind channel equalization of encoded data over galois fields 伽罗瓦域上编码数据的盲信道均衡
D. Fantinato, A. Neves, D. G. Silva, R. Attux
In communication systems, the study of elements and structures defined over Galois fields are generally limited to data coding. However, in this work, a novel perspective that combines data coding and channel equalization is considered to compose a simplified communication system over the field. Besides the coding advantages, this framework is able to restore distortions or malfunctioning processes, and can be potentially applied in network coding models. Interestingly, the operation of the equalizer is possible from a blind standpoint through the exploration of the redundant information introduced by the encoder. More specifically, we define a blind equalization criterion based on the matching of probability mass functions (PMFs) via the Kullback-Leibler divergence. Simulations involving the main aspects of the equalizer and the criterion are performed, including the use of a genetic algorithm to aid the search for the solution, with promising results.
在通信系统中,对伽罗瓦域上定义的元素和结构的研究通常局限于数据编码。然而,在这项工作中,结合数据编码和信道均衡的新观点被认为是组成一个简化的通信系统。除了编码优势之外,该框架还能够恢复扭曲或故障过程,并且可以潜在地应用于网络编码模型。有趣的是,通过探索编码器引入的冗余信息,均衡器的操作可以从盲的角度来看。更具体地说,我们通过Kullback-Leibler散度定义了一个基于概率质量函数(pmf)匹配的盲均衡准则。进行了涉及均衡器和判据主要方面的模拟,包括使用遗传算法来帮助寻找解决方案,结果很有希望。
{"title":"Blind channel equalization of encoded data over galois fields","authors":"D. Fantinato, A. Neves, D. G. Silva, R. Attux","doi":"10.1109/MLSP.2017.8168135","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168135","url":null,"abstract":"In communication systems, the study of elements and structures defined over Galois fields are generally limited to data coding. However, in this work, a novel perspective that combines data coding and channel equalization is considered to compose a simplified communication system over the field. Besides the coding advantages, this framework is able to restore distortions or malfunctioning processes, and can be potentially applied in network coding models. Interestingly, the operation of the equalizer is possible from a blind standpoint through the exploration of the redundant information introduced by the encoder. More specifically, we define a blind equalization criterion based on the matching of probability mass functions (PMFs) via the Kullback-Leibler divergence. Simulations involving the main aspects of the equalizer and the criterion are performed, including the use of a genetic algorithm to aid the search for the solution, with promising results.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"158 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86347607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The time series cluster kernel 时间序列簇核
Karl Øyvind Mikalsen, F. Bianchi, C. Soguero-Ruíz, R. Jenssen
This paper presents the time series cluster kernel (TCK) for multivariate time series with missing data. Our approach leverages the missing data handling properties of Gaussian mixture models (GMM) augmented with empirical prior distributions. Further, we exploit an ensemble learning approach to ensure robustness to parameters by combining the clustering results of many GMM to form the final kernel. In comparative experiments, we demonstrate that the TCK is robust to parameter choices and illustrate its capabilities of dealing with multivariate time series, both with and without missing data.
提出了多变量缺失数据时间序列的聚类核算法。我们的方法利用了高斯混合模型(GMM)与经验先验分布增强的缺失数据处理特性。此外,我们利用集成学习方法通过组合多个GMM的聚类结果形成最终核来确保对参数的鲁棒性。在对比实验中,我们证明了TCK对参数选择具有鲁棒性,并说明了它处理多变量时间序列的能力,无论有无丢失数据。
{"title":"The time series cluster kernel","authors":"Karl Øyvind Mikalsen, F. Bianchi, C. Soguero-Ruíz, R. Jenssen","doi":"10.1109/MLSP.2017.8168187","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168187","url":null,"abstract":"This paper presents the time series cluster kernel (TCK) for multivariate time series with missing data. Our approach leverages the missing data handling properties of Gaussian mixture models (GMM) augmented with empirical prior distributions. Further, we exploit an ensemble learning approach to ensure robustness to parameters by combining the clustering results of many GMM to form the final kernel. In comparative experiments, we demonstrate that the TCK is robust to parameter choices and illustrate its capabilities of dealing with multivariate time series, both with and without missing data.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"58 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80008304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
A deep neural network witharestricted noisy channel for identification of functional introns 基于受限噪声通道的深度神经网络识别功能内含子
Alan Joseph Bekker, M. Chorev, L. Carmel, J. Goldberger
An appreciable fraction of introns is thought to be involved in cellular functions, but there is no obvious way to predict which specific intron is likely to be functional. For each intron we are given a feature representation that is based on its evolutionary patterns. For a small subsets of introns we are also given an indication that they are functional. For all other introns it is not known whether they are functional or not. Our task is to estimate what fraction of introns are functional and, how likely it is that each individual intron is functional. We define a probabilistic classification model that treats the given functionality labels as noisy versions of labels created by a Deep Neural Network model. The maximum-likelihood model parameters are found by utilizing the Expectation-Maximization algorithm. We show that roughly 80% of the functional introns are still not recognized as such, and that roughly a third of all introns are functional.
相当一部分内含子被认为参与细胞功能,但没有明显的方法来预测哪个特定的内含子可能是功能性的。对于每个内含子,我们给出了基于其进化模式的特征表示。对于一小部分内含子,我们也得到了它们是功能性的指示。对于所有其他的内含子,我们不知道它们是否有功能。我们的任务是估计多少内含子是功能性的,以及每个内含子是功能性的可能性有多大。我们定义了一个概率分类模型,该模型将给定的功能标签视为由深度神经网络模型创建的标签的噪声版本。利用期望最大化算法找到最大似然模型参数。我们发现,大约80%的功能性内含子仍未被识别,大约三分之一的内含子是功能性的。
{"title":"A deep neural network witharestricted noisy channel for identification of functional introns","authors":"Alan Joseph Bekker, M. Chorev, L. Carmel, J. Goldberger","doi":"10.1109/MLSP.2017.8168186","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168186","url":null,"abstract":"An appreciable fraction of introns is thought to be involved in cellular functions, but there is no obvious way to predict which specific intron is likely to be functional. For each intron we are given a feature representation that is based on its evolutionary patterns. For a small subsets of introns we are also given an indication that they are functional. For all other introns it is not known whether they are functional or not. Our task is to estimate what fraction of introns are functional and, how likely it is that each individual intron is functional. We define a probabilistic classification model that treats the given functionality labels as noisy versions of labels created by a Deep Neural Network model. The maximum-likelihood model parameters are found by utilizing the Expectation-Maximization algorithm. We show that roughly 80% of the functional introns are still not recognized as such, and that roughly a third of all introns are functional.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"40 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77600545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Noise reduction in low-dose ct using a 3D multiscale sparse denoising autoencoder 基于三维多尺度稀疏去噪自编码器的低剂量ct降噪研究
K. Mentl, B. Mailhé, Florin C. Ghesu, Frank Schebesch, T. Haderlein, A. Maier, M. Nadar
This article presents a novel neural network-based approach for enhancement of 3D medical image data. The proposed networks learn a sparse representation basis by mapping the corrupted input data to corresponding optimal targets. To reinforce the adjustment of the network to the given data, the threshold values are also adaptively learned. In order to capture important image features on various scales and be able to process large computed tomography (CT) volumes in a reasonable time, a multiscale approach is applied. Recursively downsampled versions of the input are used and denoising operator of constant size are learnt at each scale. The networks are trained end-to-end from a database of real highdose acquisitions with synthetic additional noise to simulate the corresponding low-dose scans. Both 2D and 3D networks are evaluated on CT volumes and compared to the block-matching and 3D filtering (BM3D) algorithm. The presented methods achieve an increase of 4% to 11% in the SSIM and of 2.4 to 2.8 dB in the PSNR with respect to the ground truth, outperform BM3D in quantitative comparisions and present no visible texture artifacts. By exploiting volumetric information, 3D networks achieve superior results over 2D networks.
本文提出了一种新的基于神经网络的三维医学图像数据增强方法。该网络通过将损坏的输入数据映射到相应的最优目标来学习稀疏表示基础。为了加强网络对给定数据的调整,阈值也是自适应学习的。为了捕获不同尺度的重要图像特征,并能够在合理的时间内处理大的计算机断层扫描(CT)体积,应用了多尺度方法。使用递归下采样版本的输入,并在每个尺度上学习恒定大小的去噪算子。这些网络端到端从真实高剂量采集的数据库中进行训练,其中含有合成的附加噪声,以模拟相应的低剂量扫描。在CT体积上评估2D和3D网络,并与块匹配和3D滤波(BM3D)算法进行比较。所提出的方法在SSIM上实现了4%至11%的提高,在PSNR上实现了2.4至2.8 dB的提高,在定量比较中优于BM3D,并且没有出现可见的纹理伪影。通过利用体积信息,3D网络比2D网络获得更好的结果。
{"title":"Noise reduction in low-dose ct using a 3D multiscale sparse denoising autoencoder","authors":"K. Mentl, B. Mailhé, Florin C. Ghesu, Frank Schebesch, T. Haderlein, A. Maier, M. Nadar","doi":"10.1109/MLSP.2017.8168176","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168176","url":null,"abstract":"This article presents a novel neural network-based approach for enhancement of 3D medical image data. The proposed networks learn a sparse representation basis by mapping the corrupted input data to corresponding optimal targets. To reinforce the adjustment of the network to the given data, the threshold values are also adaptively learned. In order to capture important image features on various scales and be able to process large computed tomography (CT) volumes in a reasonable time, a multiscale approach is applied. Recursively downsampled versions of the input are used and denoising operator of constant size are learnt at each scale. The networks are trained end-to-end from a database of real highdose acquisitions with synthetic additional noise to simulate the corresponding low-dose scans. Both 2D and 3D networks are evaluated on CT volumes and compared to the block-matching and 3D filtering (BM3D) algorithm. The presented methods achieve an increase of 4% to 11% in the SSIM and of 2.4 to 2.8 dB in the PSNR with respect to the ground truth, outperform BM3D in quantitative comparisions and present no visible texture artifacts. By exploiting volumetric information, 3D networks achieve superior results over 2D networks.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"6 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87680203","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Upper bound performance of semi-definite programming for localisation in inhomogeneous media 非均匀介质中半确定规划的上界性能
E. Nadimi, V. Blanes-Vidal
In this paper, we regarded an absorbing inhomogeneous medium as an assembly of thin layers having different propagation properties. We derived a stochastic model for the refractive index and formulated the localisation problem given noisy distance measurements using graph realisation problem. We relaxed the problem using semi-definite programming (SDP) approach in lp realisation domain and derived upper bounds that follow Edmundson-Madansky bound of order 6p (EM6p) on the SDP objective function to provide an estimation of the techniques' localisation accuracy. Our results showed that the inhomogeneity of the media and the choice of lp norm have significant impact on the ratio of the expected value of the localisation error to the upper bound for the expected optimal SDP objective value. The tightest ratio was derived when l∞ norm was used.
本文将非均匀吸收介质看作具有不同传播特性的薄层的集合。我们推导了折射率的随机模型,并利用图形实现问题提出了给定噪声距离测量的定位问题。我们在lp实现域使用半确定规划(SDP)方法放宽问题,并在SDP目标函数上推导出遵循6p阶Edmundson-Madansky界(EM6p)的上界,以提供对技术定位精度的估计。我们的研究结果表明,介质的非均匀性和lp范数的选择对定位误差期望值与期望最优SDP目标值上界的比值有显著影响。采用l∞范数时,得到最紧比值。
{"title":"Upper bound performance of semi-definite programming for localisation in inhomogeneous media","authors":"E. Nadimi, V. Blanes-Vidal","doi":"10.1109/MLSP.2017.8168145","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168145","url":null,"abstract":"In this paper, we regarded an absorbing inhomogeneous medium as an assembly of thin layers having different propagation properties. We derived a stochastic model for the refractive index and formulated the localisation problem given noisy distance measurements using graph realisation problem. We relaxed the problem using semi-definite programming (SDP) approach in lp realisation domain and derived upper bounds that follow Edmundson-Madansky bound of order 6p (EM6p) on the SDP objective function to provide an estimation of the techniques' localisation accuracy. Our results showed that the inhomogeneity of the media and the choice of lp norm have significant impact on the ratio of the expected value of the localisation error to the upper bound for the expected optimal SDP objective value. The tightest ratio was derived when l∞ norm was used.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"1 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90588609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A two-stage training deep neural network for small pedestrian detection 一种用于小型行人检测的两阶段训练深度神经网络
Tran Duy Linh, Masayuki Arai
In the present paper, we propose a deep network architecture in order to improve the accuracy of pedestrian detection. The proposed method contains a proposal network and a classification network that are trained separately. We use a single shot multibox detector (SSD) as a proposal network to generate the set of pedestrian proposals. The proposal network is fine-tuned from a pre-trained network by several pedestrian data sets of large input size (512 × 512 pixels) in order to improve detection accuracy of small pedestrians. Then, we use a classification network to classify pedestrian proposals. We then combine the scores from the proposal network and the classification network to obtain better final detection scores. Experiments were evaluated using the Caltech test set, and, compared to other state-of-the-art methods of pedestrian detection task, the proposed method obtains better results for small pedestrians (30 to 50 pixels in height) with an average miss rate of 42%.
在本文中,我们提出了一种深度网络架构,以提高行人检测的准确性。该方法包含单独训练的提议网络和分类网络。我们使用单镜头多盒检测器(SSD)作为提议网络来生成行人提议集。为了提高对小型行人的检测精度,该网络通过多个大输入尺寸(512 × 512像素)的行人数据集对预训练网络进行微调。然后,我们使用分类网络对行人建议进行分类。然后,我们将提议网络和分类网络的分数结合起来,得到更好的最终检测分数。使用加州理工学院的测试集对实验进行了评估,与其他最先进的行人检测任务方法相比,该方法在小行人(高度为30 ~ 50像素)中获得了更好的结果,平均缺失率为42%。
{"title":"A two-stage training deep neural network for small pedestrian detection","authors":"Tran Duy Linh, Masayuki Arai","doi":"10.1109/MLSP.2017.8168106","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168106","url":null,"abstract":"In the present paper, we propose a deep network architecture in order to improve the accuracy of pedestrian detection. The proposed method contains a proposal network and a classification network that are trained separately. We use a single shot multibox detector (SSD) as a proposal network to generate the set of pedestrian proposals. The proposal network is fine-tuned from a pre-trained network by several pedestrian data sets of large input size (512 × 512 pixels) in order to improve detection accuracy of small pedestrians. Then, we use a classification network to classify pedestrian proposals. We then combine the scores from the proposal network and the classification network to obtain better final detection scores. Experiments were evaluated using the Caltech test set, and, compared to other state-of-the-art methods of pedestrian detection task, the proposed method obtains better results for small pedestrians (30 to 50 pixels in height) with an average miss rate of 42%.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"44 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85674323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
A layer-block-wise pipeline for memory and bandwidth reduction in distributed deep learning 分布式深度学习中减少内存和带宽的分层块管道
Haruki Mori, Tetsuya Youkawa, S. Izumi, M. Yoshimoto, H. Kawaguchi, Atsuki Inoue
This paper describes a pipelined stochastic gradient descent (SGD) algorithm and its hardware architecture with a memory distributed structure. In the proposed architecture, a pipeline stage takes charge of multiple layers: a “layer block.” The layer-block-wise pipeline has much less weight parameters for network training than conventional multithreading because weight memory is distributed to workers assigned to pipeline stages. The memory capacity of 2.25 GB for the four-stage proposed pipeline is about half of the 3.82 GB for multithreading when a batch size is 32 in VGG-F. Unlike multithreaded data parallelism, no parameter server for weight update or shared I/O data bus is necessary. Therefore, the memory bandwidth is drastically reduced. The proposed four-stage pipeline only needs memory bandwidths of 36.3 MB and 17.0 MB per batch, respectively, for forward propagation and backpropagation processes, whereas four-thread multithreading requires a bandwidth of 974 MB overall for send and receive processes to unify its weight parameters. At the parallelization degree of four, the proposed pipeline maintains training convergence by a factor of 1.12, compared with the conventional multithreaded architecture although the memory capacity and the memory bandwidth are decreased.
介绍了一种基于内存分布式结构的流水线随机梯度下降算法及其硬件结构。在提议的体系结构中,管道阶段负责多个层:一个“层块”。与传统多线程相比,分层块管道的网络训练权重参数要少得多,因为权重内存被分配给分配到管道阶段的工作人员。在VGG-F中,当批处理大小为32时,四级管道的内存容量为2.25 GB,大约是多线程的3.82 GB的一半。与多线程数据并行不同,权重更新或共享I/O数据总线不需要参数服务器。因此,内存带宽大大降低。所提出的四阶段管道每批仅需要36.3 MB和17.0 MB的内存带宽,用于前向传播和反向传播进程,而四线程多线程需要974 MB的带宽用于发送和接收进程以统一其权重参数。在并行度为4的情况下,与传统多线程架构相比,该管道的训练收敛性提高了1.12倍,尽管内存容量和内存带宽有所降低。
{"title":"A layer-block-wise pipeline for memory and bandwidth reduction in distributed deep learning","authors":"Haruki Mori, Tetsuya Youkawa, S. Izumi, M. Yoshimoto, H. Kawaguchi, Atsuki Inoue","doi":"10.1109/MLSP.2017.8168127","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168127","url":null,"abstract":"This paper describes a pipelined stochastic gradient descent (SGD) algorithm and its hardware architecture with a memory distributed structure. In the proposed architecture, a pipeline stage takes charge of multiple layers: a “layer block.” The layer-block-wise pipeline has much less weight parameters for network training than conventional multithreading because weight memory is distributed to workers assigned to pipeline stages. The memory capacity of 2.25 GB for the four-stage proposed pipeline is about half of the 3.82 GB for multithreading when a batch size is 32 in VGG-F. Unlike multithreaded data parallelism, no parameter server for weight update or shared I/O data bus is necessary. Therefore, the memory bandwidth is drastically reduced. The proposed four-stage pipeline only needs memory bandwidths of 36.3 MB and 17.0 MB per batch, respectively, for forward propagation and backpropagation processes, whereas four-thread multithreading requires a bandwidth of 974 MB overall for send and receive processes to unify its weight parameters. At the parallelization degree of four, the proposed pipeline maintains training convergence by a factor of 1.12, compared with the conventional multithreaded architecture although the memory capacity and the memory bandwidth are decreased.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"1 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74535184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Mutual singular spectrum analysis for bioacoustics classification 生物声学分类的互奇异谱分析
B. Gatto, J. Colonna, E. M. Santos, E. Nakamura
Bioacoustics signals classification is an important instrument used in environmental monitoring as it gives the means to efficiently acquire information from the areas, which most of the time are unfeasible to approach. To address these challenges, bioacoustics signals classification systems should meet some requirements, such as low computational resources capabilities. In this paper, we propose a novel bioacoustics signals classification method where no preprocessing techniques are involved and which is able to match sets of signals. The advantages of our proposed method include: a novel and compact representation for bioacoustics signals, which is independent of the signals length. In addition, no preprocessing is required, such as segmentation, noise reduction or syllable extraction. We show that our method is theoretically and practically attractive through experimental results employing a publicity available bioacoustics signal dataset.
生物声学信号分类是环境监测中的一项重要手段,它为有效获取环境监测中难以接近的区域信息提供了手段。为了应对这些挑战,生物声学信号分类系统必须满足一些要求,例如低计算资源能力。在本文中,我们提出了一种新的生物声学信号分类方法,该方法不涉及预处理技术,并且能够匹配信号集。该方法的优点包括:一种新颖而紧凑的生物声学信号表示,与信号长度无关。此外,不需要预处理,如分割,降噪或音节提取。我们通过使用公开可用的生物声学信号数据集的实验结果表明,我们的方法在理论上和实践上都具有吸引力。
{"title":"Mutual singular spectrum analysis for bioacoustics classification","authors":"B. Gatto, J. Colonna, E. M. Santos, E. Nakamura","doi":"10.1109/MLSP.2017.8168113","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168113","url":null,"abstract":"Bioacoustics signals classification is an important instrument used in environmental monitoring as it gives the means to efficiently acquire information from the areas, which most of the time are unfeasible to approach. To address these challenges, bioacoustics signals classification systems should meet some requirements, such as low computational resources capabilities. In this paper, we propose a novel bioacoustics signals classification method where no preprocessing techniques are involved and which is able to match sets of signals. The advantages of our proposed method include: a novel and compact representation for bioacoustics signals, which is independent of the signals length. In addition, no preprocessing is required, such as segmentation, noise reduction or syllable extraction. We show that our method is theoretically and practically attractive through experimental results employing a publicity available bioacoustics signal dataset.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"325 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76548095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 18
A regularized sequential dictionary learning algorithm for fmri data analysis 一种用于fmri数据分析的正则顺序字典学习算法
A. Seghouane, Asif Iqbal
Sequential dictionary learning algorithms have been successfully applied to a number of image processing problems. In a number of these problems however, the data used for dictionary learning are structured matrices with notions of smoothness in the column direction. This prior information which can be traduced as a smoothness constraint on the learned dictionary atoms has not been included in existing dictionary learning algorithms. In this paper, we remedy to this situation by proposing a regularized sequential dictionary learning algorithm. The proposed algorithm differs from the existing ones in their dictionary update stage. The proposed algorithm generates smooth dictionary atoms via the solution of a regularized rank-one matrix approximation problem where regularization is introduced via penalization in the dictionary update stage. Experimental results on synthetic and real data illustrating the performance of the proposed algorithm are provided.
顺序字典学习算法已经成功地应用于许多图像处理问题。然而,在许多这样的问题中,用于字典学习的数据是具有列方向平滑概念的结构化矩阵。现有的字典学习算法中没有包含这种先验信息,这种先验信息可以作为学习到的字典原子的平滑性约束。在本文中,我们通过提出一种正则化顺序字典学习算法来纠正这种情况。该算法与现有算法在字典更新阶段有所不同。该算法通过求解一个正则化的秩一矩阵近似问题来生成光滑的字典原子,其中正则化是在字典更新阶段通过惩罚引入的。给出了合成数据和真实数据的实验结果,说明了该算法的性能。
{"title":"A regularized sequential dictionary learning algorithm for fmri data analysis","authors":"A. Seghouane, Asif Iqbal","doi":"10.1109/MLSP.2017.8168146","DOIUrl":"https://doi.org/10.1109/MLSP.2017.8168146","url":null,"abstract":"Sequential dictionary learning algorithms have been successfully applied to a number of image processing problems. In a number of these problems however, the data used for dictionary learning are structured matrices with notions of smoothness in the column direction. This prior information which can be traduced as a smoothness constraint on the learned dictionary atoms has not been included in existing dictionary learning algorithms. In this paper, we remedy to this situation by proposing a regularized sequential dictionary learning algorithm. The proposed algorithm differs from the existing ones in their dictionary update stage. The proposed algorithm generates smooth dictionary atoms via the solution of a regularized rank-one matrix approximation problem where regularization is introduced via penalization in the dictionary update stage. Experimental results on synthetic and real data illustrating the performance of the proposed algorithm are provided.","PeriodicalId":6542,"journal":{"name":"2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)","volume":"51 1","pages":"1-6"},"PeriodicalIF":0.0,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79798684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
期刊
2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1