The Skip-gram with negative sampling (SGNS) method of Word2Vec is an unsupervised approach to map words in a text corpus to low dimensional real vectors. The learned vectors capture semantic relationships between co-occurring words and can be used as inputs to many natural language processing and machine learning tasks. There are several high-performance implementations of the Word2Vec SGNS method. In this paper, we introduce a new optimization called context combining to further boost SGNS performance on multicore systems. For processing the One Billion Word benchmark dataset on a 16-core platform, we show that our approach is 3.53x faster than the original multithreaded Word2Vec implementation and 1.28x faster than a recent parallel Word2Vec implementation. We also show that our accuracy on benchmark queries is comparable to state-of-the-art implementations.
Word2Vec的Skip-gram with negative sampling (SGNS)方法是一种将文本语料库中的单词映射到低维实向量的无监督方法。学习到的向量捕获了共存词之间的语义关系，可以用作许多自然语言处理和机器学习任务的输入。有几种高性能的Word2Vec SGNS方法实现。在本文中，我们引入了一种称为上下文组合的新优化，以进一步提高SGNS在多核系统上的性能。对于在16核平台上处理十亿字基准数据集，我们的方法比原始的多线程Word2Vec实现快3.53倍，比最近的并行Word2Vec实现快1.28倍。我们还展示了我们在基准查询上的准确性可以与最先进的实现相媲美。
{"title":"Optimizing Word2Vec Performance on Multicore Systems","authors":"Vasudevan Rengasamy, Tao-yang Fu, Wang-Chien Lee, Kamesh Madduri","doi":"10.1145/3149704.3149768","DOIUrl":"https://doi.org/10.1145/3149704.3149768","url":null,"abstract":"The Skip-gram with negative sampling (SGNS) method of Word2Vec is an unsupervised approach to map words in a text corpus to low dimensional real vectors. The learned vectors capture semantic relationships between co-occurring words and can be used as inputs to many natural language processing and machine learning tasks. There are several high-performance implementations of the Word2Vec SGNS method. In this paper, we introduce a new optimization called context combining to further boost SGNS performance on multicore systems. For processing the One Billion Word benchmark dataset on a 16-core platform, we show that our approach is 3.53x faster than the original multithreaded Word2Vec implementation and 1.28x faster than a recent parallel Word2Vec implementation. We also show that our accuracy on benchmark queries is comparable to state-of-the-art implementations.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134600493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Quantitative games, where quantitative objectives are defined on weighted game arenas, provide natural tools for designing faithful models of embedded controllers. Instances of these games are the so called Energy Games. Starting from a sequential baseline implementation, we investigate the use of massively data computation capabilities supported by modern GPUs to solve the initial credit problem for Energy Games. We present different parallel implementations on multi-core CPU and GPU systems. Our solution outperforms the baseline implementation by up to 36x speedup and obtains a faster convergence time on real-world graphs.
{"title":"Accelerating Energy Games Solvers on Modern Architectures","authors":"A. Formisano, R. Gentilini, Flavio Vella","doi":"10.1145/3149704.3149771","DOIUrl":"https://doi.org/10.1145/3149704.3149771","url":null,"abstract":"Quantitative games, where quantitative objectives are defined on weighted game arenas, provide natural tools for designing faithful models of embedded controllers. Instances of these games are the so called Energy Games. Starting from a sequential baseline implementation, we investigate the use of massively data computation capabilities supported by modern GPUs to solve the initial credit problem for Energy Games. We present different parallel implementations on multi-core CPU and GPU systems. Our solution outperforms the baseline implementation by up to 36x speedup and obtains a faster convergence time on real-world graphs.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"66 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133086917","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","authors":"","doi":"10.1145/3149704","DOIUrl":"https://doi.org/10.1145/3149704","url":null,"abstract":"","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128476181","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：