Evaluating the Linguistic Coverage of OpenAlex: An Assessment of Metadata Accuracy and Completeness

arXiv - CS - Digital Libraries Pub Date : 2024-09-16 DOI:arxiv-2409.10633

Lucía Céspedes, Diego Kozlowski, Carolina Pradier, Maxime Holmberg Sainte-Marie, Natsumi Solange Shokida, Pierre Benz, Constance Poitras, Anton Boudreau Ninkov, Saeideh Ebrahimy, Philips Ayeni, Sarra Filali, Bing Li, Vincent Larivière

{"title":"Evaluating the Linguistic Coverage of OpenAlex: An Assessment of Metadata Accuracy and Completeness","authors":"Lucía Céspedes, Diego Kozlowski, Carolina Pradier, Maxime Holmberg Sainte-Marie, Natsumi Solange Shokida, Pierre Benz, Constance Poitras, Anton Boudreau Ninkov, Saeideh Ebrahimy, Philips Ayeni, Sarra Filali, Bing Li, Vincent Larivière","doi":"arxiv-2409.10633","DOIUrl":null,"url":null,"abstract":"Clarivate's Web of Science (WoS) and Elsevier's Scopus have been for decades\nthe main sources of bibliometric information. Although highly curated, these\nclosed, proprietary databases are largely biased towards English-language\npublications, underestimating the use of other languages in research\ndissemination. Launched in 2022, OpenAlex promised comprehensive, inclusive,\nand open-source research information. While already in use by scholars and\nresearch institutions, the quality of its metadata is currently being assessed.\nThis paper contributes to this literature by assessing the completeness and\naccuracy of its metadata related to language, through a comparison with WoS, as\nwell as an in-depth manual validation of a sample of 6,836 articles. Results\nshow that OpenAlex exhibits a far more balanced linguistic coverage than WoS.\nHowever, language metadata is not always accurate, which leads OpenAlex to\noverestimate the place of English while underestimating that of other\nlanguages. If used critically, OpenAlex can provide comprehensive and\nrepresentative analyses of languages used for scholarly publishing. However,\nmore work is needed at infrastructural level to ensure the quality of metadata\non language.","PeriodicalId":501285,"journal":{"name":"arXiv - CS - Digital Libraries","volume":"22 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Digital Libraries","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.10633","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

Abstract

Clarivate's Web of Science (WoS) and Elsevier's Scopus have been for decades the main sources of bibliometric information. Although highly curated, these closed, proprietary databases are largely biased towards English-language publications, underestimating the use of other languages in research dissemination. Launched in 2022, OpenAlex promised comprehensive, inclusive, and open-source research information. While already in use by scholars and research institutions, the quality of its metadata is currently being assessed. This paper contributes to this literature by assessing the completeness and accuracy of its metadata related to language, through a comparison with WoS, as well as an in-depth manual validation of a sample of 6,836 articles. Results show that OpenAlex exhibits a far more balanced linguistic coverage than WoS. However, language metadata is not always accurate, which leads OpenAlex to overestimate the place of English while underestimating that of other languages. If used critically, OpenAlex can provide comprehensive and representative analyses of languages used for scholarly publishing. However, more work is needed at infrastructural level to ensure the quality of metadata on language.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

评估 OpenAlex 的语言覆盖范围：元数据准确性和完整性评估

几十年来，Clarivate 的 Web of Science (WoS) 和 Elsevier 的 Scopus 一直是文献计量信息的主要来源。这些封闭式的专有数据库虽然经过了精心策划，但在很大程度上偏重于英语出版物，低估了其他语言在研究传播中的使用。OpenAlex 于 2022 年推出，承诺提供全面、包容和开源的研究信息。本文通过与 WoS 的比较，以及对 6836 篇文章样本的深入人工验证，评估了其与语言相关的元数据的完整性和准确性，为相关文献做出了贡献。结果表明，OpenAlex 的语言覆盖范围比 WoS 更均衡。然而，语言元数据并不总是准确的，这导致 OpenAlex 高估了英语的地位，而低估了其他语言的地位。如果认真使用，OpenAlex 可以对用于学术出版的语言进行全面和有代表性的分析。不过，还需要在基础设施层面开展更多工作，以确保语言元数据的质量。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

arXiv - CS - Digital Libraries

自引率

0.00%

发文量