Using single impact metrics to assess research in business and economics: why institutions should use multi-criteria systems for assessing research

S. Olavarrieta
{"title":"Using single impact metrics to assess research in business and economics: why institutions should use multi-criteria systems for assessing research","authors":"S. Olavarrieta","doi":"10.1108/jefas-04-2021-0033","DOIUrl":null,"url":null,"abstract":"PurposeDespite the general recommendation of using a combination of multiple criteria for research assessment and faculty promotion decisions, the raise of quantitative indicators is generating an emerging trend in Business Schools to use single journal impact factors (IFs) as key (unique) drivers for those relevant school decisions. This paper aims to investigate the effects of using single Web of Science (WoS)-based journal impact metrics when assessing research from two related disciplines: Business and Economics, and its potential impact for the strategic sustainability of a Business School.Design/methodology/approachThis study collected impact indicators data for Business and Economics journals from the Clarivate Web of Science database. We concentrated on the IF indicators, the Eigenfactor and the article influence score (AIS). This study examined the correlations between these indicators and then ranked disciplines and journals using these different impact metrics.FindingsConsistent with previous findings, this study finds positive correlations among these metrics. Then this study ranks the disciplines and journals using each impact metric, finding relevant and substantial differences, depending on the metric used. It is found that using AIS instead of the IF raises the relative ranking of Economics, while Business remains basically with the same rank.Research limitations/implications This study contributes to the research assessment literature by adding substantial evidence that given the sensitivity of journal rankings to particular indicators, the selection of a single impact metric for assessing research and hiring/promotion and tenure decisions is risky and too simplistic. This research shows that biases may be larger when assessment involves researchers from related disciplines – like Business and Economics – but with different research foundations and traditions.Practical implicationsConsistent with the literature, given the sensibility of journal rankings to particular indicators, the selection of a single impact metric for assessing research, assigning research funds and hiring/promotion and tenure decisions is risky and simplistic. However, this research shows that risks and biases may be larger when assessment involves researchers from related disciplines – like Business and Economics – but with different research foundations and trajectories. The use of multiple criteria is advised for such purposes.Originality/valueThis is an applied work using real data from WoS that addresses a practical case of comparing the use of different journal IFs to rank-related disciplines like Business and Economics, with important implications for faculty tenure and promotion committees and for research funds granting institutions and decision-makers.","PeriodicalId":53491,"journal":{"name":"Journal of Economics, Finance and Administrative Science","volume":"26 1","pages":""},"PeriodicalIF":2.3000,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Economics, Finance and Administrative Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1108/jefas-04-2021-0033","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ECONOMICS","Score":null,"Total":0}
引用次数: 2

Abstract

PurposeDespite the general recommendation of using a combination of multiple criteria for research assessment and faculty promotion decisions, the raise of quantitative indicators is generating an emerging trend in Business Schools to use single journal impact factors (IFs) as key (unique) drivers for those relevant school decisions. This paper aims to investigate the effects of using single Web of Science (WoS)-based journal impact metrics when assessing research from two related disciplines: Business and Economics, and its potential impact for the strategic sustainability of a Business School.Design/methodology/approachThis study collected impact indicators data for Business and Economics journals from the Clarivate Web of Science database. We concentrated on the IF indicators, the Eigenfactor and the article influence score (AIS). This study examined the correlations between these indicators and then ranked disciplines and journals using these different impact metrics.FindingsConsistent with previous findings, this study finds positive correlations among these metrics. Then this study ranks the disciplines and journals using each impact metric, finding relevant and substantial differences, depending on the metric used. It is found that using AIS instead of the IF raises the relative ranking of Economics, while Business remains basically with the same rank.Research limitations/implications This study contributes to the research assessment literature by adding substantial evidence that given the sensitivity of journal rankings to particular indicators, the selection of a single impact metric for assessing research and hiring/promotion and tenure decisions is risky and too simplistic. This research shows that biases may be larger when assessment involves researchers from related disciplines – like Business and Economics – but with different research foundations and traditions.Practical implicationsConsistent with the literature, given the sensibility of journal rankings to particular indicators, the selection of a single impact metric for assessing research, assigning research funds and hiring/promotion and tenure decisions is risky and simplistic. However, this research shows that risks and biases may be larger when assessment involves researchers from related disciplines – like Business and Economics – but with different research foundations and trajectories. The use of multiple criteria is advised for such purposes.Originality/valueThis is an applied work using real data from WoS that addresses a practical case of comparing the use of different journal IFs to rank-related disciplines like Business and Economics, with important implications for faculty tenure and promotion committees and for research funds granting institutions and decision-makers.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
使用单一影响指标来评估商业和经济研究:为什么机构应该使用多标准系统来评估研究
尽管普遍建议在研究评估和教师晋升决策中结合使用多种标准,但定量指标的增加正在商学院中产生一种新趋势,即使用单一期刊影响因子(if)作为相关学院决策的关键(独特)驱动因素。本文旨在探讨使用基于Web of Science (WoS)的期刊影响指标评估商业和经济学两个相关学科研究的效果,以及其对商学院战略可持续性的潜在影响。设计/方法/方法本研究从Clarivate Web of Science数据库中收集商业和经济期刊的影响指标数据。我们重点研究了影响因子指标、特征因子和文章影响评分(AIS)。本研究检查了这些指标之间的相关性,然后使用这些不同的影响指标对学科和期刊进行排名。研究结果与先前的研究结果一致,本研究发现这些指标之间存在正相关。然后,本研究使用每个影响指标对学科和期刊进行排名,根据所使用的指标找到相关的和实质性的差异。结果发现,用AIS代替IF提高了经济学的相对排名,而商科的排名基本保持不变。本研究为研究评估文献做出了贡献,增加了大量证据,表明鉴于期刊排名对特定指标的敏感性,选择单一的影响指标来评估研究、招聘/晋升和终身教职决策是有风险的,而且过于简单。这项研究表明,当评估涉及来自相关学科(如商业和经济学)但研究基础和传统不同的研究人员时,偏见可能会更大。实际意义与文献一致,考虑到期刊排名对特定指标的敏感性,选择单一的影响指标来评估研究、分配研究经费、雇用/晋升和终身教职决策是有风险和简单的。然而,这项研究表明,当评估涉及来自相关学科(如商业和经济学)但具有不同研究基础和轨迹的研究人员时,风险和偏差可能更大。为此,建议使用多个标准。原创性/价值这是一项应用工作,使用WoS的真实数据,解决了一个实际案例,比较了不同期刊IFs与排名相关学科(如商业和经济学)的使用情况,对教师终身教职和晋升委员会以及研究基金授予机构和决策者具有重要意义。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Journal of Economics, Finance and Administrative Science
Journal of Economics, Finance and Administrative Science Economics, Econometrics and Finance-Economics, Econometrics and Finance (all)
CiteScore
5.10
自引率
20.80%
发文量
23
审稿时长
12 weeks
期刊介绍: The Universidad ESAN, with more than 50 years of experience in the higher education field and post graduate studies, desires to contribute to the academic community with the most outstanding pieces of research. We gratefully welcome suggestions and contributions from business areas such as operations, supply chain, economics, finance and administration. We publish twice a year, six articles for each issue.
期刊最新文献
At what age do Mexicans suffer the most financial stress? Editorial: ESAN's 60th anniversary, second special section on business economics in Ibero-America and editorial transition SMEs growth and profitability, productivity and debt relationships Spillovers between cryptocurrencies, gold and stock markets: implication for hedging strategies and portfolio diversification under the COVID-19 pandemic The specific factors of heterogeneity characterizing investors' beliefs
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1