Coordinated distributed experiments in ecology do not consistently reduce heterogeneity in effect size

IF 3.1 2区 环境科学与生态学 Q2 ECOLOGY Oikos Pub Date : 2024-04-08 DOI:10.1111/oik.10722
Julia Bebout, Jeremy W. Fox
{"title":"Coordinated distributed experiments in ecology do not consistently reduce heterogeneity in effect size","authors":"Julia Bebout, Jeremy W. Fox","doi":"10.1111/oik.10722","DOIUrl":null,"url":null,"abstract":"Ecological meta‐analyses usually exhibit high relative heterogeneity of effect size: most among‐study variation in effect size represents true variation in mean effect size, rather than sampling error. This heterogeneity arises from both methodological and ecological sources. Methodological heterogeneity is a nuisance that complicates the interpretation of data syntheses. One way to reduce methodological heterogeneity is via coordinated distributed experiments, in which investigators conduct the same experiment at different sites, using the same methods. We tested whether coordinated distributed experiments in ecology exhibit 1) low heterogeneity in effect size, and 2) lower heterogeneity than meta‐analyses, using data on 17 effects from eight coordinated distributed experiments, and 406 meta‐analyses. Consistent with our expectations, among‐site heterogeneity typically comprised <50% of the variance in effect size in distributed experiments. In contrast, heterogeneity within and among studies typically comprised >90% of the variance in effect size in meta‐analyses. However, this difference largely reflected the small size of most coordinated distributed experiments, and was no longer significant after controlling for size (number of studies or sites). These results are consistent with the hypothesis that methodological heterogeneity rarely comprises a substantial fraction of variance in effect size in ecology. We also conducted pairwise comparisons of absolute heterogeneity between coordinated distributed experiments and meta‐analyses on the same topics. Coordinated distributed experiments did not consistently exhibit lower absolute heterogeneity in effect size than meta‐analyses on the same topics. Our findings suggest that coordinated distributed experiments rarely increase uniformity of results by reducing methodological heterogeneity. Our results help refine the numerous distinct reasons for conducting coordinated distributed experiments.","PeriodicalId":19496,"journal":{"name":"Oikos","volume":null,"pages":null},"PeriodicalIF":3.1000,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Oikos","FirstCategoryId":"93","ListUrlMain":"https://doi.org/10.1111/oik.10722","RegionNum":2,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ECOLOGY","Score":null,"Total":0}
引用次数: 0

Abstract

Ecological meta‐analyses usually exhibit high relative heterogeneity of effect size: most among‐study variation in effect size represents true variation in mean effect size, rather than sampling error. This heterogeneity arises from both methodological and ecological sources. Methodological heterogeneity is a nuisance that complicates the interpretation of data syntheses. One way to reduce methodological heterogeneity is via coordinated distributed experiments, in which investigators conduct the same experiment at different sites, using the same methods. We tested whether coordinated distributed experiments in ecology exhibit 1) low heterogeneity in effect size, and 2) lower heterogeneity than meta‐analyses, using data on 17 effects from eight coordinated distributed experiments, and 406 meta‐analyses. Consistent with our expectations, among‐site heterogeneity typically comprised <50% of the variance in effect size in distributed experiments. In contrast, heterogeneity within and among studies typically comprised >90% of the variance in effect size in meta‐analyses. However, this difference largely reflected the small size of most coordinated distributed experiments, and was no longer significant after controlling for size (number of studies or sites). These results are consistent with the hypothesis that methodological heterogeneity rarely comprises a substantial fraction of variance in effect size in ecology. We also conducted pairwise comparisons of absolute heterogeneity between coordinated distributed experiments and meta‐analyses on the same topics. Coordinated distributed experiments did not consistently exhibit lower absolute heterogeneity in effect size than meta‐analyses on the same topics. Our findings suggest that coordinated distributed experiments rarely increase uniformity of results by reducing methodological heterogeneity. Our results help refine the numerous distinct reasons for conducting coordinated distributed experiments.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
生态学中的协调分布式实验并不能持续减少效应大小的异质性
生态荟萃分析通常表现出效应大小的高度相对异质性:研究间效应大小的大多数差异代表平均效应大小的真实差异,而不是抽样误差。这种异质性既有方法上的,也有生态学上的。方法异质性是一种困扰,会使数据综合解释复杂化。减少方法异质性的一种方法是通过协调分布式实验,即研究者在不同地点使用相同的方法进行相同的实验。我们利用 8 项协调分布式实验的 17 项效应数据和 406 项元分析数据,检验了生态学协调分布式实验是否表现出:1)效应大小异质性低;2)异质性低于元分析。与我们的预期一致,在分布式实验中,研究地点之间的异质性通常占效应大小方差的 50%。与此相反,在荟萃分析中,研究内部和研究之间的异质性通常占效应大小方差的 90%。然而,这种差异在很大程度上反映了大多数协调分布式实验的规模较小,在控制了规模(研究或研究地点的数量)之后,这种差异不再显著。这些结果与方法异质性很少构成生态学效应大小变异的主要部分这一假设是一致的。我们还对协调分布式实验与相同主题的荟萃分析之间的绝对异质性进行了配对比较。与相同主题的荟萃分析相比,协调分布式实验的效应大小绝对异质性并不总是较低。我们的研究结果表明,协调分布式实验很少能通过减少方法异质性来提高结果的一致性。我们的研究结果有助于完善进行协调分布式实验的众多不同原因。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Oikos
Oikos 环境科学-生态学
CiteScore
6.20
自引率
5.90%
发文量
152
审稿时长
6-12 weeks
期刊介绍: Oikos publishes original and innovative research on all aspects of ecology, defined as organism-environment interactions at various spatiotemporal scales, so including macroecology and evolutionary ecology. Emphasis is on theoretical and empirical work aimed at generalization and synthesis across taxa, systems and ecological disciplines. Papers can contribute to new developments in ecology by reporting novel theory or critical empirical results, and "synthesis" can include developing new theory, tests of general hypotheses, or bringing together established or emerging areas of ecology. Confirming or extending the established literature, by for example showing results that are novel for a new taxon, or purely applied research, is given low priority.
期刊最新文献
Linking fine‐root diameter across root orders with climatic, biological and edaphic factors in the Northern Hemisphere Do plants respond to multi‐year disturbance rhythms and are we missing the beat? Importance of accounting for imperfect detection of plants in the estimation of population growth rates Landscape structures and stand attributes jointly regulate forest productivity Evolutionary cycles in a model of nestmate recognition
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1