Advancing the personalized advantage index (PAI): a systematic review and application in two large multi-site samples in anxiety disorders.

IF 5.9 2区 医学 Q1 PSYCHIATRY Psychological Medicine Pub Date : 2024-12-16 DOI:10.1017/S0033291724003118
Charlotte Meinke, Silvan Hornstein, Johanna Schmidt, Volker Arolt, Udo Dannlowski, Jürgen Deckert, Katharina Domschke, Lydia Fehm, Thomas Fydrich, Alexander L Gerlach, Alfons O Hamm, Ingmar Heinig, Jürgen Hoyer, Tilo Kircher, Katja Koelkebeck, Thomas Lang, Jürgen Margraf, Peter Neudeck, Paul Pauli, Jan Richter, Winfried Rief, Silvia Schneider, Benjamin Straube, Andreas Ströhle, Hans-Ulrich Wittchen, Peter Zwanzger, Henrik Walter, Ulrike Lueken, Andre Pittig, Kevin Hilbert
{"title":"Advancing the personalized advantage index (PAI): a systematic review and application in two large multi-site samples in anxiety disorders.","authors":"Charlotte Meinke, Silvan Hornstein, Johanna Schmidt, Volker Arolt, Udo Dannlowski, Jürgen Deckert, Katharina Domschke, Lydia Fehm, Thomas Fydrich, Alexander L Gerlach, Alfons O Hamm, Ingmar Heinig, Jürgen Hoyer, Tilo Kircher, Katja Koelkebeck, Thomas Lang, Jürgen Margraf, Peter Neudeck, Paul Pauli, Jan Richter, Winfried Rief, Silvia Schneider, Benjamin Straube, Andreas Ströhle, Hans-Ulrich Wittchen, Peter Zwanzger, Henrik Walter, Ulrike Lueken, Andre Pittig, Kevin Hilbert","doi":"10.1017/S0033291724003118","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>The Personalized Advantage Index (PAI) shows promise as a method for identifying the most effective treatment for individual patients. Previous studies have demonstrated its utility in retrospective evaluations across various settings. In this study, we explored the effect of different methodological choices in predictive modelling underlying the PAI.</p><p><strong>Methods: </strong>Our approach involved a two-step procedure. First, we conducted a review of prior studies utilizing the PAI, evaluating each study using the Prediction model study Risk Of Bias Assessment Tool (PROBAST). We specifically assessed whether the studies adhered to two standards of predictive modeling: refraining from using leave-one-out cross-validation (LOO CV) and preventing data leakage. Second, we examined the impact of deviating from these methodological standards in real data. We employed both a traditional approach violating these standards and an advanced approach implementing them in two large-scale datasets, PANIC-net (<i>n</i> = 261) and Protect-AD (<i>n</i> = 614).</p><p><strong>Results: </strong>The PROBAST-rating revealed a substantial risk of bias across studies, primarily due to inappropriate methodological choices. Most studies did not adhere to the examined prediction modeling standards, employing LOO CV and allowing data leakage. The comparison between the traditional and advanced approach revealed that ignoring these standards could systematically overestimate the utility of the PAI.</p><p><strong>Conclusion: </strong>Our study cautions that violating standards in predictive modeling may strongly influence the evaluation of the PAI's utility, possibly leading to false positive results. To support an unbiased evaluation, crucial for potential clinical application, we provide a low-bias, openly accessible, and meticulously annotated script implementing the PAI.</p>","PeriodicalId":20891,"journal":{"name":"Psychological Medicine","volume":" ","pages":"1-13"},"PeriodicalIF":5.9000,"publicationDate":"2024-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psychological Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1017/S0033291724003118","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHIATRY","Score":null,"Total":0}
引用次数: 0

Abstract

Background: The Personalized Advantage Index (PAI) shows promise as a method for identifying the most effective treatment for individual patients. Previous studies have demonstrated its utility in retrospective evaluations across various settings. In this study, we explored the effect of different methodological choices in predictive modelling underlying the PAI.

Methods: Our approach involved a two-step procedure. First, we conducted a review of prior studies utilizing the PAI, evaluating each study using the Prediction model study Risk Of Bias Assessment Tool (PROBAST). We specifically assessed whether the studies adhered to two standards of predictive modeling: refraining from using leave-one-out cross-validation (LOO CV) and preventing data leakage. Second, we examined the impact of deviating from these methodological standards in real data. We employed both a traditional approach violating these standards and an advanced approach implementing them in two large-scale datasets, PANIC-net (n = 261) and Protect-AD (n = 614).

Results: The PROBAST-rating revealed a substantial risk of bias across studies, primarily due to inappropriate methodological choices. Most studies did not adhere to the examined prediction modeling standards, employing LOO CV and allowing data leakage. The comparison between the traditional and advanced approach revealed that ignoring these standards could systematically overestimate the utility of the PAI.

Conclusion: Our study cautions that violating standards in predictive modeling may strongly influence the evaluation of the PAI's utility, possibly leading to false positive results. To support an unbiased evaluation, crucial for potential clinical application, we provide a low-bias, openly accessible, and meticulously annotated script implementing the PAI.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
推进个性化优势指数 (PAI):系统回顾及在焦虑症两个大型多站点样本中的应用。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Psychological Medicine
Psychological Medicine 医学-精神病学
CiteScore
11.30
自引率
4.30%
发文量
711
审稿时长
3-6 weeks
期刊介绍: Now in its fifth decade of publication, Psychological Medicine is a leading international journal in the fields of psychiatry, related aspects of psychology and basic sciences. From 2014, there are 16 issues a year, each featuring original articles reporting key research being undertaken worldwide, together with shorter editorials by distinguished scholars and an important book review section. The journal''s success is clearly demonstrated by a consistently high impact factor.
期刊最新文献
Protective role of school climate for impacts of COVID-19 on depressive symptoms and psychotic experiences among adolescents: a population-based cohort study. Networks of care for the modern adolescent. Predicting remission following CBT for childhood anxiety disorders: a machine learning approach. Across the multiverse: exploring a diverse set of specifications related to cross-sectional and prospective associations between adolescent alcohol use and emotional problems. Advancing the personalized advantage index (PAI): a systematic review and application in two large multi-site samples in anxiety disorders.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1