Maren Hackenberg, Michelle Pfaffenlehner, Max Behrens, Astrid Pechmann, Janbernd Kirschner, Harald Binder
In a longitudinal clinical registry, different measurement instruments might have been used for assessing individuals at different time points. To combine them, we investigate deep learning techniques for obtaining a joint latent representation, to which the items of different measurement instruments are mapped. This corresponds to domain adaptation, an established concept in computer science for image data. Using the proposed approach as an example, we evaluate the potential of domain adaptation in a longitudinal cohort setting with a rather small number of time points, motivated by an application with different motor function measurement instruments in a registry of spinal muscular atrophy (SMA) patients. There, we model trajectories in the latent representation by ordinary differential equations (ODEs), where person-specific ODE parameters are inferred from baseline characteristics. The goodness of fit and complexity of the ODE solutions then allow to judge the measurement instrument mappings. We subsequently explore how alignment can be improved by incorporating corresponding penalty terms into model fitting. To systematically investigate the effect of differences between measurement instruments, we consider several scenarios based on modified SMA data, including scenarios where a mapping should be feasible in principle and scenarios where no perfect mapping is available. While misalignment increases in more complex scenarios, some structure is still recovered, even if the availability of measurement instruments depends on patient state. A reasonable mapping is feasible also in the more complex real SMA data set. These results indicate that domain adaptation might be more generally useful in statistical modeling for longitudinal registry data.
{"title":"Investigating a Domain Adaptation Approach for Integrating Different Measurement Instruments in a Longitudinal Clinical Registry","authors":"Maren Hackenberg, Michelle Pfaffenlehner, Max Behrens, Astrid Pechmann, Janbernd Kirschner, Harald Binder","doi":"10.1002/bimj.70023","DOIUrl":"10.1002/bimj.70023","url":null,"abstract":"<p>In a longitudinal clinical registry, different measurement instruments might have been used for assessing individuals at different time points. To combine them, we investigate deep learning techniques for obtaining a joint latent representation, to which the items of different measurement instruments are mapped. This corresponds to domain adaptation, an established concept in computer science for image data. Using the proposed approach as an example, we evaluate the potential of domain adaptation in a longitudinal cohort setting with a rather small number of time points, motivated by an application with different motor function measurement instruments in a registry of spinal muscular atrophy (SMA) patients. There, we model trajectories in the latent representation by ordinary differential equations (ODEs), where person-specific ODE parameters are inferred from baseline characteristics. The goodness of fit and complexity of the ODE solutions then allow to judge the measurement instrument mappings. We subsequently explore how alignment can be improved by incorporating corresponding penalty terms into model fitting. To systematically investigate the effect of differences between measurement instruments, we consider several scenarios based on modified SMA data, including scenarios where a mapping should be feasible in principle and scenarios where no perfect mapping is available. While misalignment increases in more complex scenarios, some structure is still recovered, even if the availability of measurement instruments depends on patient state. A reasonable mapping is feasible also in the more complex real SMA data set. These results indicate that domain adaptation might be more generally useful in statistical modeling for longitudinal registry data.</p>","PeriodicalId":55360,"journal":{"name":"Biometrical Journal","volume":"67 1","pages":""},"PeriodicalIF":1.3,"publicationDate":"2024-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/bimj.70023","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142857076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Milena Wünsch, Christina Sauer, Moritz Herrmann, Ludwig Christian Hinske, Anne-Laure Boulesteix
Gene set analysis, a popular approach for analyzing high-throughput gene expression data, aims to identify sets of genes that show enriched expression patterns between two conditions. In addition to the multitude of methods available for this task, users are typically left with many options when creating the required input and specifying the internal parameters of the chosen method. This flexibility can lead to uncertainty about the “right” choice, further reinforced by a lack of evidence-based guidance. Especially when their statistical experience is scarce, this uncertainty might entice users to produce preferable results using a “trial-and-error” approach. While it may seem unproblematic at first glance, this practice can be viewed as a form of “cherry-picking” and cause an optimistic bias, rendering the results nonreplicable on independent data. After this problem has attracted a lot of attention in the context of classical hypothesis testing, we now aim to raise awareness of such overoptimism in the different and more complex context of gene set analyses. We mimic a hypothetical researcher who systematically selects the analysis variants yielding their preferred results, thereby considering three distinct goals they might pursue. Using a selection of popular gene set analysis methods, we tweak the results in this way for two frequently used benchmark gene expression data sets. Our study indicates that the potential for overoptimism is particularly high for a group of methods frequently used despite being commonly criticized. We conclude by providing practical recommendations to counter overoptimism in research findings in gene set analysis and beyond.
{"title":"To Tweak or Not to Tweak. How Exploiting Flexibilities in Gene Set Analysis Leads to Overoptimism","authors":"Milena Wünsch, Christina Sauer, Moritz Herrmann, Ludwig Christian Hinske, Anne-Laure Boulesteix","doi":"10.1002/bimj.70016","DOIUrl":"10.1002/bimj.70016","url":null,"abstract":"<p>Gene set analysis, a popular approach for analyzing high-throughput gene expression data, aims to identify sets of genes that show enriched expression patterns between two conditions. In addition to the multitude of methods available for this task, users are typically left with many options when creating the required input and specifying the internal parameters of the chosen method. This flexibility can lead to uncertainty about the “right” choice, further reinforced by a lack of evidence-based guidance. Especially when their statistical experience is scarce, this uncertainty might entice users to produce preferable results using a “trial-and-error” approach. While it may seem unproblematic at first glance, this practice can be viewed as a form of “cherry-picking” and cause an optimistic bias, rendering the results nonreplicable on independent data. After this problem has attracted a lot of attention in the context of classical hypothesis testing, we now aim to raise awareness of such overoptimism in the different and more complex context of gene set analyses. We mimic a hypothetical researcher who systematically selects the analysis variants yielding their preferred results, thereby considering three distinct goals they might pursue. Using a selection of popular gene set analysis methods, we tweak the results in this way for two frequently used benchmark gene expression data sets. Our study indicates that the potential for overoptimism is particularly high for a group of methods frequently used despite being commonly criticized. We conclude by providing practical recommendations to counter overoptimism in research findings in gene set analysis and beyond.</p>","PeriodicalId":55360,"journal":{"name":"Biometrical Journal","volume":"67 1","pages":""},"PeriodicalIF":1.3,"publicationDate":"2024-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/bimj.70016","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142857080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dominic Edelmann, Tobias Terzer, Peter Horak, Richard Schlenk, Axel Benner
The progression-free-survival ratio is a popular endpoint in oncology trials, which is frequently applied to evaluate the efficacy of molecularly targeted treatments in late-stage patients. Using elementary calculations and simulations, numerous shortcomings of the current methodology are pointed out. As a remedy to these shortcomings, an alternative methodology is proposed, using a marginal Cox model or a marginal accelerated failure time model for clustered time-to-event data. Using comprehensive simulations, it is shown that this methodology outperforms existing methods in settings where the intrapatient correlation is low to moderate. The performance of the model is further demonstrated in a real data example from a molecularly aided tumor trial. Sample size considerations are discussed.
{"title":"The Progression-Free-Survival Ratio in Molecularly Aided Tumor Trials: A Critical Examination of Current Practice and Suggestions for Alternative Methods","authors":"Dominic Edelmann, Tobias Terzer, Peter Horak, Richard Schlenk, Axel Benner","doi":"10.1002/bimj.70028","DOIUrl":"10.1002/bimj.70028","url":null,"abstract":"<p>The progression-free-survival ratio is a popular endpoint in oncology trials, which is frequently applied to evaluate the efficacy of molecularly targeted treatments in late-stage patients. Using elementary calculations and simulations, numerous shortcomings of the current methodology are pointed out. As a remedy to these shortcomings, an alternative methodology is proposed, using a marginal Cox model or a marginal accelerated failure time model for clustered time-to-event data. Using comprehensive simulations, it is shown that this methodology outperforms existing methods in settings where the intrapatient correlation is low to moderate. The performance of the model is further demonstrated in a real data example from a molecularly aided tumor trial. Sample size considerations are discussed.</p>","PeriodicalId":55360,"journal":{"name":"Biometrical Journal","volume":"67 1","pages":""},"PeriodicalIF":1.3,"publicationDate":"2024-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/bimj.70028","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142848551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}