An algorithm to assess importance of predictors in systematic reviews of prediction models: a case study with simulations.

Abstract

Background: How to assess the importance of predictors in systematic reviews (SR) of prediction models remains largely unknown. The commonly used indicators of importance for predictors in individual models include parameter estimates, information entropy, etc., but they cannot be quantitatively synthesized through meta-analysis.

Methods: We explored the synthesis method of the importance indicators in a simulation study, which mainly solved the following four methodological issues: (1) whether to synthesize the original values of the importance indicators or the importance ranks; (2) whether to normalize the importance ranks to a same dimension; (3) whether and how to impute the missing values in importance ranks; and (4) whether to weight the importance indicators according to the sample size of the model during synthesis. Then we used an empirical SR to illustrate the feasibility and validity of the synthesis method.

Results: According to the simulation experiments, we found that ranking or normalizing the values of the importance indicators had little impact on the synthesis results, while imputation of missing values in the importance ranks had a great impact on the synthesis results due to the incorporation of variable frequency. Moreover, the results of means and weighted means of the importance indicators were similar. In consideration of accuracy and interpretability, synthesis of the normalized importance ranks by weighted mean was recommended. The synthesis method was used in the SR of prediction models for acute kidney injury. The importance assessment results were approved by experienced nephrologists, which further verified the reliability of the synthesis method.

Conclusions: An importance assessment of predictors should be included in SR of prediction models, using the weighted mean of importance ranks normalized to a same dimension in different models.