Identifying Risk and Protective Factors for Attrition Among Recently Enlisted Navy Personnel Using Variable Importance Measures.

Abstract

Introduction: Approximately 1 in 4 Navy recruits will not complete their obligated service. Identifying factors associated with Navy attrition is essential to retaining a skilled and motivated fighting force. This study utilized machine learning and variable importance measures (VIM) extracted from random forest (RF) and extreme gradient boosting (XGBoost) decision trees to identify factors that contribute to Navy attrition. These methods are more robust and sensitive when assessing hundreds of predictor variables, offering superior performance relative to regression analyses. Leveraging advanced analytics is crucial to better understand these factors to maximize military retention.

Materials and methods: The study population included Navy personnel with an accession date in 2016. The outcome of interest was a discharge date before the end of obligated service (i.e., attrition). The analysis incorporated 542 independent variables, including demographic factors, medical outpatient visits, and outpatient pharmacy medications. RF, XGBoost, and logistic regression models were trained with different subsets of features to determine which set of variables best predicted attrition. Model performance metrics were generated describing the sensitivity, specificity, positive and negative predictive values, area under the curve, and classification accuracy for the validation data.

Results: Overall, there were 39,866 Navy personnel with accessions in 2016 and, of these, 28.15% (n = 11,177) did not complete their obligated service. The RF model provided the best accuracy for predicting attrition (81.7%) and area under the curve (90.0%). Both the RF and XGBoost models outperformed the logistic regression model. VIMs were then assessed, including mean decrease accuracy, mean decrease Gini impurity, gain, and cover. The resulting VIM identified five groups described as: mental health, occupations, demographics/sex-related issues, pain management, and medical compliance. The top 3 VIM and their overall relative risk on attrition were adjustment disorders (RR = 1.39 higher), Seaman Specialists (RR = 3.01 higher), and Electronic Equipment Repairers (RR = 0.44 lower).

Conclusions: Five groups of variables were found to be predictive of Navy attrition: Mental health (MH) disorder, alcohol-related problems, occupations, sex, medical appointment compliance, and pain management. These results demonstrate the utility of machine learning models in predicting attrition compared with regression analyses. VIM is a valuable tool that could be used in decision-making processes in the context of military personnel management and retention. Furthermore, ensemble approaches, compared with a single decision tree, improve overall predictive performance and result in a more robust model that is resistant to overfitting.