To accurately predict the degree of milling (DOM), contrast images were initially acquired by randomly selecting and arranging head milled rice grains alongside head brown rice grains on two distinct color cardboards. Using computer vision techniques, milled rice grains and brown rice grains within a contrast image were segmented, identified, and followed by the extraction of contour dimensions. Subsequently, the mean area ratio, length ratio, and width ratio between milled rice and brown rice were calculated, along with the mean aspect ratio of brown rice. Additionally, the color difference value between stacked milled rice and brown rice was determined using the CIEDE2000 formula. The dataset was thereby created by combining contour dimension ratios and color difference values with augmentation data generated via a multivariate linear interpolation method. Thereafter, based on evaluation results of eleven machine learning models, the support vector regression (SVR) was employed to elucidate the feature contributions to the prediction model through Shapley additive explanations (SHAP). Both macro and micro analyses unveil that the area ratio exerts the most significant impact on DOM prediction, followed by aspect ratio and color difference, while width ratio and length ratio have a comparatively minimal influence. Furthermore, the area ratio exhibits a negative effect, whereas color difference and aspect ratio demonstrate a positive effect. Ultimately, prediction tests on three additional distinct rice varieties utilizing the SVR model, achieving accuracies of 95.63% (short-grain), 93.45% (long-grain), and 95.36% (medium-grain), respectively. Moreover, tests confirm that DOM is positively correlated with the aspect ratio as unveiled by SHAP analysis.