Selecting near-infrared reflection spectroscopy pretreatment methods by chemical components valid and invalid absorption wavebands

Abstract

Abstract An appropriate pretreatment method is crucial to establish a reliable model for near-infrared spectroscopy. It is usually selected by comparing model performance with different pretreatment methods, such as the root mean square error of internal cross-validation or the root mean square error of prediction. When the statistical indexes of the models are similar and indistinguishable, or the results of model calibration and cross-validation are too different, indicating an overfitting situation, it may be due to the selection of an inappropriate pretreatment. In this paper, an approach is proposed to select appropriate pretreatment methods by chemical components valid and invalid absorption wavebands. Due to there being no chemical groups absorption in wavebands of 1250–1350 nm and 1780–1850 nm, they are considered chemical invalid absorption bands. The root mean square distance in the invalid wavebands characterizes the difference in samples’ physical state, such as particle size and instrument error. Except for the invalid absorption band, other wavebands in the near-infrared region are considered chemical valid absorption wavebands. The root mean square distance in the valid absorption wavebands mainly characterize the chemical composition differences between different samples. So, we define effective information rate of spectra. The larger effective information rate of spectra, the more favorable it is to establish a quantitative model. In this study, spectra of 60 wheat powder samples were used to compare seven pretreatment methods. Moreover, the average effective information rates of spectra were calculated for the seven pretreatment methods respectively. The results show that the combination of standard normal variate transformation and first derivative with Savitzky–Golay smoothing is the best. Finally, a comparison of model prediction with different pretreatment methods was made. The results are the same as the former. Therefore, using the effective information rate of spectra for selecting an appropriate near-infrared spectroscopy pretreatment method is practical and can be used not only for near-infrared modeling but also for designing near-infrared instruments as an index for optimization.