Exploring the influence of training sampling strategies on time-series deep learning model in hydrology

Abstract

Numerous deep-learning models have been introduced to achieve reliable predictions in hydrology. In particular, recent works have shown that leveraging abundant training samples significantly improves the generalization performance due to the increased opportunity to learn fundamental processes from the samples. However, these studies often overlook the diverse levels of importance uncertainty in sample datasets. In our exploration of the rainfall-runoff relationship through modeling, we show that applying heterogeneous weights to training samples could yield even more enhanced predictions. We systematically examine the usefulness of the curriculum learning (CL)-based approach, which can dynamically assign weights to each training sample based on the concept of easy and hard samples. We particularly develop five distinct training strategies that prioritize different subsets of the training samples. Results demonstrate that employing the “easy-preferred mode” strategy—gradually increasing the complexity of data samples from easier ones—yields improved predictions compared to a random sampling strategy, similar to those prevalent in many studies. Specifically, the “easy-hard mode” strategy, allowing a shift in priority mode from “easy-first” to “hard-first” during the training process, exhibits the most remarkable performance. Lastly, our experiment also highlights that utilizing heterogeneous weights, known as a soft weighting scheme, for each sample proves more effective in enhancing the model’s performance compared to employing binary weights, referred to as a hard weighting scheme, for distinguishing between easy and hard samples in the proposed sampling strategy. Overall, our findings support the considerable benefits of employing the training sampling strategy for deep-learning models in hydrological tasks, ultimately enhancing prediction accuracy and contributing to improved water resource management and related policy-making.