Dynamic fluctuation is a common phenomenon in degradation processes. Hence, how to model it properly has a great impact on the degradation modeling as well as the remaining useful lifetime prediction. To capture the dynamic features and to avoid the risk of the model mis-specification, a nonparametric degradation model based on functional variance process is proposed in this article. The model is composed of a unit-specific mean trend and a degradation fluctuation which follows a stochastic process. The mean trend is estimated by the local smoother method, while the stochastic fluctuation is estimated by the functional principal component analysis method. The asymptotic properties of the estimators are proved. Also, the prediction for the remaining useful lifetime is discussed and the estimator is proved to converge in distribution. Moreover, a Bayesian scheme is developed to forecast the remaining useful lifetime for units with incomplete degradation observations. Simulation results show the superiority of the proposed method by comparing it with some existing methods. Finally, two real data sets are analyzed and used to illustrate the application of the method.