Wavefront phase recovery through Gabor holography is a well-established inverse problem in quantitative phase imaging. While traditional iterative projection algorithms provide a broadly applicable solution, reconstruction quality remains a concern. Recent advances in deep learning have introduced new possibilities, though issues with generalizability and physical interpretability persist. In this work, we present a self-supervised complex-valued neural network (CVNN) model that integrates an iterative projection framework guided by physical priors. The complex-valued operations in the CVNNs enhance performance by capturing the intrinsic relationship between amplitude and phase. Notably, the complex total variation regularization is introduced to reduce artifacts and improve phase fidelity. Comprehensive analyses demonstrate that our CVNN significantly outperforms traditional iterative algorithms and previous real-valued networks in both simulated and experimental datasets. This work highlights the potential of CVNNs in quantitative phase imaging, emphasizing the benefits of incorporating physical principles into deep learning approaches for improved interpretability and performance.