In online adaptive radiotherapy (ART), quick computation-based secondary dose verification is crucial for ensuring the quality of ART plans while the patient is positioned on the treatment couch. However, traditional dose verification algorithms are generally time-consuming, reducing the efficiency of ART workflow. This study aims to develop an ultra-fast deep-learning (DL) based secondary dose verification algorithm to accurately estimate dose distributions using computed tomography (CT) and fluence maps (FMs). We integrated FMs into the CT image domain by explicitly resolving the geometry of treatment delivery. For each gantry angle, an FM was constructed based on the optimized multi-leaf collimator apertures and corresponding monitoring units. To effectively encode treatment beam configuration, the constructed FMs were back-projected to cm away from the isocenter with respect to the exact geometry of the treatment machines. Then, a 3D U-Net was utilized to take the integrated CT and FM volume as input to estimate dose. Training and validation were performed on prostate cancer cases, with an additional testing cases for independent evaluation of model performance. The proposed model can estimate dose in ∼ ms for each patient. The average γ passing rate ( , threshold) for the estimated dose was 99.9% ± 0.15% on testing patients. The mean dose differences for the planning target volume and organs at risk were and , respectively. We have developed a geometry-resolved DL framework for accurate dose estimation and demonstrated its potential in real-time online ART doses verification.