This study investigates the microstructure and very high cycle fatigue (VHCF) behavior of a powder bed fusion–laser beam (PBF-LB) processed scandium-free Al-Mg-Zr alloy (commercially known as EOS Al5X1) through advanced microstructural characterization, defect analysis, ultrasonic fatigue testing, and detailed fractographic examination. The analysis focuses on defect-driven crack initiation, particularly process-induced volumetric defects such as pores, lack of fusion, and non-metallic (oxide) inclusions. Scanning electron microscopy-based fractography reveals that in the VHCF regime, where the number of cycles to failure (Nf) > 107 cycles, fatigue crack initiation predominantly shifts toward the subsurface or interior of the specimen. In multiple cases, process-induced volumetric defects facilitated crack initiation, resulting in characteristic 'fisheye' fracture morphologies. The chemical composition of these critical defects was also analyzed in detail. The study highlights the significant impact of process-induced volumetric defects on fracture morphology and examines the influence of defect size and location on VHCF performance. These findings provide deeper insight into the interplay between processing defects and crack nucleation, underscoring the necessity of advanced defect characterization to better understand VHCF life variability.