An all-optical method for precisely measuring of residual stress in the submillimeter depth of shot-peened structures is proposed, based on laser-induced Rayleigh waves. First, a finite element analysis is conducted to elucidate the correlation between Rayleigh wave velocity and surface roughness. The velocity of Rayleigh waves in a stress-relieved, shot-peened specimen is then established as a baseline, effectively eliminating the influence of microstructural alterations such as grain refinement and work hardening on the Rayleigh wave velocity. By numerically simulating velocity variations across different stress levels, the acoustoelastic constant of Rayleigh waves in TB6 titanium alloy is accurately determined. Additionally, the optimized frequency of Rayleigh waves is identified, enabling the precise measurement of average residual stress within the shot-peening depth. In this study, the complex interaction between surface roughness and microstructural changes on Rayleigh wave velocity is rigorously controlled through meticulous experimental design, ensuring accurate residual stress measurements using an all-optical approach. The average residual stress, quantified using laser-induced Rayleigh waves under varying shot-peening intensities, aligns closely with results from X-ray diffraction and blind hole drilling methods, demonstrating the high efficacy and reliability of the proposed methodology.