The use of renewable materials in the epoxidation reaction has gained increasing attention due to the need to reduce reliance on non-renewable resources and minimize environmental impact. To date, there is a paucity of studies on the optimization of process to produce epoxidized oleic acid by auto-catalyst epoxidation using formic acid (by product) as catalyst as it is not fully utilised. The aim of this study is to investigate the effect of hydrogen peroxide concentration, type of oxygen carrier and stirring speed on the auto catalyst epoxidation of oleic acid. In this study, auto-catalyzed epoxidation using formic acid was applied in which formic acid acts as both a reactant and a catalyst to produce oxirane. The maximum selectivity of oleic acid into oxirane was 58% by applying the optimum epoxidation reaction parameters. Based on the Fourier transform infrared (FTIR) spectrum, the absorption peak at 1100 cm−1 indicated the presence of oxirane rings (C–O–C bonds). Lastly, a mathematical model was developed using MATLAB software. In this model, the fourth-order Runge–Kutta method was integrated with genetic algorithm optimization to determine the kinetic model that fit with the experimental data. The kinetic model was validated by the fact that there was good agreement between the simulation and experimental data.