A novel method for analyzing the reliability of structures under non-stationary stochastic seismic excitations, considering the combined effect of multiple structural demand extreme values, is proposed. The spectral representation method is employed to establish a non-stationary stochastic seismic excitation model, and based on the theory of first-passage probability, multiple integral formulas for seismic reliability under multidimensional limit states are derived. The extreme value distribution is established using the Gamma mixture model (GMM). The equations for estimating the model parameters are derived based on both fractional moments and moment-generating functions, while the determination of the number of gamma distribution components is guided by the probability distribution and statistical characteristics of the samples. The joint probability density function (JPDF) for multiple demand extreme values is established by incorporating copula functions to account for correlation, and the fitting accuracy of different copula functions is assessed. The proposed method is illustrated using reinforced concrete (RC) frame structures. The results demonstrate that the fitting accuracy of extreme value distribution can be enhanced by adjusting the number of gamma distribution components in the GMM, which exhibits high accuracy in fitting both the main and tail regions. The presence of correlation can induce variations in the JPDF, thereby exerting an influence on the failure probability. Consequently, disregarding correlation is not conducive to reliability analysis.