Spectral extrema of graphs with fixed size: Forbidden triangles and pentagons

Nosal (1970) and Nikiforov (2002) showed that if graph G is $C_3$-free of size m, then the spectral radius of G satisfies $\lambda \left(G\right)\le \sqrt{m}$, equality holds if and only if G is a complete bipartite graph. Lin, Ning and Wu (2021) extended this result as: If G is a $C_3$-free non-bipartite graph of size m, then $\lambda \left(G\right)\le \sqrt{m-1}$, equality holds if and only if $G\cong C_5$. This result was extended by Li, Peng (2022) and Sun, Li (2023), independently, as the following: If G is a $\{C_3,C_5\}$-free non-bipartite graph with m edges, then $\lambda \left(G\right)\le \lambda \left(S_3\right(K_{2,\frac{m-3}{2}}\left)\right)$, equality holds if and only if m is odd and $G\cong S_3\left(K_{2,\frac{m-3}{2}}\right)$, where $S_3\left(K_{2,\frac{m-3}{2}}\right)$ is obtained from $K_{2,\frac{m-3}{2}}$ by replacing one of its edges by a path of length 4. This upper bound could be attained only if m is odd, since the extremal graph $S_3\left(K_{2,\frac{m-3}{2}}\right)$ is well-defined only in this case. Thus, it is interesting to determine the spectral extremal graph when m is even. Sun and Li (2023) proposed the following question: Determine the graphs attaining the maximum spectral radius over all $\{C_3,C_5\}$-free non-bipartite graphs of even size m. In this contribution, we answer this question for $m\ge 150$. Our proof technique is mainly based on applying Cauchy's interlacing theorem of eigenvalues of a graph, and with the aid of Ning and Zhai's triangle counting lemma in terms of both eigenvalues and the size of a graph, together with the eigenvector method from Lou, Lu and Huang (2023).