A spectral extremal problem on non-bipartite triangle-free graphs (Q6194251)

Summary: A theorem of \textit{E. Nosal} [Eigenvalues of graphs. Calgary: University of Calgary (Master's Thesis) (1970)] and \textit{V. Nikiforov} [Comb. Probab. Comput. 11, No. 2, 179--189 (2002; Zbl 1005.05029)] states that if \(G\) is a triangle-free graph with \(m\) edges, then \(\lambda(G)\leqslant \sqrt{m} \), equality holds if and only if \(G\) is a complete bipartite graph. A well-known spectral conjecture of \textit{B. Bollobás} and \textit{V. Nikiforov} [J. Comb. Theory, Ser. B 97, No. 5, 859--865 (2007; Zbl 1124.05058)] asserts that if \(G\) is a \(K_{r+1}\)-free graph with \(m\) edges, then \(\lambda_1^2(G) + \lambda_2^2(G) \leqslant (1-\frac{1}{r})2m\). Recently, \textit{H. Lin} et al. [Comb. Probab. Comput. 30, No. 2, 258--270 (2021; \url{doi:10.1017/S0963548320000462})] confirmed the conjecture in the case \(r=2\). Using this base case, they proved further that \(\lambda (G)\leqslant \sqrt{m-1}\) for every non-bipartite triangle-free graph \(G\), with equality if and only if \(m=5\) and \(G=C_5\). Moreover, \textit{M. Zhai} and \textit{J. Shu} [Discrete Math. 345, No. 1, Article ID 112630, 10 p. (2022; Zbl 1476.05130)] presented an improvement by showing \(\lambda (G) \leqslant \beta (m)\), where \(\beta(m)\) is the largest root of \(Z(x):=x^3-x^2-(m-2)x+m-3\). The equality in Zhai-Shu's result holds only if \(m\) is odd and \(G\) is obtained from the complete bipartite graph \(K_{2,\frac{m-1}{2}}\) by subdividing exactly one edge. Motivated by this observation, Zhai and Shu [loc.cit.] proposed a question to find a sharp bound when \(m\) is even. We shall solve this question by using a different method and characterize three kinds of spectral extremal graphs over all triangle-free non-bipartite graphs with even size. Our proof technique is mainly based on applying Cauchy's interlacing theorem of eigenvalues of a graph, and with the aid of a triangle counting lemma in terms of both eigenvalues and the size of a graph.