Critical dimension of a Hessian equation involving critical exponent and a related asymptotic result (Q1923665)

\textit{H. Brezis} and \textit{L. Nirenberg} [Commun. Pure Appl. Math. 36, 437-477 (1983; Zbl 0541.35029)] discovered the existence of a ``critical dimension'' for the semilinear eigenvalue problem with ciritical growth: \[ -\Delta u=\lambda u+u^{(n+2)/(n-2)}, \quad u>0\quad\text{ in }\Omega, \qquad u=0\quad\text{ on }\partial\Omega.\tag{1} \] Here \(\Omega\subset \mathbb{R}^n\) \((n\geq 3)\) is a strietly starshaped domain, and \(\lambda\in\mathbb{R}\) is an eigenvalue parameter. In general dimensions, the problem (1) has a solution for \(\lambda\in (0,\lambda_1)\), where \(\lambda_1\) in the first eigenvalue of \(-\Delta\); while a dimension is called critical with respect to (1), if one has nonexistence for \(\lambda\) close to 0. Brezis and Nirenberg found, that \(n=3\) is the critical dimension for (1). In the present paper, the authors show that the Brezis-Nirenberg result can also be carried over to the nonlinear eigenvalue problems \[ S_k(D^2u)= (-u)^{\gamma(k)}+ \lambda(-u)^k, \quad u<0\quad\text{ in }B, \qquad u=0\quad\text{ on }\partial B.\tag{2} \] Here \(k\in\mathbb{N}\), \(B\subset\mathbb{R}^n\) \((n\geq 3)\) is the unit ball and \(S_k(D^2u)\) is the \(k\)-th Hessian operator: \[ S_k(D^2u)= \sum_{1\leq i_1<\dots< i_k\leq n}\lambda_{i_1}\cdot \dots\cdot \lambda_{i_k}, \] where \(\lambda_1,\dots, \lambda_n\) are the eigenvalues of the Hessian matrix \(D^2u\). For \(1\leq k<n/2\) a finite ``critical exponent'' exists: \(\gamma(k)= (n+2)^k/(n-2k)\). A ``first eigenvalue'' \(\lambda_1\) is defined with respect to the homogeneous problem corresponding to (2). As only radial solutions are considered the problem (2) may be transformed into \[ {1\over n}{n\choose k}r^{1-n} (r^{n-k}(u_r)^k)_r= (-u)^{\gamma(k)}+ \lambda(-u)^k, \;u<0, \;r\in(0,1], \;\;u=0\text{ for }r=1.\tag{3} \] It is shown that the critical dimensions \(n\) with respect to the nonlinear problem (3) are precisely those with \(2k<n< 2k(k+1)\). Moreover, for the noncritical dimensions an asymptotic expansion of \(u=u_\lambda\) is given, as \(\lambda\) approaches 0.