Nontrivial solutions for critical potential elliptic systems (Q633067)

Extensions of some results on Brezis-Nirenberg type problems are obtained. More precisely, the following potential elliptic system involving the \(p\)-Laplace operator is studied: \[ \begin{aligned} -\Delta_pu = f(u) + g(u) &\quad \text{in } \Omega\subset \mathbb{R}^n,\\ u= 0 &\quad \text{on } \partial\Omega, \end{aligned} \] where \(f=\frac{1}{P^*}\nabla F\), \(g = \frac{1}{p}\nabla G\), for \(F, G : \mathbb R^k \to \mathbb R\) of class \(C^1\) with \(F\) being positively homogeneous of degree \(p^*\) and \(G\) being homogeneous of degree \(p\). Necessary and sufficient conditions for the existence of a nontrivial solution to this system are presented. More precisely: {\parindent=6,5mm \begin{itemize}\item[(1)] If \(\Omega\) is star-shaped with smooth boundary and \(G\) is negative on \(\mathbb{R}^k\backslash\{0\}\), then the system has no nontrivial solution. \item[(2)] If \(n \geq p^2 > 1\), \(M_G = \max_{|t|_p=1}G(t) < \lambda_1\) and \(G\) is positive at a suitable point, then the system has a nontrivial solution. \item[(3)] If \(p^2>n>p>1\), \(M_G < \lambda_1\), and \(F, G\) satisfy suitable conditions at some point, then the system has a nontrivial solution. \end{itemize}}