Least energy nodal solutions of Hamiltonian elliptic systems with Neumann boundary conditions (Q1797837)

The paper deals with the Hamiltonian elliptic system \[ -\Delta u=|v|^{q-1}v,\quad -\Delta v=|u|^{p-1}u,\quad \] with Neumann boundary conditions in a smooth bounded domain \(\Omega\subset\mathbb{R}^N\). The coefficients \(p,q>0\) are subcritical, i.e., \((p,q)\) lies below the critical hyperbola. Both the sublinear (\(pq<1\)) and the superlinear (\(pq>1\)) case are treated. It is proved that for \(\Omega\) being a ball or an annulus, the least energy solution is not radial but foliated Schwarz symmetric. Other results are concerned with unique continuation, the regularity of solutions, and monotonicity in 1D or for least energy radial solutions. The proofs are based on variational methods applied to a dual functional. An important ingredient is a new transformation using decreasing rearrangements.