Positivity of the shape Hessian and instability of some equilibrium shapes (Q2568728)

An energy \(E\) over open sets \(\omega\subset{\mathbf R}^2\) is defined by \[ E(\omega)=J(\omega)+\tau P(\omega)-\Lambda S(\omega), \] where \(\tau>0\), \(\Lambda\in\mathbf R\), \(P(\omega)\) is the perimeter of \(\omega\), \(S(\omega)\) denotes the Lebesgue measure of \(\omega\), and \(J(\omega)=\int_{\omega}{1\over 2} |\nabla u_{\omega}|^2-fu_{\omega}\) is the Dirichlet energy associated to the solution of the Dirichlet problem \[ u_{\omega}\in H_{0}^1(\omega), \quad -\Delta u_{\omega}=f\text{ in }\omega, \] for a given function \(f:{\mathbf R}^2\to{\mathbf R}\). The constant \(\Lambda\) is the Lagrange multiplier associated to the functional \(J(\omega)+\tau P(\omega)\) under the constraint \(S(\omega)=S_{0}\). The authors study in this paper the second derivative of the functional \(E\) around \`\` critical shapes\'\', i.e., the critical points of \(E\), and prove that any convex or nearly convex equilibrium is \`\` stable\'\'\ (the second derivative of \(E\) is strictly positive amongst variations preserving the area of the set). Furthermore, they give some examples where positivity of the second derivative occurs while \`\` small\'\'\ perturbations, which are not normal perturbations of the set, may strictly decrease the energy.