An indefinite eigenvalue problem with eigenparameter in the two boundary conditions (Q1273340)

The following regular indefinite eigenvalue problem for a second-order ordinary differential equation with the eigenvalue parameter appearing in the boundary conditions is considered: \[ {1\over{r(x)}}[-(pu')'+qu]=\lambda u, \quad x\in [a,b], \] \[ \alpha_1u(a)-\alpha_2(pu')(a)=\lambda\alpha_3u(a)-\alpha_4(pu')(a), \] \[ \beta_1u(b)-\beta_2(pu')(b)=\lambda \beta_3u(b)-\beta_4(pu')(b). \] Both, the weight function \(r(x)\) and the potential function \(q(x)\) change sign on \([a,b]\) which causes the problem to be indefinite. The author proves that this problem is equivalent to the eigenvalue problem for a \(J\)-selfadjoint operator \(A\) in a suitable Krein space \(H\). He proves that \(A\) in \(H\) is unbounded from above and bounded from below and has a compact resolvent. The main result of the paper is an eigenfunction expansion theorem.