Inverse spectral problem for analytic domains. I: Balian-Bloch trace formula (Q1885628)

Let \(\Omega\) be a bounded domain in \(\mathbb{R}^n\). The inverse spectral problem asks whether the shape of \(\Omega\) is determined by the spectrum of the Laplace operator on \(\Omega\) subject to Dirichlet or Neumann boundary conditions. There are isospectral, but non-isometric pairs of domains, but these domains are neither convex and nor analytic. Many related questions in the subject are still widely open, in particular, it is still unknown whether isospectral examples with analytic boundary exist. Motivated from this inverse problem, the author establishes strong relations between the wave trace invariants and the Taylor development of the boundary in the reflection points of a non-degenerated periodic billiard trajectory in \(\Omega\). If \(\Omega\) is analytic, then a recursive formula for this Taylor development would permit to determine the shape of the domain out of spectral data, and hence solve the inverse problem. The article achieves tremendous progress for solving this very difficult task. In particular, a mathematically rigorous proof of the Balian Bloch trace formula is given. In the follow-up article [``Inverse Spectral problems for analytic domains II'', ArXiv math.SP/0111078], \textit{S. Zelditch} then used this rigorous approach to prove that analytic domains with a mirror-symmetry and an invariant bouncing orbit are determined by their Dirichlet spectrum. For further applications we refer to the survey article [\textit{S. Zelditch}, Surv. Differ. Geom. 9, 401--467 (2004; Zbl 1061.58029)] and the literature cited there. Let us explain the results of the article under review in more detail. For simplicity we will restrict to the case of dimension \(n=2\) and of Dirichlet boundary conditions. Similar results also hold in the Neumann case, and in higher dimension. Let \[ E_\Omega(t)= \cos t \sqrt{\Delta} \] denote the even part of the wave group of the Dirichlet Laplacian. Then the singular points of the distribution \(f(t):=\operatorname {Tr} E_\Omega(t)\) are contained in the set of lengths of periodic billiard trajectories. The coefficients of the singularity expansions are called the wave trace invariants. In previous work Zelditch has proved that these wave trace invariants may be obtained from the semiclassical asymptotic expansion \(k\to \infty\) for the Dirichlet resolvent \[ R(k+i\tau):=-(\Delta + (k+i\tau)^2)^{-1}. \] For simplicity, we fix a length \(L\), for which only one periodic billiard trajectory \(\gamma\) of length \(L\) exists. We furthermore assume that \(\gamma\) hits the boundary only transversally and that the boundary at the reflection points has non-zero curvature. The Dirichlet resolvent is regularized with the help of a cut-off function having support in a small neighborhood of \(L\). It already follows from the Poisson relations for manifolds with boundary [\textit{V. Guillemin} and \textit{R. B. Melrose}, Adv. Math. 32, 204--232 (1979; Zbl 0421.35082) and \textit{V. M. Petkov} and \textit{L. N. Stoyanov}, `Geometry of reflecting rays and inverse spectral problems' Chichester: Wiley (1992; Zbl 0761.35077), shortly [GM] and [PS]] that the expansion of the trace \(\operatorname{Tr} 1_\Omega R_\rho(k+i\tau)\) of this regularized Dirichlet resolvent is related to certain curvature expressions \(b_{\gamma,j}\) of the boundary at the reflection points of the billiard trajectory \(\gamma\) via the formula \[ \operatorname{Tr} 1_\Omega R_\rho(k+i\tau) \sim \operatorname{Re} e^{(ik-\tau)L}\sum_{j=1}^\infty b_{\gamma,j}k^{-j} \qquad k\to \infty. \] The goal now is to obtain explicit formula for \(b_{\gamma,j}\) in terms of curvature data close to the reflection points. In principle, one could obtain such a formula for \(b_{\gamma,j}\) with the method of stationary phase as described in [GM], [PS] or via the Birkhoff normal form for \(\Delta\) at \(\gamma\) [\textit{S. Zelditch}, Math. Res. Lett. 6, No. 3--4, 457--464 (1999; Zbl 0960.58017); Geom. Funct. Anal. 10, No. 3, 628--677 (2000; Zbl 0961.58012); \textit{J. Sjöstrand} and \textit{M. Zworski}, J. Math. Pures Appl., IX. Sér. 81, No. 1, 1--33 (2002; Zbl 1038.58033); \textit{A. Iantchenko, J. Sjöstrand} and \textit{M. Zworski}, Math. Res. Lett. 9, No. 2--3, 337--362 (2002; Zbl 1258.35208)]. However, the author applies a method of \textit{R. Balian} and \textit{C. Bloch} [Ann. Phys. 60, 401--447 (1970; Zbl 0207.40202)] to the situation as they are more effective. This method is particularly effective if \(\gamma\) is a bouncing orbit. The starting point for the Balian-Bloch method is to write the Green's kernel of \(\Delta +(k+i\tau)^2\) as a Neumann series in terms of the Green's function for the Laplacian on \(\mathbb{R}^2\).