Upper bounds and asymptotics in a quantitative version of the Oppenheim conjecture (Q1130225)

Let \(Q\) be an indefinite nondegenerate quadratic form in \(n\) variables. \textit{G. Margulis} [Math. Ann. 286, 101-128 (1990; Zbl 0679.22007)] proved the Oppenheim conjecture, which states that if \(n\geq 3\) and \(Q\) is not proportional to a form with rational coefficients, then the set of values of \(Q\) at integral points is dense. Subsequently, using Ratner's measure classification theorem, as well as their own method of studying unipotent flows on homogeneous spaces via linearization, \textit{S. Dani} and \textit{G. Margulis} [Adv. Sov. Math. 16, 91-137 (1993; Zbl 0814.22003)] proved a refined version of Ratner's uniform distribution theorem. This theorem is the main ergodic result used in this paper. The main results of this paper are as follows: Let \(Q\) be an indefinite form of signature \((p,q)\) with \(p\geq 3\), \(q\geq 1\) and \(Q\) not proportional to a rational form. Then for any interval \((a,b)\) as \(T\to\infty\) \[ | V_{(a,b)}(\mathbb{Z})\cap T\Omega|\sim \lambda_{Q,\Omega} (b-a)T^{n-2}, \tag{1} \] where \(n=p+q\), \(\Omega= \{v\in\mathbb{R}^n\mid \| v\|< \rho(v/\| v\|)\}\), \(\rho\) is a continuous positive function on the unit sphere, \(T\Omega\) is the dilate of \(\Omega\) by \(T\), \(V_{(a,b)}(Z)= \{z\in\mathbb{Z}^n\mid a<Q(x)< b\}\) and \(\lambda_{Q,\Omega}\) satisfies \[ \text{Vol} (V_{(a,b)} (\mathbb{R})\cap T\Omega)\sim \lambda_{Q,\Omega} (b-a)T^{n-2}. \tag{2} \] Only the upper bound is new. If the signature of \(Q\) is \((2,1)\) or \((2,2)\), then no such universal formula exists. However an upper bound of the form \(T^q\log T\) is obtained. This upper bound is effective and is uniform over compact sets in the set of quadratic forms. An effective uniform upper bound of the form \(T^{n-2}\) for the case \(p\geq 3\) is also obtained. It is also proved that for almost all quadratic forms \(Q\) of signature \((p,q)= (2,1)\) or \((2,2)\) the asymptotic relation (1) holds. Finally, let \(p\geq 3\), \(q\geq 1\), \(D\) be a compact subset of the space of quadratic forms of signature \((p,q)\) and discriminant \(\pm 1\). Then for every interval \(\lfloor a,b\rfloor\) and every \(\theta>0\), there exists a finite subset \(P\) of \(D\) such that each \(Q\in P\) is a scalar multiple of a rational form and for every compact subset \(F\) of \(D-P\) there exists \(T_0\) such that for all \(Q\) in \(F\) and \(T\geq T_0\) \[ (1-\theta) \lambda_{Q,\Omega} (b-a)T^{n-2}\leq V_{(a,b)}(\mathbb{Z})\cap T\Omega\leq (1+\theta) \lambda_{Q,\Omega} (b-a)T^{n-2}. \] Only the upper bound is new. In this paper these problems have also been related to certain integral expressions involving the orthogonal group of the quadratic forms and the space of lattices \(SL(n,\mathbb{R})/ SL(n,\mathbb{Z})\).