Convexity theorems for generalized Riemann derivatives (Q919470)

Let \(f:[a,b]\to R\) be continuous, and \(a_ i,b_ i\in R\) \((i=1,...,n)\) satisfying \(\sum^{n}_{1}a_ i=\) \(\sum^{n}_{1}a_ ib_ i=0,\) \(\sum^{n}_{1}a_ ib_ i^ 2=1\) be given. Consider \(D^ 2f(x)=\lim_{h\to 0}\frac{2}{h}\cdot \sum^{n}_{1}a_ if(x+b_ ih),\) and let \(\underline D^ 2f(x)\), \(\underline D^ 2_+f(x)\) be similarly defined by replacing \(\lim_{h\to 0}\) by \(\liminf_{h\to 0}\) and \(\liminf_{h\to 0^+},\) respectively. Let \(F_ f^ 2\) consist of all \(x\in [a,b]\) such that f is not convex in any relatively open subinterval containing x. The results of the paper concern connections between the sign of \(\underline D^ 2\) or \(\underline D^ 2_+\) and the structure of \(F_ f^ 2\). Thus if \(\underline D^ 2_+f(x)\geq 0\) for \(x\in (a,b),\) then \(F_ f^ 2\) is nowhere dense. In the remaining results, let \(n=3.\) If \(b_ 2=0\) and \(\underline D^ 2_+f(x)\geq 0\) (always for \(x\in (a,b)),\) then f is convex; if \(b_ 2\neq 0,\quad F\subset [a,b]\) is closed, nowhere dense, and neither a nor b are isolated points of F, there is f such that \(\underline D^ 2_+f(x)\geq 0\) and \(F=F_ f^ 2.\) If \(b_ 1<0<b_ 2<b_ 3,\quad \underline D^ 2_+f(x)\geq 0\) and \(\bar f'_+>-\infty,\) then f is convex. If \(\underline D^ 2f(x)\geq 0,\) then the set of isolated points of \(F_ f^ 2\) is dense in \(F_ f^ 2\). If \(\sum^{3}_{1}a_ i \ln | b_ i| \geq 0\quad (\ln 0=-\infty)\) and \(\underline D^ 2f(x)\geq 0,\) then f is convex. If \(\sum^{3}_{1}a_ i \ln | b_ i| <0,\quad F\subset [a,c]\) is closed, nowhere dense, neither a nor b are isolated points of F and the set of isolated points of F is dense in F, then there is f such that \(D^ 2f(x)>0\) and \(F=F_ f^ 2.\) If \(b_ 1<0<b_ 3\) and \(\underline D^ 2_+f(x)=0,\) then f is linear (answer to a problem of \textit{J. M. Ash} [Real Anal. Exch. 12, 366-371 (1987; Zbl 0647.26007)]).