On the exceptional set of transcendental functions with integer coefficients in a prescribed set: the problems A and C of Mahler (Q2004943)

A \textit{transcendental function} is a function \( f(x) \) such that the only complex polynomial \( P \) satisfying \( P(x, f(x)) =0 \), for all \( x \) in its domain, is the zero polynomial. Trigonometric functions, the exponential function, and their inverses are some of the examples of transcendental functions. Denote by \( \bar{\mathbb{Q}} \) the field of algebraic numbers. For a function \( f \) analytic in the complex domain \( \mathcal{D} \), define the exceptional set \( S_f \) of \( f \) as \(\displaystyle{S_f=\left\{ \alpha \in \bar{\mathbb{Q}}\cap \mathcal{D}: f(\alpha) \in \bar{\mathbb{Q}} \right\}}\). For example, the exceptional sets of the functions \( 2^{z} \) and \( e^{z\pi+1} \) are \( \mathbb{Q} \) and \( \emptyset \), respectively, as shown by the Gelfond-Schneider theorem and Baker's theorem. In the paper under review, the authors consider Problem A and Problem C in the book of \textit{K. Mahler} [Lectures of transcendental numbers (1976; Zbl 0332.10019)], who suggested three problems, which he named Problem A, B and C, on the arithmetic behaviour of transcendental functions. Problems B and C have been completely solved by the authors in [Math. Ann. 368, No. 3--4, 1059--1062 (2017; Zbl 1387.11056); Bull. Aust. Math. Soc. 98, No. 1, 60--63 (2018; Zbl 1422.11162)], but Problem A remains open in general. Recall that, as usual, \( \mathbb{Z}{\{z\}} \) denotes the set of the power series analytic in the unit ball \( B(0,1) \) and with integer coefficients. Problems A and C are stated as follows. \begin{itemize} \item[A.] Does there exist a transcendental function \( f\in \mathbb{Z}{\{z\}} \) with bounded coefficients and such that \( f(\bar{\mathbb{Q}}\cap B(0,1)) \subseteq \bar{\mathbb{Q}} \)? \item[C.] Does there exist for every choice of \( S \) (closed under complex conjugation and such that \( 0\in S \)) a transcendental entire function with rational coefficients for which \( S_f=S \)? \end{itemize} In this paper, the authors generalize the main result of \textit{J. Haung}, et al. [Bull. Aust. Math. Soc. 82, No. 2, 322--327 (2010; Zbl 1204.11113)]. As a consequence, the authors improve their main result in [Acta Arith. 192, No. 4, 313--327 (2020; Zbl 1450.11078)] as well as providing a variant version of Problem A (for coefficients belonging to some zero asymptotic density sets). Recall that an \( n \)-smooth integer is an integer (possibly negative) whose prime factors are all less than or equal to \( n \). The main result in this paper is the following. Theorem. Let \( A \) be a countable subset of \( B(0,1) \) which is closed under complex conjugation. For each \( \alpha \in A \), fix a dense subset \( E_\alpha \subseteq \mathbb{C} \) (such that \( 0\in A \) , then \( 1\in E_0 \), \( E_\alpha \) is dense in \( \mathbb{R} \) whenever \( \alpha \in \mathbb{R} \), and such that \(\bar{E_\alpha} = E_{\bar{\alpha}}\), for all \( \alpha\in A \)). Then there exist uncountably many transcendental functions \( \displaystyle{f(z)=\sum_{n\ge 0}a_nz^{n} \in \mathbb{Z}\{z\}} \), such that \( a_n \) is a \( 3 \)-smooth number (for all \( n\ge 0 \)) and \( f(\alpha) \in E_\alpha \), for all \( \alpha \in A \).