scientific article; zbMATH DE number 1145156

Publication date 26 April 1998

optical character recognition speech recognition

Introductory exposition (textbooks, tutorial papers, etc.) pertaining to computer science (68-01) Learning and adaptive systems in artificial intelligence (68T05)

Review text

Everyone who has used a speech recognition or Optical Character Recognition (OCR) software remembers their first experience with these software to be very different from the common applications. Going through the relatively short period of training steps with the software dramatically increases its accuracy in speech recognition, while OCR programs output shows also signs of learning from correction of the preceding mistakes. Intelligence is the word that comes to mind! Where does the softwares intelligent behavior comes from? The secret lies in the remarkable progress made in the subject often referred to as learning theory. Artificial neural networks (ANN) offer a popular implementation of learning-theoretic algorithms. The terminology certainly reminds one of the cellular structure of the nervous system of human and other living creatures with intelligent behavior. This choice of terminology is intentional: artificial neural networks use the metaphor of brain anatomy and in part, try to model the architecture of the nervous system. For readers unfamiliar with methods and applications of ANN, perhaps a useful analogy is to try to solve the problem of approximating a function f(t) whose graph describes the motion of a article in the Cartesian coordinates. You are given a set of values \(E = \{ f(a) , f(b) , f(c) , .\}\) where \(T = \{a, b, c, \}\) are moments in time that snap-shots of the motion of the particle are taken. \(E\) is called the set of examples or the training set. Values of any proposed solution \(f\) at times \(x\) that are not included in the given set \(T\) are called generalizations. Since the problem is under-determined, many possible functions of an argument \(t\) can take the given values in \(E\) at time-instances given in \(T\). Therefore, the generalizations of the proposed function \(f\) at times t are not well-defined. However, in practical applications, one asks for the best approximate solution, say within a prescribed error range, and provides an adequate set of examples as the initial data. Making some additional realistic assumptions, such as smoothness of the path, there is a chance to find at least one solution. A heuristic yet feasible method to find an approximate solution may proceed as follows. If you start with a proposed function \(g(t)\), then the measure of how far the function \(g(t)\) is from being an acceptable solution could be provided by the sum of squares of \(f(t)-g(t)\) at given points in \(T\). Therefore, to minimize this error one could choose a strategy to improve the choice of g by successively applying the gradient descent method. In the process, one may reach a local minimum, and have a solution that is valuable only in the proximity of a small set of example, or find a global minimum that would provide a possible answer to the approximation problem. If the reader is more ambitious, she/he might try to write a code to provide a computational solution, that is certainly the only method beyond some elementary easy cases, such as when the examples provide a set of collinear points in the plane, etc. One conclusion of a computational attempt is the overwhelming demand on computation when the number of examples is adequate for a realistic error range! One quickly concludes that a more clever approach is needed if this method is to apply to solve serious problems. Indeed, such approaches have been thoroughly investigated in the last 30 years, both for software (algorithms, code effectiveness), and hardware (parallel architectures, VLSI, embedded systems). The above-mentioned discussion is in fact the intuition for one of the most popular ANN architectures, the so-called back propagation algorithm. There are many other powerful architectures for ANNs, and the book Optical Neural Networks provides a readable introduction to some of the popular ones. What makes this book entirely distinct from a large number of other accounts on ANNs is the attempt to develop the foundation and algorithms of the subject for implementation with optical computation. The reader certainly would think of the speed of light having something to do with accelerating the demanding computations in ANNs. This is certainly true, but much more lies ahead that suggests the fundamental role that optical neural networks will potentially play in the future. The following quotes selected from the preface describes the rationale and advantages of the authors approach: What have optics to do with mathematical algorithms for neural networks? In order to compute, we need to transport data from place to place, connect components together, store data and be able to switch on and off. All these features seem to be realized in an easy way by electronic components as there are wires or conducting pathways on silicon, electrical junctions, a memory and the transistor. However, for the purpose of neural networks, there are many practical difficulties in implementation. The most obvious of these is that neural networks require by their nature an adaptive feature, which means that they are able to distinguish between states and change their decision rules themselves as necessary. It is very difficult to implement such a feature in integrated circuit technology. Moreover, the high interconnectivity also demands a lot of effort in electronics. Thus the question is obvious if there is another area in physics that is able to fulfill the requirements that neural network structures demand. Optics already have a long tradition in interacting with electronics, so why not having a look at how optics are able to fulfill these requirements. Optical computing uses light to transport information instead of electric signals. That approach holds three major advantages for computation in general and artificial neural networks in particular. The first is the inherently high speed achievable data can flow at the speed of light. Optical switches can therefore go much faster than electronic ones, where the transmission time between e.g. neurons is limited by their distance or in the case of the high speed electronic machines where additional power is used to provide speed and elements are located close to one another by the mechanism of heat transfer out of the system. Another advantage of optics is the high bandwidth that enables more information to be carried. While electronic communication along wires requires changing of a capacity that depends on its length, optical signals in fibers, optically ichange anything and are therefore inherently fast. However, for neural computing in particular, the most important reason is due to the fact that one beam of light can cross another and emerges completely unaffected by its encounter without distorting the information carried on it, whereas two electrical wires can not. This effect is commonly known as the principle of superposition in optics. It is possible because the information carrying elements of light the photons are uncharged particles, which do not interact with one another as readily do electrons. This opens up the potential for massive interconnections within a small place, especially in the case of highly parallel systems, where loops of connections are necessary. In the case of electronics, loops of that set will generate noise voltage spikes whenever the electromagnetic field through the loop changes. Furthermore, high frequency or fast switching pulses will cause influence or interference in neighboring wires. Even when they are routed in silicon, a certain distance between neighboring pathways has to be established in order to avoid interference. In contrast, signals in adjacent optical fibers or in optical interconnection channels do not interfere with each other nor do they pick up noise due to loops. The third advantage of optics is that images or arrays of light points so called pixels may be handled in parallel. Consequently, a simple lens can be thought of as a powerful interconnection device. The image it forms is a collection of rays of light reflected from different points of the object. Therefore the lens effectively connects countless of these rays from the object to the image. These reasons have led to investigations how to explore optics for computing purposes and how to realize the equivalent and on the same time more adequate and powerful components of electronic systems with optical devices. Storage in optical systems can be accomplished using holograms a sort of three dimensional photograph containing enough information to reconstruct an image of a solid object. Holograms can also be used as distributors by directing light that falls onto them in different directions depending on the initial incident angle of the beam on the hologram. The amount of information that can be stored on a hologram is high, since a single one can hold and discriminate many images. Optical switches, which use a small signal to control a larger one for producing gain as in a transistor can also be made optically. One approach is to affect a crystal structure with an electric or magnetic field, which alters its optical properties (in most cases the refractive index of the material) and so affect incoming light differently. Other switching devices use nonlinear crystals that alter the amount of light that they transmit depending on the intensity of the incoming beam. Here, best optical switching times are about 1014 seconds using e.g. the non-linearities in Gallium Arsenide devices. Thus, these devices have the speed edge even over electronic ones. To summarize, optics have the most promising potentialities of realizing neural network models which may be able to solve the sort of problems, humans deal with so easily. Therefore, I will focus my attention in this book on optical neural network implementations, composed of linear and nonlinear optical devices for storage, switching and decision purposes. To understand these systems in detail, an insight in both areas, the functional principles of artificial neural networks and in basic concepts of optics especially nonlinear optics is necessary. It will be provided in the first chapters, beginning with the basic models of neural networks. Because both optics, especially nonlinear optics, as well as neural computing are two forthcoming, rapidly expanding fields, this book will focus on the major concepts, combined with some of the most actual examples of applications, chosen mainly out of the field of photo-refractive optics. I will not be able to describe all the fascinating new concepts in both areas throughout the book, but Further Reading hints for those who want to plunge deeper into the field it are provided at the end of each chapter. This sets the tone for the remainder of this wonderful monograph that could be easily used as a textbook or a supplement reading for modern treatments of ANNS. Dr. Denz has filled an gap in literature by her contribution. The book is detailed with figures, examples, references and user-friendly iconic road-signs in 12 chapters and an extensive, yet selective bibliography. In more than 400 pages with small fonts, a wealth of information introduces the reader to what could well be one of the key technologies of the future of computation. There are a number of points where the usage of English grammar or spelling is less than perfect. Nonetheless, the reader should have no difficulty to correct these minor imperfections! The reviewer recommends reading this book to any serious practitioner or student learning theory, from computer science, engineering and many other backgrounds.

Cited in

(11)

This page was built for publication:

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4384474)