A comprehensive review for industrial applicability of artificial neural networks

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

585

A Comprehensive Review for Industrial Applicability of Artificial Neural Networks Magali R. G. Meireles, Paulo E. M. Almeida, Student Member, IEEE, and Marcelo Godoy Simões, Senior Member, IEEE

Abstract—This paper presents a comprehensive review of the industrial applications of artificial neural networks (ANNs), in the last 12 years. Common questions that arise to practitioners and control engineers while deciding how to use NNs for specific industrial tasks are answered. Workable issues regarding implementation details, training and performance evaluation of such algorithms are also discussed, based on a judiciously chronological organization of topologies and training methods effectively used in the past years. The most popular ANN topologies and training methods are listed and briefly discussed, as a reference to the application engineer. Finally, ANN industrial applications are grouped and tabulated by their main functions and what they actually performed on the referenced papers. The authors prepared this paper bearing in mind that an organized and normalized review would be suitable to help industrial managing and operational personnel decide which kind of ANN topology and training method would be adequate for their specific problems. Index Terms—Architecture, industrial control, neural network (NN) applications, training.

I. INTRODUCTION

I

N engineering and physics domains, algebraic and differential equations are used to describe the behavior and functioning properties of real systems and to create mathematical models to represent them. Such approaches require accurate knowledge of the system dynamics and the use of estimation techniques and numerical calculations to emulate the system operation. The complexity of the problem itself may introduce uncertainties, which can make the modeling nonrealistic or inaccurate. Therefore, in practice, approximate analysis is used and linearity assumptions are usually made. Artificial neural networks (ANNs) implement algorithms that attempt to achieve a neurological related performance, such as learning from experience, making generalizations from similar situations and judging states where poor results were achieved in the past. ANN history begins in the early 1940s. However, only in the mid-1980s these algorithms became scientifically sound and caManuscript received October 23, 2001; revised September 20, 2002. Abstract published on the Internet March 4, 2003. This work was supported by the National Science Foundation under Grant ECS 0134130. M. R. G. Meireles was with the Colorado School of Mines, Golden, CO 80401 USA. She is now with the Mathematics and Statistics Department, Pontific Catholic University of Minas Gerais, 30.535-610 Belo Horizonte, Brazil (e-mail: [email protected]). P. E. M. Almeida was with the Colorado School of Mines, Golden, CO 80401 USA. He is now with the Federal Center for Technological Education of Minas Gerais, 30.510-000 Belo Horizonte, Brazil (e-mail: [email protected]). M. G. Simões is with the Colorado School of Mines, Golden, CO 80401 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TIE.2003.812470

pable of application. Since the late 1980s, ANN started to be utilized in a plethora of industrial applications. Nowadays, ANN are being applied to a lot of real world, industrial problems, from functional prediction and system modeling (where physical processes are not well understood or are highly complex), to pattern recognition engines and robust classifiers, with the ability to generalize while making decisions about imprecise input data. The ability of ANN to learn and approximate relationships between input and output is decoupled from the size and complexity of the problem [49]. Actually, as relationships based on inputs and outputs are enriched, approximation capability improves. ANN offers ideal solutions for speech, character and signal recognition. There are many different types of ANN. Some of the more popular include multilayer perceptron (MLP) (which is generally trained with the back-propagation of error algorithm), learning vector quantization, radial basis function (RBF), Hopfield and Kohonen, to name a few. Some ANN are classified as feed forward while others are recurrent (i.e., implement feedback) depending on how data is processed through the network. Another way of classifying ANN types is by their learning method (or training), as some ANN employ supervised training, while others are referred to as unsupervised or self-organizing. This paper concentrates on industrial applications of neural networks (NNs). It was found that training methodology is more conveniently associated with a classification of how a certain NN paradigm is supposed to be used for a particular industrial problem. There are some important questions to answer, in order to adopt an ANN solution for achieving accurate, consistent and robust modeling. What is required to use an NN? How are NNs superior to conventional methods? What kind of problem functional characteristics should be considered for an ANN paradigm? What kind of structure and implementation should be used in accordance to an application in mind? This article will follow a procedure that will bring such managerial questions together and into a framework that can be used to evaluate where and how such technology fits for industrial applications, by laying out a classification scheme by means of clustered concepts and distinctive characteristics of ANN engineering. II. NN ENGINEERING Before laying out the foundations for choosing the best ANN topology, learning method and data handling for classes of industrial problems, it is important to understand how artificial intelligence (AI) evolved with required computational resources.

0278-0046/03$17.00 © 2003 IEEE

586

Artificial intelligence applications moved away from laboratory experiments to real world implementations. Therefore, software complexity also became an issue since conventional Von Neumann machines are not suitable for symbolic processing, nondeterministic computations, dynamic execution, parallel, distributed processing, and management of extensive knowledge bases [118]. In many AI applications, the knowledge needed to solve a problem may be incomplete, because the source of the knowledge is unknown at the time the solution is devised, or the environment may be changing and cannot be anticipated at design time. AI systems should be designed with an open concept that allows continuous refinement and acquisition of new knowledge. There exist engineering problems for which finding the perfect solution requires a practically impossible amount of resources and an acceptable solution would be fine. NNs can give good solutions for such classes of problems. Tackling the best ANN topology, learning method and data handling themselves become engineering approaches. The success of using ANN for any application depends highly on the data processing, (i.e., data handling before or during network operation). Once variables have been identified and data has been collected and is ready to use, one can process it in several ways, to squeeze more information out of and filter it. A common technique for coping with nonnormal data is to perform a nonlinear transform to the data. To apply a transform, one simply takes some function of the original variable and uses the functional transform as a new input to the model. Commonly used nonlinear transforms include powers, roots, inverses, exponentials, and logarithms [107]. A. Assessment of NN Performance An ANN must be used in problems exhibiting knottiness, nonlinearity, and uncertainties that justify its utilization [45]. They present the following features to cope with such complexities: • learning from training data used for system identification; finding a set of connection strengths will allow the network to carry out the desired computation [96]; • generalization from inputs not previously presented during the training phase; by accepting an input and producing a plausible response determined by the internal ANN connection structure makes such a system robust against noisy data, features exploited in industrial applications [59]; • mapping of nonlinearity making them suitable for identification in process control applications [90]; • parallel processing capability, allowing fast processing for large-scale dynamical systems; • applicable to multivariable systems; they naturally process many inputs and have many outputs. • used as a black-box approach (no prior knowledge about a system) and implemented on compact processors for space and power constrained applications. In order to select a good NN configuration, there are several factors to take into consideration. The major points of interest

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

regarding the ANN topology selection are related to network design, training, and practical considerations [25]. B. Training Considerations Considerations, such as determining the input and output variables, choosing the size of the training data set, initializing network weights, choosing training parameter values (such as learning rate and momentum rate), and selecting training stopping criteria, are important for several network topologies. There is no generic formula that can be used to choose the parameter values. Some guidelines can be followed as an initial trial. After a few trials, the network designer should have enough experience to set appropriate criteria that suit a given problem. The initial weights of an NN play a significant role in the convergence of the training method. Without a priori information about the final weights, it is a common practice to initialize all weights randomly with small absolute values. In linear vector quantization and derived techniques, it is usually required to renormalize the weights at every training epoch. A critical parameter is the speed of convergence, which is determined by the learning coefficient. In general, it is desirable to have fast learning, but not so fast as to cause instability of learning iterations. Starting with a large learning coefficient and reducing it as the learning process proceeds, results in both fast learning and stable iterations. The momentum coefficients are, usually, set according to a schedule similar to the one for the learning coefficients [128]. Selection of training data plays a vital role in the performance of a supervised NN. The number of training examples used to train an NN is sometimes critical to the success of the training process. If the number of training examples is not sufficient, then the network cannot correctly learn the actual input–output relation of the system. If the number of training examples is too large, then the network training time will be longer. For some applications, such as real-time adaptive neural control, training time is a critical variable. For others, such as training the network to perform fault detection, the training can be performed off-line and more training data are preferred, over using insufficient training data to achieve greater network accuracy. Generally, rather than focusing on volume, it is better to concentrate on the quality and representational nature of the data set. A good training set should contain routine, unusual and boundary-condition cases [8]. Popular criteria used to stop network training are small meansquare training error and small changes in network weights. Definition about how small is usually up to the network designer and is based on the desired accuracy level of the NN. Using as an example a motor bearing fault diagnosis process, they used a learning rate of 0.01 and momentum of 0.8; the training was stopped when the root mean-square error of the training set or the change in network weights was sufficiently small for that application (less than 0.005) [72]. Therefore, if any prior information about the relationship between inputs and outputs is available and used correctly, the network structure and training time can be reduced and the network accuracy can be significantly improved.

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

TABLE I ORGANIZATION OF NNS BASED ON THEIR FUNCTIONAL CHARACTERISTICS

C. Network Design Some of the design considerations include determining the number of input and output nodes to be used, the number of hidden layers in the network and the number of hidden nodes used in each hidden layer. The number of input nodes is typically taken to be the same as the number of state variables. The number of output nodes is typically the number that identifies the general category of the state of the system. Each node constitutes a processing element and it is connected through various weights to other elements. In the past, there was a general practice of increasing the number of hidden layers, to improve training performance. Keeping the number of layers at three and adjusting the number of processing elements in the hidden layer, can achieve the same goal. A trial-and-error approach is usually used to determine the number of hidden layer processing elements, starting with a low number of hidden units and increasing this number as learning problems occur. Even though choosing these parameters is still a trial-and-error process, there are some guidelines that can be used, (i.e., testing the network’s performance). It is a common practice to choose a set of training data and a set of testing data that are statistically significant and representative of the system under consideration. The training data set is used to train the NN, while the testing data is used to test the network performance, after the training phase finishes. D. Practical Considerations Practical considerations regarding the network accuracy, robustness and implementation issues must be addressed, for real-world implementation. For ANN applications, it is usually considered a good estimation performance when pattern recognition achieves more than 95% of accuracy in overall and comprehensive data recalls [25]. Selection and implementation of the network configuration needs to be carefully studied since it is desirable to use the smallest possible number of nodes while maintaining a suitable level of conciseness. Pruning algorithms try to make NNs smaller by trimming unnecessary links or units, so the cost of runtime, memory and hardware implementation can be minimized and generalization is improved. Depending on the application, some system functional characteristics are important in deciding which ANN topology should be used [81]. Table I summarizes the most common ANN structures used for pattern recognition, associative memory, optimization, function approximation, modeling and control, image processing, and classification purposes.

587

III. EVOLUTION OF UNDERLYING FUNDAMENTALS THAT PRECEDED INDUSTRIAL APPLICATIONS While there are several tutorials and reviews discussing the full range of NNs topologies, learning methods and algorithms, the authors in this paper intend to cover what had actually been applied to industrial applications. An initial historical perspective is important to get the picture for the age of industrial applications, which started in 1988, just after the release of [123] by Widrow. It is well known that the concept of NNs came into existence around the Second World War. In 1943, McCulloch and Pitts proposed the idea that a mind-like machine could be manufactured by interconnecting models based on behavior of biological neurons, laying out the concept of neurological networks [77]. Wiener gave this new field the popular name cybernetics, whose principle is the interdisciplinary relationship among engineering, biology, control systems, and brain functions [125]. At that time, computer architecture was not fully defined and the research led to what is today defined as the Von Neumann-type computer. With the progress in research on the brain and computers, the objective changed from the mind-like machine to manufacturing a learning machine, for which Hebb’s learning model was initially proposed [53]. In 1958, Rosenblatt from the Cornell Aeronautical Laboratory put together a learning machine, called the “perceptron.” That was the predecessor of current NNs. He gave specific design guidelines used by the early 1960s [91]. Widrow and Hoff proposed the “ADALINE” (ADAptive LINear Element), a variation on the pPerceptron, based on a supervised learning rule (the “error correction rule”) which could learn in a faster and more accurate way: synaptic strengths were changed in proportion to the error (what the output is and what it should have been) multiplied by the input. Such a scheme was successfully used for echo cancellation in telephone lines and is considered to be the first industrial application of NNs [124]. During the 1960s, the forerunner for current associative memory systems was the work of Steinbuch with his “Learning Matrix,” which was a binary matrix accepting a binary vector as input, producing a binary vector as output and capable of forming associations between pairs with a Boolean Hebbian learning procedure [108]. The perceptron received considerable excitement, when it was first introduced, because of its conceptual simplicity. The ADALINE is a weighted sum of the inputs, together with a least-mean-square (LMS) algorithm to adjust the weights and to minimize the difference between the desired signal and the actual output. Because of the rigorous mathematical foundation of the LMS algorithm, ADALINE has become a powerful tool for adaptive signal processing and adaptive control, leading to work on competitive learning and self-organization. However, Minsky and Papert proved mathematically that the Perceptron could not be used for a class of problems defined as nonseparable logic functions [80]. Very few investigators conducted research on NNs during the 1970s. Albus developed his adaptive “Cerebellar Model Articulation Controller” (CMAC), which is a distributed table-lookup system based on his view of models of human memory [1]. In 1974, Werbos originally developed the backpropagation algorithm. Its first practical application was to estimate a dynamic

588

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

model, to predict nationalism and social communications [120]. However, his work remained almost unknown in the scientific community for more than ten years. In the early 1980s, Hopfield introduced a recurrent-type NN, which was based on Hebbian learning law. The model consisted of a set of first-order (nonlinear) differentiable equations that minimize a certain energy function [55]. In the mid-1980s, backpropagation was rediscovered by two independent groups led by Parker and Rumelhart et al., as the learning algorithm of feedforward NNs [88], [95]. Grossberg and Carpenter made significant contributions with the “Adaptive Resonance Theory” (ART) in the mid-1980s, based on the idea that the brain spontaneously organizes itself into recognition codes and neurons organize themselves to tune various and specific patterns defined as self-organizing maps [20]. The dynamics of the network were modeled by first order differentiable equations based on implementations of pattern clustering algorithms. Furthermore, Kosko extended some of the ideas of Grossberg and Hopfield to develop his adaptive “Bi-directional Associative Memory” (BAM) [67]. Hinton, Sejnowski, and Ackley developed the “Boltzmann Machine,” which is a kind of Hopfield net that settles into solutions by a simulated annealing process as a stochastic technique [54]. Broomhead and Lowe first introduced “RBF networks” in 1988 [15]. Although the basic idea of RBF was developed before, under the name method of potential function, their work opened another NN frontier. Chen proposed functional-link networks (FLNs), where a nonlinear functional transform of the network inputs aimed lower computational efforts and fast convergence [22]. The 1988 DARPA NN study listed various NN applications supporting the importance of such technology for commercial and industrial applications and triggering a lot of interest in the scientific community, leading eventually to applications in industrial problems. Since then, the application of ANN to sophisticated systems has skyrocketed. NNs found widespread relevance for several different fields. Our literature review showed that practical industrial applications were reported in peer-reviewed engineering journals from as early as 1988. Extensive use has been reported in pattern recognition and classification for image and speech recognition, optimization in planning of actions, motions, and tasks and modeling, identification, and control. Fig. 1 shows some industrial applications of NNs reported in the last 12 years. The main purpose here is to give an idea of the most used ANN topologies and training algorithms and to relate them to common fields in the industrial area. For each entry, the type of the application, used ANN topology, implemented training algorithm, and the main authors are presented. The collected data give a good picture of what has actually migrated from academic research to practical industrial fields and shows some of the authors and groups responsible for this migration. IV. DILEMMATIC PIGEONHOLE OF NEURAL STRUCTURES Choosing an ANN solution for an immediate application is a situation that requires a choice between options that are (or seem) equally unfavorable or mutually exclusive. Several issues must be considered when regarding the problem point of view. The main features of an ANN can be classified as follows [81].

• Topology of the networks: multilayered, single-layered, or recurrent. The network is multilayered if it has distinct layers such as input, hidden and output. There are no connections among the neurons within the same layer. If each neuron can be connected with every other neuron in the network through directed edges, except the output node, this network is called single layered (i.e., there is no hidden layer). A recurrent network distinguishes itself from the feedforward topologies in that it has at least one feedback loop [49]. • Data flow: recurrent or nonrecurrent. A nonrecurrent or feedforward model where the outputs always propagate from left to right in the diagrams. The outputs from the input layer neurons propagate to the right, becoming inputs to the hidden layer neurons and then, outputs from the hidden layer neurons propagate to the right, becoming inputs to the output layer neurons. An NN, in which the outputs can propagate in both directions, forward and backward, is called a recurrent model. • Types of input values: binary, bipolar or continuous. Neurons in an artificial network can be defined to process different kinds of signals. The most common types are binary (restricted to either 0 or 1), bipolar (either 1 or 1) and continuous (continuous real numbers in a certain range). • Forms of activation: linear, step, or sigmoid. Activation functions will define the way neurons will behave inside the network structure and, therefore, the kind of relationship that will occur between input and output signals. A common classification of ANNs is based on the way in which their elements are interconnected. There is a study that presents the approximate percentage of network utilization as: MLP, 81.2%; Hopfield, 5.4%; Kohonen, 8.3%; and the others, 5.1% [49]. This section will cover the main types of networks that have been used in industrial applications, in a reasonable number of reports and applications. A comprehensive listing of all available ANN structures and topologies is out of the scope of this discussion. A. MLPs In this structure, each neuron output is connected to every neuron in subsequent layers connected in cascade with no connections between neurons in the same layer. A typical diagram of this structure is detailed in Fig. 2. MLP has been reported in several applications. Some examples are speed control of dc motors [94], [117], diagnostics of induction motor faults [24], [25], [41], [42], induction motor control [17], [18], [56], [127], and current regulator for pulsewidth-modulation (PWM) rectifiers [31]. Maintenance and sensor failure detection was reported in [82], check valves operating in a nuclear power plant [57], [114], and vibration monitoring in rolling element bearings [2]. It was widely applied in feedback control [19], [40], [52], [59], [87], [89], [109], [110] and fault diagnosis of robotic systems [116]. This structure was also used in a temperature control system

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

589

Fig. 1. Selected industrial applications reported since 1989.

[63], [64], monitoring feed water flow rate and component thermal performance of pressurized water reactors [61], and

fault diagnosis in a heat exchanger continuous stirred tank reactor system [102]. It was used in a controller for turbo

590

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

Fig. 2. MLP basic topology. Fig. 4. Hopfield network structure.

Fig. 3.

Typical recurrent network structure.

generators [117], digital current regulation of inverter drivers [16], and welding process modeling and control [4], [32]. The MLP was used in modeling chemical process systems [12], to produce quantitative estimation of concentration of chemical components [74], and to select powder metallurgy materials and process parameters [23]. Optimization of the gas industry was reported by [121], as well as prediction of daily natural gas consumption needed by gas utilities [65]. The MLP is indeed the most used structure and spread out across several disciplines like identification and defect detection on woven fabrics [99], prediction of paper cure in the papermaking industry [39], controller steering backup truck [85], and modeling of plate rolling processes [46]. B. Recurrent NNs (RNNs) A network is called recurrent when the inputs to the neurons come from external input, as well as from the internal neurons, consisting of both feed forward and feedback connections between layers and neurons. Fig. 3 shows such a structure; it was demonstrated that recurrent network could be effectively used for modeling, identification, and control of nonlinear dynamical systems [83]. The trajectory-tracking problem, of controlling the nonlinear dynamic model of a robot, was evaluated using an RNN; this network was used to estimate the dynamics of the system and its inverse dynamic model [97]. It was also used to control robotic

manipulators, facilitating the rapid execution of the adaptation process [60]. A recurrent network was used to approximate a trajectory tracking to a very high degree of accuracy [27]. It was applied to estimate the spectral content of noisy periodic waveforms that are common in engineering processes [36]. The Hopfield network model is the most popular type of recurrent NN. It can be used as associative memory and can also be applied to optimization problems. The basic idea of the Hopfield network is that it can store a set of exemplar patterns as multiple stable states. Given a new input pattern, which may be partial or noisy, the network can converge to one of the exemplar patterns nearest to the input pattern. As shown in Fig. 4, a Hopfield network consists of a single layer of neurons. The network is recurrent and fully interconnected (every neuron in the network is connected to every other neuron). Each input/output takes a discrete bipolar value of either 1 or 1 [81]. A Hopfield network was used to indicate how to apply it to the problem of linear system identification, minimizing the least square of error rates of estimates of state variables [29]. A modified Hopfield structure was used to determine the imperfection by the degree of orthogonality between the automated extracted feature, from the send-through image and the class feature of early good samples. The performance measure used for such an automatic feature extraction is based on a certain mini-max cost function useful for image classification [112]. Simulation results illustrated the Hopfield network’s use, showing that this technique can be used to identify the frequency transfer functions of dynamic plants [28]. An approach to detect and isolate faults in linear dynamic systems was proposed and systems parameters were estimated by Hopfield network [105]. C. Nonnrecurrent Unsupervised Kohonen Networks A Kohonen network is a structure of interconnected processing units that compete for the signal. It uses unsupervised learning and consists of a single layer of computational nodes (and an input layer). This type of network uses lateral feedback, which is a form of feedback whose magnitude is dependent on the lateral distance from the point of application. Fig. 5 shows the architecture with two layers. The first is the input layer and the second is the output layer, called the Kohonen layer.

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

591

Fig. 7. Adaptive resonance theory network. Fig. 5.

Fig. 6.

Kohonen network structure.

CMAC network structure.

Every input neuron is connected to every output neuron with its associated weight. The network is nonrecurrent, input information propagates only from the left to right. Continuous (rather than binary or bipolar) input values representing patterns are presented sequentially in time through the input layer, without specifying the desired output. The output neurons can be arranged in one or two dimensions. A neighborhood parameter, or radius, , can be defined to indicate the neighborhood of a specific neuron. It has been used as a self-organization map for classification [98] and pattern recognition purposes in general.

D. CMAC The input mapping of the CMAC algorithm can be seen as a set of multidimensional interlaced receptive fields, each one with finite and sharp borders. Any input vector to the network excites some of these fields, while the majority of the receptive fields remain unexcited, not contributing to the corresponding output. On the other hand, the weighted average of the excited receptive fields will form the network output. Fig. 6 shows a

schematic diagram of this structure. This figure depicts the nonlinear input mapping in the Albus approach and a hashing operation that can be performed to decrease the amount of memory needed to implement the receptive fields. CMAC networks are considered local algorithms because, for a given input vector, only a few receptive fields will be active and contribute to the corresponding network output [3]. In the same way, the training algorithm for a CMAC network should affect only the weights corresponding to active fields, excluding the majority of inactive fields in the network. This increases the efficiency of the training process, minimizing the computational efforts needed to perform adaptation in the whole network. CMAC was primarily applied to complex robotic systems involving multiple feedback sensors and multiple command variables. Common experiments involved control of position and orientation of an object using a video camera mounted at the end of a robot arm and moving objects with arbitrary orientation relative to the robot [79]. This network was also used for air-to-fuel ratio control of automotive fuel-injection systems. Experimental results showed that the CMAC is very effective in learning the engine nonlinearities and in dealing with the significant time delays inherent in engine sensors [76]. E. Adaptive Resonance Theory (ART) (Recurrent, Unsupervised) The main feature of ART, when compared to other similar structures, is its ability to not forget after learning. Usually, NNs are not able to learn new information without damaging what was previously ascertained. This is caused by the fact that when a new pattern is presented to an NN in the phase of learning, the network tries modifying the weights at node inputs, which only represent what was previously learned. The ART network is recurrent and self-organizing. Its structure is shown in Fig. 7. It has two basic layers and no hidden layers. The input layer is also called “comparing” while the output layer is called “recognizing.” This network is composed of two completely interconnected layers in both directions. [58]. It was successfully used for sensor pattern interpretation problems [122], among others.

592

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

Fig. 9. Probabilistic ANN structure. Fig. 8. RBF network structure.

F. RBF Networks Fig. 8 shows the basic structure of an RBF network. The input nodes pass the input values to the connecting arcs and the first layer connections are not weighted. Thus, each hidden node receives each input value unaltered. The hidden nodes are the RBF units. The second layer of connections is weighted and the output nodes are simple summations. This network does not extend to more layers. For applications such as fault diagnosis, RBF networks offer advantages over MLP. It is faster to train because training of the two layers is decoupled [70]. This network was used to improve the quantity and the quality of the galvanneale sheet produced on galvanizing lines, integrating various approaches, including quality monitoring, diagnosis, control, and optimization methods [13]. RBF was trained to evaluate and compare the different grasping alternatives by a robotic hand, according to geometrical and technological aspects of object surfaces, as well as the specific task to be accomplished. The adoption of RBF topology was justified by two reasons [34]. • In most cases, it presents higher training speed when compared with ANN based on back-propagation training methods. • It allows an easier optimization of performance, since the only parameter that can be used to modify its structure is the number of neurons in the hidden layer. Results using RBF networks were presented to illustrate that it is possible to successfully control a generator system [43]. Power electronic drive controllers have also been implemented with these networks, in digital signal processors (DSPs), to attain robust properties [38]. G. Probabilistic NNs (PNNs) PNNs are somewhat similar in structure to MLPs. Some basic differences among them are the use of activation by exponential functions and the connection patterns between neurons. As a matter of fact, the neurons at the internal layers are not fully connected, depending on the application in turn. Fig. 9 depicts this structure, showing its basic differences from ordinary MLP structure. PNN training is normally easy and instantaneous, because of the smaller number of connections.

Fig. 10.

Polynomial ANN structure.

A practical advantage is that, unlike other networks, it operates completely in parallel and the signal flows in a unique direction, without a need for feedback from individual neurons to the inputs. It can be used for mapping, classification and associative memory, or to directly estimate a posteriori probabilities [103], [104]. Probabilistic NNs were used to assist operators while identifying transients in nuclear power plants, such as plant accident scenario, equipment failure or an external disturbance to the system, at the earliest stages of their developments [6]. H. Polynomial Networks Fig. 10 depicts a polynomial network. It has its topology formed during the training process. Due to this feature, it is defined as a plastic network. The neuron activation function is based on elementary polynomials of arbitrary order. In this example, the network has seven inputs, although the network uses only five of them. This is due to the automatic input selection capability of the training algorithm. Automatic feature selection is very useful in control applications when the plant model order is unknown. Each neuron output can be expressed by a secondorder polynomial function , where and are inputs and and are polynomial coefficients which are equivalent to the network weights and is the neuron output. The Group Method of Data Handling (GMDH) is a statistics-based training method largely used in modeling economic, ecological, environmental and medical problems. The GMDH training algorithm can be

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

593

Fig. 12.

Fig. 11.

Functional link ANN structure.

used to adjust the polynomial coefficients and to find the network structure. This algorithm employs two sets of data: One for estimating the network weights and the other for testing with neurons should survive during the training process. A new form of implementation of a filter was proposed using a combination of recurrent NN and polynomial NN [101]. I. FLNs Since NNs are used for adaptive identification and control, the learning capabilities of the networks can have significant effects on the performance of closed-loop systems. If the information content of data input to a network can be modified online, then it will more easily extract salient features of the data. The functional link acts on an element of an input vector, or on all the input vectors, by generating a set of linearly independent functions, then evaluating these functions with the pattern as the argument. Thus, both the training time and training error of the network can be improved [113]. Fig. 11 shows a functional link NN, which can be considered a one-layer feedforward network with an input preprocessing element. Only the weights in the output layer are adjusted [66]. The application of an FLN was presented for heating, ventilating, and air conditioning (HVAC) thermal dynamic system identification and control. The use of an NN provided a means of adapting a controller online, in an effort to minimize a given cost index. The identification networks demonstrated the capacity to learn changes in the plant dynamics and to accurately predict future plan behavior [113]. A robust ANN controller, to the motion control of rigid-link electrically driven robot using an FLN, had been presented. The method did not require the robot dynamics to be exactly known [69]. Multilayer feedforward and FLN forecasters were used to model the complex relationship between weather parameters and previous gas intake with future consumption [65]. The FLN was used to improve performance in the face of unknown nonlinear characteristics by adding nonlinear effects to the linear optimal controller of robotic systems [66]. J. Functional Polynomial Networks (FPNs) This network structure merges both models of functional link and polynomial network resulting in a very powerful ANN

Common grids of CNNs.

model due to the automatic input selection capability of the polynomial networks. The FPN presents advantages such as fast convergence, no local minima problem, structure automatically defined by the training process, and no adjustment of learning parameters. It has been tested for speed control with a dc motor and the results have been compared with the ones provided by an indirect adaptive control scheme based on MLPs trained by backpropagation [101]. K. Cellular NNs (CNNs) The most general definition for such networks is that they are arrays of identical dynamical systems, called cells, which are only locally connected. Only adjacent cells interact directly with each other [78]. In the simplest case, a CNN is an array of simple, identical, nonlinear, dynamical circuits placed on a two–dimensional (2-D) geometric grid, as shown in Fig. 12. If these grids are duplicated in a three–dimensional (3-D) form, a multilayer CNN can be constructed [30]. It is an efficient architecture for performing image processing and pattern recognition [51]. This kind of network has been applied to problems of image classification for quality control. Gulglielmi et al. [47] described a fluorescent magnetic particle inspection, which is a nondestructive method for quality control of ferromagnetic materials. V. TRAINING METHODS There are basically two main groups of training (or learning) algorithms: supervised learning (which includes reinforcement learning) and unsupervised learning. Once the structure of an NN has been selected, a training algorithm must be attached, to minimize the prediction error made by the network (for supervised learning) or to compress the information from the inputs (for unsupervised learning). In supervised learning, the correct results (target values, desired outputs) are known and are given to the ANN during training so that the ANN can adjust its weights to try match its outputs to the target values. After training, an ANN is tested as follows. One gives it only input values, not target values and sees how close the network comes to outputting the correct target values. Unsupervised learning involves no target values; it tries to auto-associate information from the inputs with an intrinsic reduction of data dimension, similar to extracting principal components in linear systems. This is the role of the training algorithms (i.e., fitting the model represented by the network to the training data available). The error, of a particular configuration of the network, can be

594

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

LMS or Widrow–Hoff Learning: The LMS algorithm is quite similar to perceptron learning algorithm. The differences are as follows. 1) The error is based on the sum of inputs to the unit rather than the binary-valued output to the unit . Therefore, (2)

Fig. 13.

Supervised learing scheme.

determined by running all the training cases through the network and comparing the actual output generated with the desired or target outputs or clusters. In learning algorithms considerations, one is interested in whether a particular algorithm converges, its speed of convergence and the computational complexity of the algorithm. In supervised learning, a set of inputs and correct outputs is used to train the network. Before the learning algorithms are applied to update the weights, all the weights are initialized randomly [84]. The network, using this set of inputs, produces its own outputs. These outputs are compared with the correct outputs and the differences are used to modify the weights, as shown in Fig. 13. A special case of supervised learning is reinforcement learning, shown in Fig. 14, where there is no set of inputs and correct outputs. Training is commanded only by signals indicating if the produced output is bad or good, according to a defined criterion. Fig. 15 shows the principles of unsupervised learning, also known as self-organized learning, where a network develops its classification rules by extracting information from the inputs presented to the network. In other words, by using the correlation of the input vectors, the learning rule changes the network weights to group the input vectors into clusters. By doing so, similar input vectors will produce similar network outputs since they belong to a same cluster.

2) The linear sum of the inputs is passed through bipolar sigmoid functions, which produces the output 1 or 1, depending on the polarity of the sum. This algorithm can be used in structures as RBF networks and was successfully applied [43], [70]. Some CMAC approaches can also use this algorithm to adapt a complex robotic system involving multiple feedback sensors and multiple command variables [79]. Grossberg Learning: Sometimes known as in-star and out-star training, this algorithm is updated as follows: (3) where could be the desired input values (in-star training) or the desired output values (out-star training) depending on the network structure. B. First-Order Gradient Methods Backpropagation: Backpropagation is a generalization of the LMS algorithm. In this algorithm, an error function is defined as the mean-square difference between the desired output and the actual output of the feedforward network [45]. It is based on steepest descent techniques extended to each of the layers in the network by the chain rule. Hence, the algorithm , of the computes the partial derivative error function with respect to the weights. The error function , where is the is defined as is the network output. The objective is to desired output, by taking the error gradient minimize the error function with respect to the parameters or weight vector, for example, , that is to be adapted. The weights are then updated by using

A. Early Supervised Learning Algorithms Early learning algorithms were designed for single layer NNs. They are generally more limited in their applicability, but their importance in history is remarkable. Perceptron Learning: A single-layer perceptron is trained as follows. 1) Randomly initialize all the networks weights. 2) Apply the inputs and calculate the sum of each unit . 3) The outputs from each unit are

(4) where

is the learning rate and (5)

(1)

This algorithm is simple to implement and computationally less complex than other modified forms. Despite some disadvantages, it is popularly used and there are numerous extensions to improve it. Some of these techniques will be presented. Backpropagation With Momentum (BPM): The basic improvement to the backpropagation algorithm is to introduce a momentum term in the weights updating equation

where 4) Compute the error is the known desired output value. 5) Update each weight as . 6) Repeat steps 2)–4) until the errors reach the satisfactory level.

(6) is commonly selected inside where the momentum factor [0,1]. Adding the momentum term improves the convergence speed and helps the network from being trapped in a local minimum.

threshold otherwise

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

Fig. 14.

595

Reinforcement learning scheme.

where (10) (11)

Fig. 15.

Unsupervised learning scheme.

A modification to (6) was proposed in 1990, inserting the constant , defined by the user [84]

(7) The idea was to reduce the possibility of the network being trapped in the local minimum. Delta–Bar–Delta (DBD): The DBD learning rules use adaptive learning rates, to speed up the convergence. The adaptive learning rate adopted is based on a local optimization method. This technique uses gradient descent for the search direction and then applies individual step sizes for each weight. This means the actual direction taken in weight space is not necessarily along the line of the steepest gradient. If the weight updates between consecutive iterations are in opposite directions, the step size is decreased; otherwise, it is increased. This is prompted by the idea that if the weight changes are oscillating, the minimum is between the oscillations and a smaller step size might find that minimum. The step size may be increased again once the error has stopped oscillating. denotes the learning rate for the weight , then If (8) and

is as follows:

(9) otherwise

) are specified by The positive constant and parameters ( is basically an exponentially dethe user. The quantity caying trace of gradient values. When the and are set to zero, the learning rates assume a constant value as in the standard backpropagation algorithm. Using momentum along with the DBD algorithm can enhance performance considerably. However, it can make the search diverge wildly, especially if is moderately large. The reason is that momentum magnifies learning rate increments and quickly leads to inordinately large learning steps. One possible solution is to keep factor very small, but this can easily lead to slow increase in and little speedup [84]. C. Second-Order Gradient Methods These methods use the Hessian matrix , which is the matrix of second derivatives of with respect to the weights . This matrix contains information about how the gradient changes in different directions in weight space (12) Newton Method: The Newton method weights update is processed as follows: (13) However, the Newton method is not commonly used because computing the Hessian matrix is computationally expensive. Furthermore, the Hessian matrix may not be positive definite at every point in the error surface. To overcome the problem, several methods have being proposed to approximate the Hessian matrix [84]. Gauss–Newton Method: The Gauss–Newton method promatrix that is an approximation to the Hessian duces an matrix, having elements represented by (14)

596

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

where . However, the Gauss–Newton method may still have ill conditioning if close to or is singular [75]. Levenberg–Marquardt (LM) Method: The LM method overcomes this difficulty by including an additional term, which is added to the Gauss–Newton approximation of the Hessian giving (15) where is a small positive value and is the identity matrix. could also be made adaptive by having if if

(16)

where is a value defined by the user. It is important to notice that when is large, the algorithm becomes backpropagation and, when is small, the algorithm bewith learning rate comes Gauss–Newton. An NN, trained by using this algorithm, can be found in the diagnosis of various motor bearing faults through appropriate measurement and interpretation of motor bearing vibration signals [72].

eligibility traces can be generated using the following linear difference equation: (22) determines the trace decay rate, is the where input, and is the output. Adaptive Critic Learning: The weights update in a critic network is as follows [9]: (23) , is a constant discount factor, is the where , is the reinforcement signal from the environment, and is the prediction at time of trace of the input variable . eventual reinforcement, which can be described as a linear func. The adaptive critic network output tion , the improved or internal reinforcement signal, is computed from the predictions as follows: (24) E. Unsupervised Learning

D. Reinforcement Learning Algorithms Reinforcement learning has one of its roots in psychology, from the idea of rewards and penalties used to teach animals to do simple tasks with small amounts of feedback information. Barto and others proposed the Adaptive Critic Learning algorithm to solve discrete domain decision-making problems in the 1980s. Their approach was generalized to the NNs field later used by Sutton, who used CMAC and other ANN structures to learn paths for mobile robots in maze-like environments [111]. Linear Reward–Penalty Learning: When the reinforcement signal is positive ( 1), the learning rule is

for

(17) (18)

If the reinforcement signal is negative ( 1), the learning rule is (19) for

(20)

denotes the probability where and are learning rates, at iteration , and is the number of actions taken. For positive reinforcement, the probability of the current action is increased with relative decrease in the probabilities of the other actions. The adjustment is reversed in the case of negative reinforcement. Associative Search Learning: In this algorithm, the weights are updated as follows [9]: (21) is the reinforcement signal and is eligibility. where Positive indicates the occurrence of a rewarding event and negative indicates the occurrence of a punishing event. It can be regarded as a measure of the change in the value of a performance criterion. Eligibility reflects the extent of activity in the pathway or connection link. Exponentially decaying

Hebbian Learning: Weights are updated as follows: (25) (26) is the weight from th unit to th unit at time where is the excitation level of the source unit or th step , is the excitation level of the destination unit unit, and or the th output unit. In this system, learning is a purely local phenomenon, involving only two units and a synapse. No global feedback system is required for the neural pattern to develop. A special case of Hebbian learning is correlation and learning, which uses binary activation for function is defined as the desired excitation level for the destination unit. While Hebbian learning is performed in unsupervised environments, correlation learning is supervised [128]. Boltzmann Machine Learning: The Boltzmann Machine training algorithm uses a kind of stochastic technique known as simulated annealing, to avoid being trapped in local minima of the network energy function. The algorithm is as follows. 1) Initialize weights. 2) Calculate activation as follows. a) Select an initial temperature. b) Until thermal equilibrium, repeatedly calculate the probability that is active by (23). c) Exit when the lowest temperature is reached. Otherwise, reduce temperature by a certain annealing schedule and repeat step 2) (27) is the total input Above, is the temperature, received by the th unit, and the activation level of unit is set according to this probability.

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

597

Kohonen Self-Organizing Learning: The network is trained according to the following algorithm, frequently called the “winner-takes-all” rule. 1) Apply an input vector . (in -dimensional space) be2) Calculate the distance of each unit. In Eutween and the weight vectors clidean space, this is calculated as follows: (28) 3) The unit that has the weight vector closest to is declared , becomes the winner unit. This weight vector, called the center of a group of weight vectors that lie within a . distance from , train 4) For all weight vectors within a distance of this group of nearby weight vectors according to the formula that follows: (29) 5) Perform steps 1)–4), cycling through each input vector until convergence. F. Practical Considerations NNs are unsurpassed at identifying patterns or trends in data and well suited for prediction or forecasting needs including sales and customer research, data validation, risk management, and industrial process control. One of the fascinating aspects, of the practical implementation of NNs to industrial applications, is the ability to manage data interaction between electrical and mechanical behavior and often other disciplines, as well. The majority of the reported applications involve fault diagnosis and detection, quality control, pattern recognition, and adaptive control [14], [44], [74], [115]. Supervised NNs can mimic the behavior of human control systems, as long as data corresponding to the human operator and the control input are supplied [7], [126]. Most of the existing, successful applications in control use supervised learning, or any form of a reinforcement learning approach that is also supervised. Unsupervised learning is not suitable, particularly for online control, due the slow adaptation and required time for the network to settle into stable conditions. Unsupervised learning schemes are used mostly for pattern recognition, by defining group of patterns into a number of clusters or classes. There are some advantages to NNs over multiple data regression. There is no need to select the most important independent variables in the data set. The synapses associated with irrelevant variables readily show negligible weight values; in their turn, relevant variables present significant synapse weight values. There is also no need to propose a model function as required in multiple regressions. The learning capability of NNs allows them to discover more complex and subtle interactions between the independent variables, contributing to the development of a model with maximum precision. NNs are intrinsically robust showing more immunity to noise, an important factor in modeling industrial processes. NNs have been applied within industrial domains, to address the inherent complexity of interacting processes under the lack of robust analytical models of real industrial processes. In many cases, network topologies and training parameters are systematically

varied until satisfactory convergence is achieved. Currently, the most widely used algorithm for training MLPs is the backpropagation algorithm. It minimizes the mean square error between the desired and the actual output of the network. The optimization is carried out with a gradient-descent technique. There are two critical issues in network learning: estimation error and training time. These issues may be affected by the network architecture and the training set. The network architecture includes the number of hidden nodes, number of hidden layers and values of learning parameters. The training set is related to the number of training patterns, inaccuracies of input data and preprocessing of data. The backpropagation algorithm does not always find the global minimum, but may stop at a local minimum. In practice, the type of minimum has little importance, as long as the desired mapping or classification is reached with a desired accuracy. The optimization criterion of the backpropagation algorithm is not very good, from the pattern recognition point of view. The algorithm minimizes the square error between the actual and the desired output, not the number of faulty classifications, which is the main goal in pattern recognition. The algorithm is too slow for practical applications, especially if many hidden layers are used. In addition, a backpropagation net has poor memory. When the net learns something new it forgets the old. Despite its shortcomings, bac-propagation is broadly used. Although the back-propagation algorithm has been a significant milestone, many attempts have been made to speed up the convergence and significant improvement are observed by using various second order approaches, namely, Newton’s method, conjugate gradient’s, or the LM optimization technique [5], [10], [21], [48]. The issues to be dealt with are [84], [102] as follows: 1) 2) 3) 4)

slow convergence speed; sensitivity to initial conditions; trapping in local minima; instability if learning rate is too large.

One of the alternatives for the problem of being trapped in a local minimum is adding the momentum term using the BPM, which also improves the convergence speed. Another alternative used when a backpropagation algorithm is difficult to implement, as in analog hardware, is the random weight change (RWC). This algorithm has shown to be immune to offset sensitivity and nonlinearity errors. It is a stochastic learning that makes sure that the error function decreases on average, since it is going up or down at any one time. It is often called simulated annealing because of its operational similarity to annealing processes [73]. Second order gradient methods use the matrix with respect to the weights . of second derivatives of However, computing this matrix is computationally expensive and the methods presented tried to approximate this matrix to make algorithms more accessible. Linear reward–penalty, associative search, and adaptive critic algorithms are characterized as a special case of supervised learning called reinforcement learning. They do not need to explicitly compute derivatives. Computation of derivatives usually introduces a lot of high frequency noise in the control loop. Therefore, they are very suitable for some complex systems, where basic training algorithms may fail or produce suboptimal results. On the other hand, those methods present

598

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

slower learning processes; and, because of this, they are adopted especially in the cases where only a single bit of information (for example, whether the output is right or wrong) is available.

themselves, have to be found from data in the training phase, without supervision. C. Process Control

VI. TAXONOMY OF NN APPLICATIONS From the viewpoint of industrial applications, ANN applications can be divided into four main categories. A. Modeling and Identification Modeling and identification are techniques to describe the physical principles existing between the input and the output of a system. The ANN can approximate these relationships independent of the size and complexity of the problem. It has been found to be an effective system for learning discriminants, for patterns from a body of examples. MLP is used as the basic structure for a bunch of applications [4], [12], [17], [18], [32], [46], [56], [85], [94], [119], [127]. The Hopfield network can be used to identify problems of linear time-varying or time-invariant systems [28]. Recurrent network topology [36], [83] has received considerable attention for the identification of nonlinear dynamical systems. A functional-link NN approach (FLN) was used to perform thermal dynamical system identification [113]. B. Optimization and Classification Optimization is often required for design, planning of actions, motions, and tasks. However, as known in the Traveling Salesman Problem, many parameters cause the amount of calculation to be tremendous and the ordinary method cannot be applied. An affective approach is to find the optimal solution by defining an energy function and using the NN with parallel processing, learning and self-organizing capabilities to operate in such a way that the energy is reduced. It is shown that application of the optimal approach makes effective use of ANN sensing, recognizing, and forecasting capabilities, in the control of robotic manipulators with impact taken into account [45]. Classification using an ANN can also be viewed as an optimization problem, provided that the existing rules to distinguish the various classes of events/materials/objects can be described in functional form. In such cases, the networks will decide if a particular input belongs to one of the defined classes by optimizing the functional rules and a posteriori evaluating the achieved results. Different authors [34], [70] have proposed RBF approaches. For applications such as fault diagnosis, RBF networks offer clear advantages over MLPs. They are faster to train, because layer training is decoupled [70]. Cellular networks [47], ART networks [122], and Hopfield networks [105], [112] can be used as methods to detect, to isolate faults, and to promote industrial quality control. MLPs are also widely used for these purposes [23], [25], [35], [102], [114], [121]. This structure can be found in induction motor [41], [42] and bearing [72] fault diagnosis, for nondestructive evaluation of check valve performance and degradation [2], [57], in defect detection on woven fabrics [99], and in robotic systems [116]. Finally, it is important to mention clustering applications, which are special cases of classification where there is no supervision during the training phase. The relationships between elements of the existing classes and even the classes

The NN makes use of nonlinearity, learning, parallel processing, and generalization capabilities for application to advanced intelligent control. They can be classified into some major methods, such as supervised control, inverse control, neural adaptive control, back-propagation of utility (which is an extended method of a back-propagation through time) and adaptive critics (which is an extended method of reinforcement learning algorithm) [2]. MLP structures were used for digital current regulation of inverter drives [16], to predict trajectories in robotic environments [19], [40], [52], [73], [79], [87], [89], [110], to control turbo generators [117], to monitor feed water flow rate and component thermal performance of pressurized water reactors [61], to regulate temperature [64], and to predict natural gas consumption [65]. Dynamical versions of MLP networks were used to control a nonlinear dynamic model of a robot [60], [97], to control manufacturing cells [92], and to implement a programmable cascaded low-pass filter [101]. A dynamic MLP is a classical MLP structure where the outputs are fed back to the inputs by means of time delay elements. Other structures can be found as functional link networks to control robots [69] and RBF networks to predict, from operating conditions and from features of a steel sheet, the thermal energy required to correct alloying [13]. RBF networks can be observed, as well, in predictive controllers for drive systems [38]. Hopfield structures were used for torque minimization control of redundant manipulators [33]. FPNs can be used for function approximation inside specific control schemes [100]. CMAC networks were implemented in research automobiles [76] and to control robots [79]. D. Pattern Recognition Some specific ANN structures, such as Kohonen and probabilistic networks, are studied and applied mainly for image and voice recognition. Research in image recognition includes initial vision (stereo vision of both eyes, outline extraction, etc.) close to the biological (particularly brain) function, manually written character recognition by cognition at the practical level and cell recognition for mammalian cell cultivation by using NNs [45]. Kohonen networks were used for image inspection and for disease identification from mammographic images [98]. Probabilistic networks were used for transient detection to enhance nuclear reactors’ operational safety [6]. As in the others categories, the MLP is widely used as well. The papermaking industry [39] is one such example. VII. CONCLUSION This paper has described theoretical aspects of NNs related to their relevance for industrial applications. Common questions that an engineer would ask when choosing an NN for a particular application were answered. Characteristics of industrial processes, which would justify the ANN utilization, were discussed and some areas of importance were proposed. Important structures and training methods, with relevant references that illustrated the utilization of those concepts, were presented.

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

This survey observed that, although ANNs have a history of more than 50 years, most of industrial applications were launched in the last ten years, where it was justified that the investigators provided either an alternative or a complement to other classical techniques. Those ANN applications demonstrated adaptability features integrated with the industrial problem, thus becoming part of the industrial processes. The authors firmly believe that such an intricate field of NNs is just starting to permeate a broad range of interdisciplinary problem solving streams. The potential of NNs will be integrated into a still larger and all-encompassing field of intelligence systems and will soon be taught for students and engineers as an ordinary mathematical tool. REFERENCES [1] J. S. Albus, “A new approach to manipulator control: The cerebellar model articulation controller,” Trans. ASME, J. Dyn. Syst., Meas. Control, vol. 97, pp. 220–227, Sept. 1975. [2] I. E. Alguíndigue and R. E. Uhrig, “Automatic fault recognition in mechanical components using coupled artificial neural networks,” in Proc. IEEE World Congr. Computational Intelligence, June–July 1994, pp. 3312–3317. [3] P. E. M. Almeida and M. G. Simões, “Fundamentals of a fast convergence parametric CMAC network,” in Proc. IJCNN’01, vol. 3, 2001, pp. 3015–3020. [4] K. Andersen, G. E. Cook, G. Karsai, and K. Ramaswamy, “Artificial neural networks applied to arc welding process modeling and control,” IEEE Trans. Ind. Applicat., vol. 26, pp. 824–830, Sept./Oct. 1990. [5] T. J. Andersen and B. M. Wilamowski, “A modified regression algorithm for fast one layer neural network training,” in Proc. World Congr. Neural Networks, vol. 1, Washington DC, July 17–21, 1995, pp. 687–690. [6] I. K. Attieh, A. V. Gribok, J. W. Hines, and R. E. Uhrig, “Pattern recognition techniques for transient detection to enhance nuclear reactors’ operational safety,” in Proc. 25th CNS/CNA Annu. Student Conf., Knoxville, TN, Mar. 2000. [7] S. M. Ayala, G. Botura Jr., and O. A. Maldonado, “AI automates substation control,” IEEE Comput. Applicat. Power, vol. 15, pp. 41–46, Jan. 2002. [8] D. L. Bailey and D. M. Thompson, “Developing neural-network applications,” AI Expert, vol. 5, no. 9, pp. 34–41, 1990. [9] A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike elements that can solve difficult control problems,” IEEE Trans. Syst., Man, Cybern., vol. SMC-13, pp. 834–846, Sept./Oct. 1983. [10] R. Battiti, “First- and second-order methods for learning: Between steepest descent and Newton’s method,” Neural Computation, vol. 4, no. 2, pp. 141–166, 1992. [11] B. Bavarian, “Introduction to neural networks for intelligent control,” IEEE Contr. Syst. Mag., vol. 8, pp. 3–7, Apr. 1988. [12] N. V. Bhat, P. A. Minderman, T. McAvoy, and N. S. Wang, “Modeling chemical process systems via neural computation,” IEEE Contr. Syst. Mag., vol. 10, pp. 24–30, Apr. 1990. [13] G. Bloch, F. Sirou, V. Eustache, and P. Fatrez, “Neural intelligent control for a steel plant,” IEEE Trans. Neural Networks, vol. 8, pp. 910–918, July 1997. [14] Z. Boger, “Experience in developing models of industrial plants by large scale artificial neural networks,” in Proc. Second New Zealand International Two-Stream Conf. Artificial Neural Networks and Expert Systems, 1995, pp. 326–329. [15] D. S. Broomhead and D. Lowe, “Multivariable functional interpolation and adaptive network,” Complex Syst., vol. 2, pp. 321–355, 1988. [16] M. Buhl and R. D. Lorenz, “Design and implementation of neural networks for digital current regulation of inverter drives,” in Conf. Rec. IEEE-IAS Annu. Meeting, 1991, pp. 415–423. [17] B. Burton and R. G. Harley, “Reducing the computational demands of continually online-trained artificial neural networks for system identification and control of fast processes,” IEEE Trans. Ind. Applicat., vol. 34, pp. 589–596, May/June 1998. [18] B. Burton, F. Kamran, R. G. Harley, T. G. Habetler, M. Brooke, and R. Poddar, “Identification and control of induction motor stator currents using fast on-line random training of a neural network,” in Conf. Rec. IEEE-IAS Annu. Meeting, 1995, pp. 1781–1787.

599

[19] R. Carelli, E. F. Camacho, and D. Patiño, “A neural network based feed forward adaptive controller for robots,” IEEE Trans. Syst., Man, Cybern., vol. 25, pp. 1281–1288, Sept. 1995. [20] G. A. Carpenter and S. Grossberg, “Associative learning, adaptive pattern recognition and cooperative- competitive decision making,” in Optical and Hybrid Computing, H. Szu, Ed. Bellingham, WA: SPIE, 1987, vol. 634, pp. 218–247. [21] C. Charalambous, “Conjugate gradient algorithm for efficient training of artificial neural networks,” Proc. Inst. Elect. Eng., vol. 139, no. 3, pp. 301–310, 1992. [22] S. Chen and S. A. Billings, “Neural networks for nonlinear dynamic system modeling and identification,” Int. J. Control, vol. 56, no. 2, pp. 319–346, 1992. [23] R. P. Cherian, L. N. Smith, and P. S. Midha, “A neural network approach for selection of powder metallurgy materials and process parameters,” Artif. Intell. Eng., vol. 14, pp. 39–44, 2000. [24] M. Y. Chow, P. M. Mangum, and S. O. Yee, “A neural network approach to real-time condition monitoring of induction motors,” IEEE Trans. Ind. Electron., vol. 38, pp. 448–453, Dec. 1991. [25] M. Y. Chow, R. N. Sharpe, and J. C. Hung, “On the application and design of artificial neural networks for motor fault detection—Part II,” IEEE Trans. Ind. Electron., vol. 40, pp. 189–196, Apr. 1993. , “On the application and design of artificial neural networks for [26] motor fault detection—Part I,” IEEE Trans. Ind. Electron., vol. 40, pp. 181–188, Apr. 1993. [27] T. W. S. Chow and Y. Fang, “A recurrent neural-network based real-time learning control strategy applying to nonlinear systems with unknown dynamics,” IEEE Trans. Ind. Electron., vol. 45, pp. 151–161, Feb. 1998. [28] S. R. Chu and R. Shoureshi, “Applications of neural networks in learning of dynamical systems,” IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 160–164, Jan./Feb. 1992. [29] S. R. Chu, R. Shoureshi, and M. Tenorio, “Neural networks for system identification,” IEEE Contr. Syst. Mag., vol. 10, pp. 31–35, Apr. 1990. [30] L. O. Chua, T. Roska, T. Kozek, and Á. Zarándy, “The CNN Paradigm—A short tutorial,” in Cellular Neural Networks, T. Roska and J. Vandewalle, Eds. New York: Wiley, 1993, pp. 1–14. [31] M. Cichowlas, D. Sobczuk, M. P. Kazmierkowski, and M. Malinowski, “Novel artificial neural network based current controller for PWM rectifiers,” in Proc. 9th Int. Conf. Power Electronics and Motion Control, 2000, pp. 41–46. [32] G. E. Cook, R. J. Barnett, K. Andersen, and A. M. Strauss, “Weld modeling and control using artificial neural network,” IEEE Trans. Ind. Applicat., vol. 31, pp. 1484–1491, Nov./Dec. 1995. [33] H. Ding and S. K. Tso, “A fully neural-network-based planning scheme for torque minimization of redundant manipulators,” IEEE Trans. Ind. Electron., vol. 46, pp. 199–206, Feb. 1999. [34] G. Dini and F. Failli, “Planning grasps for industrial robotized applications using neural networks,” Robot. Comput. Integr. Manuf., vol. 16, pp. 451–463, Dec. 2000. [35] M. Dolen and R. D. Lorenz, “General methodologies for neural network programming,” in Proc. IEEE Applied Neural Networks Conf., Nov. 1999, pp. 337–342. , “Recurrent neural network topologies for spectral state estimation [36] and differentiation,” in Proc. ANNIE Conf., St. Louis, MO, Nov. 2000. [37] M. Dolen, P. Y. Chung, E. Kayikci, and R. D. Lorenz, “Disturbance force estimation for CNC machine tool feed drives by structured neural network topologies,” in Proc. ANNIE Conference, St. Louis, MO, Nov. 2000. [38] Y. Dote, M. Strefezza, and A. Suyitno, “Neuro fuzzy robust controllers for drive systems,” in Proc. IEEE Int. Symp. Industrial Electronics, 1993, pp. 229–242. [39] P. J. Edwards, A. F. Murray, G. Papadopoulos, A. R. Wallace, J. Barnard, and G. Smith, “The application of neural networks to the papermaking industry,” IEEE Trans. Neural Networks, vol. 10, pp. 1456–1464, Nov. 1999. [40] M. J. Er and K. C. Liew, “Control of adept one SCARA robot using neural networks,” IEEE Trans. Ind. Electron., vol. 44, pp. 762–768, Dec. 1997. [41] F. Filippetti, G. Franceschini, and C. Tassoni, “Neural networks aided on-line diagnostics of induction motor rotor faults,” IEEE Trans. Ind. Applicat., vol. 31, pp. 892–899, July/Aug. 1995. [42] F. Filippetti, G. Franceschini, C. Tassoni, and P. Vas, “Recent developments of induction motor drives fault diagnosis using AI techniques,” IEEE Trans. Ind. Electron., vol. 47, pp. 994–1004, Oct. 2000. [43] D. Flynn, S. McLoone, G. W. Irwin, M. D. Brown, E. Swidenbank, and B. W. Hogg, “Neural control of turbogenerator systems,” Automatica, vol. 33, no. 11, pp. 1961–1973, 1997.

600

[44] D. B. Fogel, “Selecting an optimal neural network industrial electronics society,” in Proc. IEEE IECON’90, vol. 2, 1990, pp. 1211–1214. [45] T. Fukuda and T. Shibata, “Theory and applications of neural networks for industrial control systems,” IEEE Trans. Ind. Applicat., vol. 39, pp. 472–489, Nov./Dec. 1992. [46] A. A. Gorni, “The application of neural networks in the modeling of plate rolling processes,” JOM-e, vol. 49, no. 4, electronic document, Apr. 1997. [47] N. Guglielmi, R. Guerrieri, and G. Baccarani, “Highly constrained neural networks for industrial quality control,” IEEE Trans. Neural Networks, vol. 7, pp. 206–213, Jan. 1996. [48] M. T. Hagan and M. Menhaj, “Training feedforward networks with the Marquardt algorithm,” IEEE Trans. Neural Networks, vol. 5, pp. 989–993, Nov. 1994. [49] S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. New York: Prentice-Hall, 1995. [50] S. M. Halpin and R. F. Burch, “Applicability of neural networks to industrial and commercial power systems: A tutorial overview,” IEEE Trans. Ind. Applicat., vol. 33, pp. 1355–1361, Sept./Oct. 1997. [51] H. Harrer and J. Nossek, “Discrete-time cellular neural networks,” in Cellular Neural Networks, T. Roska and J. Vandewalle, Eds. New York: Wiley, 1993, pp. 15–29. [52] H. Hashimoto, T. Kubota, M. Sato, and F. Harashima, “Visual control of robotic manipulator based on neural networks,” IEEE Trans. Ind. Electron., vol. 39, pp. 490–496, Dec. 1992. [53] D. O. Hebb, The Organization of Behavior. New York: Wiley, 1949. [54] G. E. Hinton and T. J. Sejnowski, “Learning and relearning in Boltzmann machines,” in The PDP Research Group, D. Rumelhart and J. McClelland, Eds. Cambridge, MA: MIT Press, 1986. [55] J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” in Proc. Nat. Acad. Sci., vol. 79, Apr. 1982, pp. 2445–2558. [56] C. Y. Huang, T. C. Chen, and C. L. Huang, “Robust control of induction motor with a neural-network load torque estimator and a neural-network identification,” IEEE Trans. Ind. Electron., vol. 46, pp. 990–998, Oct. 1999. [57] A. Ikonomopoulos, R. E. Uhrig, and L. H. Tsoukalas, “Use of neural networks to monitor power plant components,” in Proc. American Power Conf., vol. 54-II, Apr. 1992, pp. 1132–1137. [58] M. Jelínek. (1999) Everything you wanted to know about ART neural networks, but were afraid to ask. [Online]. Available: http://cs.felk.cvut.cz/~xjeline1/semestralky/nan [59] S. Jung and T. C. Hsia, “Neural network impedance force control of robot manipulator,” IEEE Trans. Ind. Electron., vol. 45, pp. 451–461, June 1998. [60] A. Karakasoglu and M. K. Sundareshan, “A recurrent neural networkbased adaptive variable structure model-following control of robotic manipulators,” Automatica, vol. 31, no. 10, pp. 1495–1507, 1995. [61] K. Kavaklioglu and B. R. Upadhyaya, “Monitoring feedwater flow rate and component thermal performance of pressurized water reactors by means of artificial neural networks,” Nucl. Technol., vol. 107, pp. 112–123, July 1994. [62] M. Kawato, Y. Uno, M. Isobe, and R. Suzuki, “Hierarchical neural network model for voluntary movement with application to robotics,” IEEE Contr. Syst. Mag., vol. 8, pp. 8–15, Apr. 1988. [63] M. Khalid and S. Omatu, “A neural network controller for a temperature control system,” IEEE Contr. Syst. Mag., vol. 12, pp. 58–64, June 1992. [64] M. Khalid, S. Omatu, and R. Yusof, “Temperature regulation with neural networks and alternative control schemes,” IEEE Trans. Neural Networks, vol. 6, pp. 572–582, May 1995. [65] A. Khotanzad, H. Elragal, and T. L. Lu, “Combination of artificial neural-network forecasters for prediction of natural gas consumption,” IEEE Trans. Neural Networks, vol. 11, pp. 464–473, Mar. 2000. [66] Y. H. Kim, F. L. Lewis, and D. M. Dawson, “Intelligent optimal control of robotic manipulators using neural network,” Automatica, vol. 36, no. 9, pp. 1355–1364, 2000. [67] B. Kosko, “Adaptive bi-directional associative memories,” Appl. Opt., vol. 26, pp. 4947–4960, 1987. [68] S. Y. Kung and J. N. Hwang, “Neural network architectures for robotic applications,” IEEE Trans. Robot. Automat., vol. 5, pp. 641–657, Oct. 1989. [69] C. Kwan, F. L. Lewis, and D. M. Dawson, “Robust neural-network control of rigid-link electrically driven robots,” IEEE Trans. Neural Networks, vol. 9, pp. 581–588, July 1998. [70] J. A. Leonard and M. A. Kramer, “Radial basis function networks for classifying process faults,” IEEE Contr. Syst. Mag., vol. 11, pp. 31–38, Apr. 1991.

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 50, NO. 3, JUNE 2003

[71] F. L. Lewis, A. Yesildirek, and K. Liu, “Multilayer neural-net robot controller with guaranteed tracking performance,” IEEE Trans. Neural Networks, vol. 7, pp. 388–399, Mar. 1996. [72] B. Li, M. Y. Chow, Y. Tipsuwan, and J. C. Hung, “Neural-network based motor rolling bearing fault diagnosis,” IEEE Trans. Ind. Electron., vol. 47, pp. 1060–1069, Oct. 2000. [73] J. Liu, B. Burton, F. Kamran, M. A. Brooke, R. G. Harley, and T. G. Habetler, “High speed on-line neural network of an induction motor immune to analog circuit nonidealities,” in Proc. IEEE Int. Symp. Circuits and Systems, June 1997, pp. 633–636. [74] Y. Liu, B. R. Upadhyaya, and M. Naghedolfeizi, “Chemometric data analysis using artificial neural networks,” Appl. Spectrosc., vol. 47, no. 1, pp. 12–23, 1993. [75] L. Ljung, System Identification: Theory for the User. New York: Prentice-Hall, 1987. [76] M. Majors, J. Stori, and D. Cho, “Neural network control of automotive fuel-injection systems,” IEEE Contr. Syst. Mag., vol. 14, pp. 31–36, June 1994. [77] W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” Bull. Math. Biophys., vol. 5, pp. 115–133, 1943. [78] M. Milanova, P. E. M. Almeida, J. Okamoto Jr., and M. G. Simões, “Applications of cellular neural networks for shape from shading problem,” in Proc. Int. Workshop Machine Learning and Data Mining in Pattern Recognition, Lecture Notes in Artificial Intelligence, P. Perner and M. Petrou, Eds., Leipzig, Germany, September 1999, pp. 52–63. [79] W. T. Miller III, “Real-time application of neural networks for sensorbased control of robots with vision,” IEEE Trans. Syst., Man, Cybern., vol. 19, pp. 825–831, July/Aug. 1989. [80] M. L. Minsky and S. Papert, Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press, 1969. [81] T. Munakata, Fundamentals of the New Artificial Intelligence—Beyond Traditional Paradigms. Berlin, Germany: Springer-Verlag, 1998. [82] S. R. Naidu, E. Zafiriou, and T. J. McAvoy, “Use of neural networks for sensor failure detection in a control system,” IEEE Contr. Syst. Mag., vol. 10, pp. 49–55, Apr. 1990. [83] K. S. Narendra and K. Parthasarathy, “Identification and control of dynamical systems using neural networks,” IEEE Trans. Neural Networks, vol. 1, pp. 4–27, Mar. 1990. [84] G. W. Ng, Application of Neural Networks to Adaptive Control of Nonlinear Systems. London, U.K.: Research Studies Press, 1997. [85] D. H. Nguyen and B. Widrow, “Neural networks for self-learning control systems,” IEEE Contr. Syst. Mag., vol. 10, pp. 18–23, Apr. 1990. [86] J. R. Noriega and H. Wang, “A direct adaptive neural-network control for unknown nonlinear systems and its application,” IEEE Trans. Neural Networks, vol. 9, pp. 27–34, Jan. 1998. [87] T. Ozaki, T. Suzuki, T. Furuhashi, S. Okuma, and Y. Uchikawa, “Trajectory control of robotic manipulators using neural networks,” IEEE Trans. Ind. Electron., vol. 38, June 1991. [88] D. B. Parker, “A comparison of algorithms for neuron-like cells,” in Neural Networks for Computing, J. S. Denker, Ed. New York: American Institute of Physics, 1986, pp. 327–332. [89] P. Payeur, H. Le-Huy, and C. M. Gosselin, “Trajectory prediction for moving objects using artificial neural networks,” IEEE Trans. Ind. Electron., vol. 42, pp. 147–158, Apr. 1995. [90] M. H. Rahman, R. Fazlur, R. Devanathan, and Z. Kuanyi, “Neural network approach for linearizing control of nonlinear process plants,” IEEE Trans. Ind. Electron., vol. 47, pp. 470–477, Apr. 2000. [91] F. Rosenblatt, “The perceptron: A probabilistic model for information storage and organization in the brain,” Psych. Rev., vol. 65, pp. 386–408, 1958. [92] G. A. Rovithakis, V. I. Gaganis, S. E. Perrakis, and M. A. Christodoulou, “Real-time control of manufacturing cells using dynamic neural networks,” Automatica, vol. 35, no. 1, pp. 139–149, 1999. [93] A. Rubaai and M. D. Kankam, “Adaptive real-time tracking controller for induction motor drives using neural designs,” in Conf. Rec. IEEE-IAS Annu. Meeting, vol. 3, Oct. 1996, pp. 1709–1717. [94] A. Rubaai and R. Kotaru, “Online identification and control of a DC motor using learning adaptation of neural networks,” IEEE Trans. Ind. Applicat., vol. 36, pp. 935–942, May/June 2000. [95] D. D. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, pp. 533–536, 1986. [96] D. E. Rumelhart, B. Widrow, and M. A. Lehr, “The basic ideas in neural networks,” Commun. ACM, vol. 37, no. 3, pp. 87–92, Mar. 1994. [97] M. Saad, P. Bigras, L. A. Dessaint, and K. A. Haddad, “Adaptive robot control using neural networks,” IEEE Trans. Ind. Electron., vol. 41, pp. 173–181, Apr. 1994.

MEIRELES et al.: INDUSTRIAL APPLICABILITY OF ANNs

[98] S. Sardy and L. Ibrahim, “Experimental medical and industrial applications of neural networks to image inspection using an inexpensive personal computer,” Opt. Eng., vol. 35, no. 8, pp. 2182–2187, Aug. 1996. [99] S. Sardy, L. Ibrahim, and Y. Yasuda, “An application of vision system for the identification and defect detection on woven fabrics by using artificial neural networks,” in Proc. Int. Joint Conf. Neural Networks, 1993, pp. 2141–2144. [100] A. P. A. Silva, P. C. Nascimento, G. L. Torres, and L. E. B. Silva, “An alternative approach for adaptive real-time control using a nonparametric neural network,” in Conf. Rec. IEEE-IAS Annu. Meeting, 1995, pp. 1788–1794. [101] L. E. B. Silva, B. K. Bose, and J. O. P. Pinto, “Recurrent-neural-network-based implementation of a programmable cascaded low-pass filter used in stator flux synthesis of vector-controlled induction motor drive,” IEEE Trans. Ind. Electron., vol. 46, pp. 662–665, June 1999. [102] T. Sorsa, H. N. Koivo, and H. Koivisto, “Neural networks in process fault diagnosis,” IEEE Trans. Syst., Man. Cybern., vol. 21, pp. 815–825, July/Aug. 1991. [103] D. F. Specht, “Probabilistic neural networks for classification, mapping, or associative memory,” in Proc. IEEE Int. Conf. Neural Networks, July 1988, pp. 525–532. , “Probabilistic neural networks,” Neural Networks, vol. 3, pp. [104] 109–118, 1990. [105] A. Srinivasan and C. Batur, “Hopfield/ART-1 neural network-based fault detection and isolation,” IEEE Trans. Neural Networks, vol. 5, pp. 890–899, Nov. 1994. [106] W. E. Staib and R. B. Staib, “The intelligence arc furnace controller: A neural network electrode position optimization system for the electric arc furnace,” presented at the IEEE Int. Joint Conf. Neural Networks, New York, NY, 1992. [107] R. Steim, “Preprocessing data for neural networks,” AI Expert, pp. 32–37, Mar. 1993. [108] K. Steinbuch and U. A. W. Piske, “Learning matrices and their applications,” IEEE Trans. Electron. Comput., vol. EC-12, pp. 846–862, Dec. 1963. [109] F. Sun, Z. Sun, and PY. Woo, “Neural network-based adaptive controller design of robotic manipulators with an observer,” IEEE Trans. Neural Networks, vol. 12, pp. 54–67, Jan. 2001. [110] M. K. Sundareshan and C. Askew, “Neural network-assisted variable structure control scheme for control of a flexible manipulator arm,” Automatica, vol. 33, no. 9, pp. 1699–1710, 1997. [111] R. S. Sutton, “Generalization in reinforcement learning: Successful examples using sparse coarse coding,” in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1996, vol. 8, pp. 1038–1044. [112] H. H. Szu, “Automatic fault recognition by image correlation neural network techniques,” IEEE Trans. Ind. Electron., vol. 40, pp. 197–208, Apr. 1993. [113] J. Teeter and M. Y. Chow, “Application of functional link neural network to HVAC thermal dynamic system identification,” IEEE Trans. Ind. Electron., vol. 45, pp. 170–176, Feb. 1998. [114] L. Tsoukalas and J. Reyes-Jimenez, “Hybrid expert system-neural network methodology for nuclear plant monitoring and diagnostics,” in Proc. SPIE Applications of Artificial Intelligence VIII, vol. 1293, Apr. 1990, pp. 1024–1030. [115] R. E. Uhrig, “Application of artificial neural networks in industrial technology,” in Proc. IEEE Int. Conf. Industrial Technology, 1994, pp. 73–77. [116] A. T. Vemuri and M. M. Polycarpou, “Neural-network-based robust fault diagnosis in robotic systems,” IEEE Trans. Neural Networks, vol. 8, pp. 1410–1420, Nov. 1997. [117] G. K. Venayagamoorthy and R. G. Harley, “Experimental studies with a continually online-trained artificial neural network controller for a turbo generator,” in Proc. Int. Joint Conf. Neural Networks, vol. 3, Washington, DC, July 1999, pp. 2158–2163. [118] B. W. Wah and G. J. Li, “A survey on the design of multiprocessing systems for artificial intelligence applications,” IEEE Trans. Syst., Man, Cybern., vol. 19, pp. 667–692, July/Aug. 1989. [119] S. Weerasooriya and M. A. El-Sharkawi, “Identification and control of a DC motor using back-propagation neural networks,” IEEE Trans. Energy Conversion, vol. 6, pp. 663–669, Dec. 1991. [120] P. J. Werbos, “Beyond regression: New tools for prediction and analysis in the behavioral sciences,” Ph.D. dissertation, Harvard Univ., Cambridge, MA, 1974. [121] , “Maximizing long-term gas industry profits in two minutes in lotus using neural network methods,” IEEE Trans. Syst., Man, Cybern., vol. 19, pp. 315–333, Mar./Apr. 1989.

601

[122] J. R. Whiteley, J. F. Davis, A. Mehrotra, and S. C. Ahalt, “Observations and problems applying ART2 for dynamic sensor pattern interpretation,” IEEE Trans. Syst., Man, Cybern. A, vol. 26, pp. 423–437, July 1996. [123] B. Widrow, DARPA Neural Network Study. Fairfax, VA: Armed Forces Communications and Electronics Assoc. Int. Press, 1988. [124] B. Widrow and M. E. Hoff Jr., “Adaptive switching circuits,” 1960 IRE Western Electric Show Conv. Rec., pt. 4, pp. 96–104, Aug. 1960. [125] N. Wiener, Cybernetics. Cambridge, MA: MIT Press, 1961. [126] M. J. Willis, G. A. Montague, D. C. Massimo, A. J. Morris, and M. T. Tham, “Artificial neural networks and their application in process engineering,” in IEE Colloq. Neural Networks for Systems: Principles and Applications, 1991, pp. 71–74. [127] M. Wishart and R. G. Harley, “Identification and control of induction machines using artificial neural networks,” IEEE Trans. Ind. Applicat., vol. 31, pp. 612–619, May/June 1995. [128] J. M. Zurada, Introduction to Artificial Neural Networks. Boston, MA: PWS–Kent, 1995.

Magali R. G. Meireles received the B.E. degree from the Federal University of Minas Gerais, Belo Horizonte, Brazil, in 1986, and the M.Sc. degree from the Federal Center for Technological Education, Belo Horizonte, Brazil, in 1998, both in electrical engineering. She is an Associate Professor in the Mathematics and Statistics Department, Pontific Catholic University of Minas Gerais, Belo Horizonte, Brazil. Her research interests include applied artificial intelligence and engineering education. In 2001, she was a Research Assistant in the Division of Engineering, Colorado School of Mines, Golden, where she conducted research in the Mechatronics Laboratory.

Paulo E. M. Almeida (S’00) received the B.E. and M.Sc. degrees from the Federal University of Minas Gerais, Belo Horizonte, Brazil, in 1992 and 1996, respectively, both in electrical engineering, and the Dr.E. degree from São Paulo University, São Paulo, Brazil. He is an Assistant Professor at the Federal Center for Technological Education of Minas Gerais, Belo Horizonte, Brazil. His research interests are applied artificial intelligence, intelligent control systems, and industrial automation. In 2000–2001, he was a Visiting Scholar in the Division of Engineering, Colorado School of Mines, Golden, where he conducted research in the Mechatronics Laboratory. Dr. Almeida is a member of the Brazilian Automatic Control Society. He received a Student Award and a Best Presentation Award from the IEEE Industrial Electronics Society at the 2001 IEEE IECON, held in Denver, CO.

Marcelo Godoy Simões (S’89–M’95–SM’98) received the B.S. and M.Sc. degrees in electrical engineering from the University of São Paulo, São Paulo, Brazil, in 1985 and 1990, respectively, the Ph.D. degree in electrical engineering from the University of Tennessee, Knoxville, in 1995, and the Livre-Docencia (D.Sc.) degree in mechanical engineering from the University of São Paulo, in 1998. He is currently an Associate Professor at the Colorado School of Mines, Golden, where he is working to establish several research and education activities. His interests are in the research and development of intelligent applications, fuzzy logic and neural networks applications to industrial systems, power electronics, drives, machine control, and distributed generation systems. Dr. Simões is a recipient of a National Science Foundation (NSF)—Faculty Early Career Development (CAREER) Award, which is the NSF’s most prestigious award for new faculty members, recognizing activities of teacher/scholars who are considered most likely to become the academic leaders of the 21st century.