The original idea for this book came from a conference on applications of agricultural expert systems, which may not seem obvious. During the conference, the ceaseless reports and repetitious content made me think that the problems the attendees discussed so intensely, no matter which kind of crop planting was involved, could be thought of as the same problem, i.e. a "functional problem" from the viewpoint of a mathematical expert. To achieve some planting indexes, e.g. output or quality, whatever the crop grown, different means of control performed by the farmers, e.g. reasonable fertilization, control of illumination, temperature, humidity, concentration of CO2, etc., all can be seen as diversified time-varying control processes starting from sowing and ending at harvest. They could just as easily be seen as the inputs for the whole crop growth process.

Process Neural Networks Theory and Applications proposes the concept and model of a process neural network for the first time, showing how it expands the mapping relationship between the input and output of traditional neural networks and enhances the expression capability for practical problems, with broad applicability to solving problems relating to processes in practice. Some theoretical problems such as continuity, functional approximation capability, and computing capability, are closely examined. The application methods, network construction principles, and optimization algorithms of process neural networks in practical fields, such as nonlinear time-varying system modeling, process signal pattern recognition, dynamic system identification, and process forecast, are discussed in detail. The information processing flow and the mapping relationship between inputs and outputs of process neural networks are richly illustrated.

1 Introduction

 1.1 Development of Artificial Intelligence

 1.2 Characteristics of Artificial Intelligent System

 1.3 Computational Intelligence

1.3.1 Fuzzy Computing

1.3.2 Neural Computing

1.3.3 Evolutionary Computing

1.3.4 Combination of the Three Branches

 1.4 Process Neural Networks

 References

2 Artificial Neural Networks

 2.1 Biological Neuron

 2.2 Mathematical Model of a Neuron

 2.3 Feedforward/Feedback Neural Networks

2.3.1 Feedforward/Feedback Neural Network Model

2.3.2 Function Approximation Capability of Feedforward Neural Networks

2.3.3 Computing Capability of Feedforward Neural Networks

2.3.4 Learning Algorithm for Feedforward Neural Networks

2.3.5 Generalization Problem for Feedforward Neural Networks

2.3.6 Applications of Feedforward Neural Networks

  2.4 Fuzzy Neural Networks

2.4.1 Fuzzy Neurons

2.4.2 Fuzzy Neural Networks

 2.5 Nonlinear Aggregation Artificial Neural Networks

2.5.1 Structural Formula Aggregation Artificial Neural Networks

2.5.2 Maximum (or Minimum) Aggregation Artificial Neural Networks

2.5.3 Other Nonlinear Aggregation Artificial Neural Networks

 2.6 Spatio-temporal Aggregation and Process Neural Networks

 2.7 Classification of Artificial Neural Networks

 References

3 Process Neurons

 3.1 Revelation of Biological Neurons

 3.2 Definition of Process Neurons

 3.3 Process Neurons and Functionals

 3.4 Fuzzy Process Neurons

3.4.1 Process Neuron Fuzziness

3.4.2 Fuzzy Process Neurons Constructed using Fuzzy Weighted Reasoning Rule

 3.5 Process Neurons and Compound Functions

 References

4 Feedforward Process Neural Networks

 4.1 Simple Model of a Feedforward Process Neural Network

 4.2 A General Model of a Feedforward Process Neural Network

 4.3 A Process Neural Network Model Based on Weight Function Basis Expansion

 4.4 Basic Theorems of Feedforward Process Neural Networks

4.4.1 Existence of Solutions

4.4.2 Continuity

4.4.3 Functional Approximation Property

4.4.4 Computing Capability

 4.5 Structural Formula Feedforward Process Neural Networks

4.5.1 Structural Formula Process Neurons

4.5.2 Structural Formula Process Neural Network Model

 4.6 Process Neural Networks with Time-varying Functions as Inputs and Outputs

4.6.1 Network Structure

4.6.2 Continuity and Approximation Capability of the Model

 4.7 Continuous Process Neural Networks

4.7.1 Continuous Process Neurons

4.7.2 Continuous Process Neural Network Model

4.7.3 Continuity, Approximation Capability, and Computing Capability of the Model

 4.8 Functional Neural Network

4.8.1 Functional Neuron

4.8.2 Feedforward Functional Neural Network Model

 4.9 Epilogue

 References

5 Learning Algorithms for Process Neural Networks

 5.1 Learning Algorithms Based on the Gradient Descent Method and Newton Descent Method

5.1.1 A General Learning Algorithm Based on Gradient Descent

5.1.2 Learning Algorithm Based on Gradient-Newton Combination

5.1.3 Learning Algorithm Based on the Newton Descent Method

 5.2 Learning Algorithm Based on Orthogonal Basis Expansion

5.2.1 Orthogonal Basis Expansion of Input Functions

5.2.2 Learning Algorithm Derivation

5.2.3 Algorithm Description and Complexity Analysis

 5.3 Learning Algorithm Based on the Fourier Function Transformation

5.3.1 Fourier Orthogonal Basis Expansion of the function in L2[0,2π]

5.3.2 Learning Algorithm Derivation

 5.4 Learning Algorithm Based on the Walsh Function Transformation

5.4.1 Learning Algorithm Based on Discrete Walsh Function Transformation

5.4.2 Learning Algorithm Based on Continuous Walsh Function Transformation

 5.5 Learning Algorithm Based on Spline Function Fitting

5.5.1 Spline Function

5.5.2 Learning Algorithm Derivation

5.5.3 Analysis of the Adaptability and Complexity of a Learning Algorithm

 5.6 Learning Algorithm Based on Rational Square Approximation and Optimal Piecewise Approximation

5.6.1 Learning Algorithm Based on Rational Square Approximation

5.6.2 Learning Algorithm Based on Optimal Piecewise Approximation

 5.7 Epilogue

 References

6 Feedback Process Neural Networks

 6.1 A Three-Layer Feedback Process Neural Network

6.1.1 Network Structure

6.1.2 Learning Algorithm

6.1.3 Stability Analysis

 6.2 Other Feedback Process Neural Networks

6.2.1 Feedback Process Neural Network with Time-varying Functions as Inputs and Outputs

6.2.2 Feedback Process Neural Network for Pattern Classification

6.2.3 Feedback Process Neural Network for Associative Memory Storage

 6.3 Application Examples

 References

7 Multi-aggregation Process Neural Networks

 7.1 Multi-aggregation Process Neuron

 7.2 Multi-aggregation Process Neural Network Model

7.2.1 A General Model of Multi-aggregation Process Neural Network

7.2.2 Multi-aggregation Process Neural Network Model with Multivariate Process Functions as Inputs and Outputs

 7.3 Learning Algorithm

7.3.1 Learning Algorithm of General Models of Multi-aggregation Process Neural Networks

7.3.2 Learning Algorithm of Multi-aggregation Process Neural

 Networks with Multivariate Functions as Inputs and Outputs

 7.4 Application Examples

 7.5 Epilogue

 References

8 Design and Construction of Process Neural Networks

 8.1 Process Neural Networks with Double Hidden Layers

8.1.1 Network Structure

8.1.2 Learning Algorithm

8.1.3 Application Examples

 8.2 Discrete Process Neural Network

8.2.1 Discrete Process Neuron

8.2.2 Discrete Process Neural Network

8.2.3 Learning Algorithm

8.2.4 Application Examples

 8.3 Cascade Process Neural Network

8.3.1 Network Structure

8.3.2 Learning Algorithm

8.3.3 Application Examples

 8.4 Self-organizing Process Neural Network

8.4.1 Network Structure

8.4.2 Learning Algorithm

8.4.3 Application Examples

 8.5 Counter Propagation Process Neural Network

8.5.1 Network Structure

8.5.2 Learning Algorithm

8.5.3 Determination of the Number of Pattern Classifications

8.5.4 Application Examples

 8.6 Radial-Basis Function Process Neural Network

8.6.1 Radial-Basis Process Neuron

8.6.2 Network Structure

8.6.3 Learning Algorithm

8.6.4 Application Examples

 8.7 Epilogue

 References

9 Application of Process Neural Networks

 9.1 Application in Process Modeling

 9.2 Application in Nonlinear System Identification

9.2.1 The Principle of Nonlinear System Identification

9.2.2 The Process Neural Network for System Identification

9.2.3 Nonlinear System Identification Process

 9.3 Application in Process Control

9.3.1 Process Control of Nonlinear System

9.3.2 Design and Solving of Process Controller

9.3.3 Simulation Experiment

 9.4 Application in Clustering and Classification

 9.5 Application in Process Optimization

 9.6 Application in Forecast and Prediction

 9.7 Application in Evaluation and Decision

 9.8 Application in Macro Control

 9.9 Other Applications

 References

Postscript

Index