Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences
Introduction
Neural networks, or more precisely artificial neural networks, are a branch of artificial intelligence. Multilayer perceptrons form one type of neural network as illustrated in the taxonomy in Fig. 1. This article only considers the multilayer perceptron since a growing number of articles are appearing in the atmospheric literature that cite its use. Many of these papers describe the benefits that neural networks offer when compared to more traditional statistical modelling techniques. Most of the papers briefly describe the workings of neural networks and provide references, to books and papers, from which the reader may obtain further information. This review is aimed at readers with little or no understanding of neural networks and is designed to act as a guide through the literature so that they may better appreciate this tool.
This review is divided into several sections, beginning with a brief introduction to the multilayer perceptron followed by a description of the most basic algorithm for training a multilayer perceptron, known as backpropagation. A review of some of the recent applications of the multilayer perceptron to atmospheric problems will be presented followed by a discussion of some of the common practical problems and limitations associated with a neural network approach.
Section snippets
The multi-layer perceptron: a brief introduction
Environmental modelling involves using a variety of approaches, possibly in combination. Choosing the most suitable approach depends on the complexity of the problem being addressed and the degree to which the problem is understood. Assuming adequate data and computing resources and if a strong theoretical understanding of the problem is available then a full numerical model is perhaps the most desirable solution. However, in general, as the complexity of a problem increases the theoretical
Training a multilayer perceptron—the back-propagation algorithm
Training a multilayer perceptron is the procedure by which the values for the individual weights are determined such that the relationship the network is modelling is accurately resolved. At this point we will consider a simple multilayer perceptron that contains only two weights. For any combination of weights the network error for a given pattern can be defined. By varying the weights through all possible values, and by plotting the errors in three-dimensional space, we end up with a plot
Multilayer perceptron applications in general
The multilayer perceptron has been applied to a wide variety of tasks, all of which can be categorised as either prediction, function approximation, or pattern classification. Prediction involves the forecasting of future trends in a time series of data given current and previous conditions. Function approximation is concerned with modelling the relationship between variables. Pattern classification involves classifying data into discrete classes. All of these applications are closely related
Multilayer perceptron applications in the atmospheric sciences
There is no space to discuss in detail all atmospheric science applications of the multilayer perceptron. Instead a brief overview of applications from prediction, function approximation and pattern classification will be presented. It is hoped that these papers illustrate the main principles of applying the multilayer perceptron to real-world atmospheric problems. Other papers will be mentioned for reference purposes.
Limits, problems and solutions—back-propagation and the multilayer perceptron in practice
The benefits of using multilayer perceptrons have been illustrated. One of the reasons often cited for not using multilayer perceptrons in practice, and artificial neural networks in general, is that they are difficult to implement and interpret. Although this is true to a certain degree, there is an abundance of useful information available that can assist in the process, enabling common pitfalls to be avoided. Commercially available software will often provide built in solutions to protect
Conclusion
The multilayer perceptron has been shown to be a useful tool for prediction, function approximation and classification. The practical benefits of a modelling system that can accurately reproduce any measurable relationship is huge. The benefits of the multilayer perceptron approach are particularly apparent in applications where a full theoretical model cannot be constructed, and especially when dealing with non-linear systems. The numerous difficulties in implementing, training and
Acknowledgements
We are grateful to Gavin Cawley, School of Information Systems, University of East Anglia and the anonymous reviewers for their helpful comments, and also to the School of Environmental Sciences, University of East Anglia, for supporting this work.
References (53)
- et al.
A neural network-based method for the short-term predictions of ambient SO2 concentrations in highly polluted industrial areas of complex terrain
Atmospheric Environment
(1993) - et al.
Development of a neural network model to predict daily solar radiation
Agricultural and Forest Meteorology
(1994) - et al.
Multilayer feedforward networks are universal approximators
Neural Networks
(1989) - et al.
A neural network approach to the classification of electron and proton whistlers
Journal of Atmospheric and Terrestrial Physics
(1996) - et al.
Forecasting carbon monoxide concentrations near a sheltered intersection using video traffic surveillance and neural networks
Transportation Research
(1996) - et al.
A neural network model forecasting for prediction of daily maximum ozone concentration in an industrialised urban area
Environmental Pollution
(1996) - et al.
Wind ambiguity removal by the use of neural network techniques
Journal of Geophysical Research
(1991) Cloud classification of AVHRR imagery in maritime regions using a probabilistic neural network
Journal of Applied Meteorology
(1994)First- and second-order methods for learningbetween steepest descent and Newtons method
Neural Computation
(1992)