Click here for linear version of these pages.
  PT flag DE flag
About Us
New Account
Access Your Account
Info & Links
ANN Documentation
Data Security
User License Agreement
How to quote
Scientific Publications Related to the microCortex Algorithm
Why ANN?
To Get Started
WalkThrough Guide
Guide: Credit Risk Assessment
Guide: How to Use and Features
Guide: How to Analyse the ANN
Data Types
Very Quick Guide to Neural Networks
ANN: More Details logo
Privacy Statement
Find at our Site
Contact us

A Very Quick Guide to Developing and Training Feed Forward Neural Networks

Jonas Almeida, 1 Oct 99

Among the most important tools deriving from bioinformatics research is the artificial neural network technique (ANN). An ANN is a massively parallel distributed processor able to store experience-based knowledge and make it available for use. It resembles the human brain in two respects: a) knowledge is acquired by the network through a learning process; b) interneuron connection strengths known as synaptic weights are used to store knowledge (Haykin 1994). The ability of ANN to identify mathematical models that closely fit data without the need for mechanistic assumptions is ideally suited for the analysis of complex data sets such as ecosystem variables and DNA microarray hybridization patterns. If properly implemented, the resulting mathematical formulation distinguishes signal from noise in the associations among an arbitrary number of parameters. ANN have a wide range of biological applications, from modeling soil bioremediation (Almeida 1998), to genomic sequence analysis (Baldi and Brunak 1998). Clustering like samples (Gyllenberg and Koski, 1995, Noble et al. 1997) and learning from examples without a priori knowledge of causality (Hinton 1992) are some of the most important uses of ANN. The reader is referred to Hagan et al. (1996) for applied design and implementation of artificial neural networks, to Haykin (1994) for the theoretical foundations, and to Montagne et al. (1994) for a review of contributions in biotechnology.

The steps involved in developing an ANN are 1) assembling the network of unit processors; 2) training them by changing the connection weights by conjugate-gradient minimization of the predictive errors; 3) regularizing the solution in order to ensure its general nature; and finally, 4) repeating the previous steps for the domain of ANN topologies in order to find the optimal extraction of signal versus noise. This procedure is inspired in the process of natural learning and is detailed bellow for a feed-forward ANN architecture:

1. Assembling parallel processing units in fully connected layers

Figure 3
Schematic architecture of a three layer feedforward network used to associate microbial community typing profiles (MCT) with classification vectors. Symbols correspond to neuronal nodes (eq. 1).

Each neuronal node is equated to a sigmoidal function, f, of the product of a weight matrix, w, by the input vector x:


2. Training ANN with cross-validation as a stop criterion, i.e. part of the data are excluded from training and are used to evaluate the generality of predictions.

3. Bootstrapping each topology evaluated.

The validation subset is re-sampled repetitively and the ANN is retrained with cross-validation each time. The median performer is selected. This procedure avoids the possibility that the validation data set may not be representative of the whole.

4. Optimization of topology

The range of topologies is searched for the one with the best performance. The complexity of the topology selected is proportional to the complexity of the signal extracted (Reed and Marks 1999).

References Cited.

  1. Almeida, J.S., K. Leung, S.J. Macnaughton, C. Flemming, M. Wimpee, G. Davis, D.C.White. 1998. Mapping changes in soil microbial community composition signaling bioremediation. Bioremediation J. 1:255-264.
  2. Baldi, P., S. Brunak. 1998. Bioinformatics the machine learning approach. MIT Press
  3. Gyllenberg, M., T. Koski. 1995. A taxonomic associative memory based on neural computation. Binary 7:61-66
  4. Hagan, M.T., H.B. Demuth, and M. Beale. 1996. Neural Network Design. Prindle, Weber & Schmidt Pub., Boston, MA.
  5. Haykin, S. 1994. Neural Networks-a Comprehensive Foundation, pp.397-443. Macmillan College Pub., New York.
  6. Hinton, G.E. 1992. How neural networks learn from experience. Sci. Am. 9:145-151.
  7. Montague, G., J.N. Morris. 1994. Neural Network Contributions in Biotechnology. Trends Biotech. 12:312-324.
  8. Noble, P.A., K.D. Biddle, M. Fletcher. 1997. Natural microbial community compositions compared by a back-propagating neural network and cluster analysis of 5S rRNA. Appl. Environ. Microbiol. 63:1762-1770.
  9. Reed, R.D., R.J. Marks (1999) Neural Smithing supervised learning in feedforward artificial neural networks. MIT Press, Cambridge MS, p.257-264.
  10. Bishop, C.M. 1995. Neural networks for pattern recognition. Clarendon Press, Oxford. P. 364-371
  11. Cheng, B., D.M. Titterington (1994) Neural networks: a review from a statistical perspective. Statistical Science, 9: 2-54.