A Very Quick Guide to Developing and Training Feed Forward Neural Networks

Among the most important tools deriving from bioinformatics research is the artificial neural network technique (ANN). An ANN is a massively parallel distributed processor able to store experience-based knowledge and make it available for use. It resembles the human brain in two respects: a) knowledge is acquired by the network through a learning process; b) interneuron connection strengths known as synaptic weights are used to store knowledge (Haykin 1994). The ability of ANN to identify mathematical models that closely fit data without the need for mechanistic assumptions is ideally suited for the analysis of complex data sets such as ecosystem variables and DNA microarray hybridization patterns. If properly implemented, the resulting mathematical formulation distinguishes signal from noise in the associations among an arbitrary number of parameters. ANN have a wide range of biological applications, from modeling soil bioremediation (Almeida 1998), to genomic sequence analysis (Baldi and Brunak 1998). Clustering like samples (Gyllenberg and Koski, 1995, Noble et al. 1997) and learning from examples without a priori knowledge of causality (Hinton 1992) are some of the most important uses of ANN. The reader is referred to Hagan et al. (1996) for applied design and implementation of artificial neural networks, to Haykin (1994) for the theoretical foundations, and to Montagne et al. (1994) for a review of contributions in biotechnology.

The steps involved in developing an ANN are 1) assembling the network of unit processors; 2) training them by changing the connection weights by conjugate-gradient minimization of the predictive errors; 3) regularizing the solution in order to ensure its general nature; and finally, 4) repeating the previous steps for the domain of ANN topologies in order to find the optimal extraction of signal versus noise. This procedure is inspired in the process of natural learning and is detailed bellow for a feed-forward ANN architecture:

1. Assembling parallel processing units in fully connected layers

Figure 3
Schematic architecture of a three layer feedforward network used to associate microbial community typing profiles (MCT) with classification vectors. Symbols correspond to neuronal nodes (eq. 1).

Each neuronal node is equated to a sigmoidal function, f, of the product of a weight matrix, w, by the input vector x:

Eq.1

2. Training ANN with cross-validation as a stop criterion, i.e. part of the data are excluded from training and are used to evaluate the generality of predictions.

3. Bootstrapping each topology evaluated.

The validation subset is re-sampled repetitively and the ANN is retrained with cross-validation each time. The median performer is selected. This procedure avoids the possibility that the validation data set may not be representative of the whole.

4. Optimization of topology

The range of topologies is searched for the one with the best performance. The complexity of the topology selected is proportional to the complexity of the signal extracted (Reed and Marks 1999).

References Cited.