Volume 2, Issue 3 (2014) 676-683
ISSN 2347 - 3258
International Journal of Advance Research and Innovation
677
IJARI
a set of nodes (units) connected together via directed links.
Each node in the network has a numeric activation level
associated with it. The overall pattern vector of activation
represents the current state of the network.
Activation rule is
a local procedure that each node follows in updating its
activation level in the context of input from neighbouring
nodes.
Learning rule is a local procedure that describes how
the weights on connections should be altered as a function
of time. Types of activation functions include: threshold
function; Piecewise-linear function, and sigmoid function.
The sigmoid function, whose graph is s-shaped graph, is by
far the most common form of activation function used in the
construction of neural networks.
A simple process element of the artificial neural
network has three layers; the input, hidden, and output
layers. The input and output layers are defined as nodes, and
the hidden layer provides a relation between the input and
output layers. Initially, the weights of the nodes are random
and the network has not any knowledge. For a given input
pattern, the network produces an associated output pattern.
Its learning and update procedure is based on a relatively
simple concept: the network is provided with both a set of
patterns to be learned and the desired system response for
each pattern. If the network generates the wrong answer,
then the weights are updated to be less error. Finally, future
responses of the network are more likely to be correct.
Regarding the number of layers, the only certainty is that
there should be an input and an output layer so as to be able
to present and obtain data to and from the ANN,
respectively. The number of neurons in each of these two
layers is specified by the number of input and output
parameters that are used to model each problem so it is
readily determined. Therefore, the objective is to find the
number of hidden layers and the number of neurons in each
hidden layer. Unfortunately, it is not possible to
theoretically determine how many hidden layers or neurons
are needed for each problem. The activation functions are
chosen based on the kind of data that are available (binary,
bipolar, decimal, etc.) and the type of layer. For instance,
the identity function is almost always used in the input
layer, while continuous non-linear sigmoid functions are
used in the hidden layers (usually the hyperbolic tangent
function). The training algorithm influences to a far greater
extent the training speed, performance of an ANN (training
error) or the necessary computing power rather than the
architecture itself.
Fig: 1. The Mathematical Model of Neuron
The basic rules are that neurons are added when
training is slow or when the mean squared error is larger
than a specified value, and that neurons are removed when a
change in a neuron‟s value does not correspond to a change
in the network‟s response or when the weight values that are
associated with this neuron remain constant for a large
number of training epochs.
Learning is the process by which the free parameters of
a neural network get adapted through a process of
stimulation by the environment in which the network is
embedded. The type of learning is determined by the
manner in which the parameter changes take place. The set
of well-defined rules of the solution of a learning problem is
called a learning algorithm. Each learning algorithm differ
from the other in the way in which the adjustment to a
synaptic weight of a neuron is formulated. Also, the
manner in which a neural network is made up of inter-
connected neurons relating to its environment, is also to be
considered. There are various learning rules. Hebb‟s
learning rule is the oldest and most famous of all learning
rules. It states that, “when an axon of cell A is near enough
to excite a cell B and repeatedly or persistently takes part in
firing it, some growth process or metabolic changes take
place in one or both cells such that A‟s efficiency as one of
the cells firing B, is increased”. This learning can also be
called correlational learning. This statement may be split
into a two-part rule: 1. If two neurons on either side of a
synapse are activated simultaneously, then the strength of
that synapse is selectively increased. 2. If two neurons on
either side of a synapse are activated simultaneously, then
that synapse is selectively weakened or eliminated. This
type of synapse is called hebbian synapse. The four key
mechanisms that characterize a hebbian synapse are
dependent mechanism, local mechanism, interactive
mechanism and correlational mechanism.
For the perception learning rule, the learning rule, the
learning signal is the difference between the desired and
actual neuron‟s response. This type of learning is
supervised. The fact that the weight vector is perpendicular
to the plane separating the input patterns during the learning
processes, can be used to interpret the degree of difficulty of
training a perceptron for different types of input. There is a
perceptron learning rule convergence theorem which states,
`` if there is a weight vector w* such that f (x (p) w*) = t (p)
for all p, then for any starting vector w
1
the perceptron
learning rule will converge to a weight vector that gives the
correct response for all training patterns, and this will be
done in a finite number of steps”.
Dostları ilə paylaş: