Scribd
Upload
117 views5 pages

Back Propagation Algorithm Explained

Backpropagation is an algorithm used in neural networks to calculate the error contribution of each neuron in the network and use this to update connection weights so as to reduce the overall error. It involves: 1) Calculating the output of the network given input and current weights 2) Computing the error between actual and predicted output 3) Updating weights to reduce error based on the contribution of each neuron to the total error 4) Repeating this process until the error is minimized or falls below a threshold. An example shows initializing weights, feeding input through the network, calculating output error, updating weights proportionally to reduce error, and iterating until error is sufficiently low.

Uploaded by

Anonymous 4iFUXUT
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
117 views5 pages

Back Propagation Algorithm Explained

Backpropagation is an algorithm used in neural networks to calculate the error contribution of each neuron in the network and use this to update connection weights so as to reduce the overall error. It involves: 1) Calculating the output of the network given input and current weights 2) Computing the error between actual and predicted output 3) Updating weights to reduce error based on the contribution of each neuron to the total error 4) Repeating this process until the error is minimized or falls below a threshold. An example shows initializing weights, feeding input through the network, calculating output error, updating weights proportionally to reduce error, and iterating until error is sufficiently low.

Uploaded by

Anonymous 4iFUXUT
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Back Propagation Concept

1
E= * Err 2
2

1
E= * ( yh(x))2
2

Where, y= actual output and

h(x)= calculated output = g*


wj x j
j

xj
wj

E
wj wj
+* W j

E 1
= ( * Err 2 )
W j W j 2

( yg w j x j )
= Err
*(
( j ) ),
W j


g w j x j
= Err
*( W j j
( ) )

= Err
* g
'
* ( w x )
j
j j
* xj

E
wj wj
Therefore, old
+* W j
= wj old
+ *[- Err
* g' * ( w x )
j
j j
* xj

Let, j= - Err
* g' * ( w x )
j
j j

Err g' ( inj )


=- * *

Where, inj =
wj x j
j

wj wj xj
new
= old
+ * j*
Back Propagation Algorithm
1. Initialize all the weights by small random values.
2. Find Output using feed forward network.
i.e. y= g*
wj x j
j

3. Compute error for each output unit, for unit k, it is,


k =( t k y k )g'( y k )

4. Update
w jk =k z k

Where, z k = hidden layer signal or activation.



= learning rate (small constant)
5. Propagate the terms (errors) through the weightsof hidden units
th
where the unit for j hidden unit is
m
inj= k w jk
k=1

th
The delta term for j hidden unit
j= injg'( z inj )

6. Calculate the weight correction term for the hidden unit


w ij = ix i

7. Update
w jk =w jk w jk
w ij =w ij w ij

8. It is Repeated until the weights is adjusted for required output.


(from step 1 to 6)
Example of Back Propagation Algorithm
Let us consider we have 2 input symptoms. If the symptom is selected it
will be encoded as 1 else 0. So we will have 2 hidden neurons and 1
output neuron.
First we initialize the weights to small random value. Then feed the
training sample let
(0 0) through the network and calculate output. Let expected
output is 0.

Since output is not 0, we calculate error for each unit and update
weight.
Taking 0.2,

w13=0.2*(-0.143)*0.57 =-0.0163
w12=0.2*(-0.143)*0.43 =-0.012
Now,

w13(new)=(-0.2)+w13=-0.2163
w12(new)=(0.3)+w12=0.288

Repeating similar steps, until (calculated output - expected output)<0.0001(i.e. error bound)

we obtain new adjusted weights.

You might also like