0
$\begingroup$

I have implemented in matlab a neural network that uses rprop's algorithm to update its weights.

Strangely the error on the training set does not converge to a local minimum, but oscillates.

Here is the graphics plot of the error function on trainingset:

enter image description here

Here is the rprop's algorithm:

function [net] = resilientBackPropagation(net, DW, DB, ETA_PLUS, ETA_MINUS)

for i=1:length(net.W) productDW = net.DW{i}.*DW{i}; productDB = net.DB{i}.*DB{i};

indDW_gt_0 = find(productDW > 0);   
indDB_gt_0 = find(productDB > 0);      
indDW_lt_0 = find(productDW < 0); 
indDB_lt_0 = find(productDB < 0); 
indDW_eq_0 = find(productDW == 0);
indDB_eq_0 = find(productDB == 0); 

net.deltaMarginW{i}(indDW_gt_0) = min(ETA_PLUS.*net.deltaMarginW{i}(indDW_gt_0), 50);
net.deltaMarginB{i}(indDB_gt_0) = min(ETA_PLUS.*net.deltaMarginB{i}(indDB_gt_0), 50); 

   net.deltaMarginW{i}(indDW_lt_0) = max(ETA_MINUS.*net.deltaMarginW{i}(indDW_lt_0), exp(-6));
net.deltaMarginB{i}(indDB_lt_0) = max(ETA_MINUS.*net.deltaMarginB{i}(indDB_lt_0), exp(-6));     

    net.W{i}(indDW_gt_0) = net.W{i}(indDW_gt_0)-(sign(DW{i}(indDW_gt_0).*net.deltaMarginW{i}(indDW_gt_0)));
net.B{i}(indDB_gt_0) = net.B{i}(indDB_gt_0)-(sign(DB{i}(indDB_gt_0).*net.deltaMarginB{i}(indDB_gt_0)));   

  DW{i}(indDW_lt_0) = 0;
DB{i}(indDB_lt_0) = 0;  

     net.W{i}(indDW_eq_0) = net.W{i}(indDW_eq_0)-(sign(DW{i}(indDW_eq_0).*net.deltaMarginW{i}(indDW_eq_0)));
net.B{i}(indDB_eq_0) = net.B{i}(indDB_eq_0)-(sign(DB{i}(indDB_eq_0).*net.deltaMarginB{i}(indDB_eq_0)));   

Totale nella Rete Neurale.
net.DW{i} = DW{i};
net.DB{i} = DB{i};

end

What is the problem? thanks

$\endgroup$

marked as duplicate by Geoff Oxberry Sep 2 '15 at 17:43

This question was marked as an exact duplicate of an existing question.

  • $\begingroup$ are you plotting the error on the test set? Is your training data ordered in some way? $\endgroup$ – Memming Sep 2 '15 at 14:46
  • 1
    $\begingroup$ This question has been reasked (using a proper account) at scicomp.stackexchange.com/q/20631; this version is flagged for closing. $\endgroup$ – Christian Clason Sep 2 '15 at 16:35

Browse other questions tagged or ask your own question.