I have implemented in matlab a neural network that uses rprop's algorithm to update its weights.
Strangely the error on the training set does not converge to a local minimum, but oscillates.
Here is the graphics plot of the error function on trainingset:
Here is the rprop's algorithm:
function [net] = resilientBackPropagation(net, DW, DB, ETA_PLUS, ETA_MINUS)
for i=1:length(net.W) productDW = net.DW{i}.*DW{i}; productDB = net.DB{i}.*DB{i};
indDW_gt_0 = find(productDW > 0);
indDB_gt_0 = find(productDB > 0);
indDW_lt_0 = find(productDW < 0);
indDB_lt_0 = find(productDB < 0);
indDW_eq_0 = find(productDW == 0);
indDB_eq_0 = find(productDB == 0);
net.deltaMarginW{i}(indDW_gt_0) = min(ETA_PLUS.*net.deltaMarginW{i}(indDW_gt_0), 50);
net.deltaMarginB{i}(indDB_gt_0) = min(ETA_PLUS.*net.deltaMarginB{i}(indDB_gt_0), 50);
net.deltaMarginW{i}(indDW_lt_0) = max(ETA_MINUS.*net.deltaMarginW{i}(indDW_lt_0), exp(-6));
net.deltaMarginB{i}(indDB_lt_0) = max(ETA_MINUS.*net.deltaMarginB{i}(indDB_lt_0), exp(-6));
net.W{i}(indDW_gt_0) = net.W{i}(indDW_gt_0)-(sign(DW{i}(indDW_gt_0).*net.deltaMarginW{i}(indDW_gt_0)));
net.B{i}(indDB_gt_0) = net.B{i}(indDB_gt_0)-(sign(DB{i}(indDB_gt_0).*net.deltaMarginB{i}(indDB_gt_0)));
DW{i}(indDW_lt_0) = 0;
DB{i}(indDB_lt_0) = 0;
net.W{i}(indDW_eq_0) = net.W{i}(indDW_eq_0)-(sign(DW{i}(indDW_eq_0).*net.deltaMarginW{i}(indDW_eq_0)));
net.B{i}(indDB_eq_0) = net.B{i}(indDB_eq_0)-(sign(DB{i}(indDB_eq_0).*net.deltaMarginB{i}(indDB_eq_0)));
Totale nella Rete Neurale.
net.DW{i} = DW{i};
net.DB{i} = DB{i};
end
What is the problem? thanks