My problem is something similar to this.
I'm trying to implement a (Neural Network) Cost function, Back propogation algorithm in Python.
The Neural Network has 3 layers. Hence 2 parameters to optimize (Theta1 and Theta2; depicted as T1 and T2 in the program).
I tried the solution offered in above link, but doesn't work for me.
my Cost Function is defined with following parameters.
def CostFunction(Thetas, X, y, Lambda = None): T1 = Thetas T2 = Thetas if(Lambda == None): Lambda = 0 m = X.shape # so on....... return J, [Theta1_grad, Theta2_grad]
How would I optimize the above function using
I have the following code, doesn't seem to work:
initial_values = np.zeros()) myargs = (Thetas ,X, y) def decorated_cost(Thetas): return NeuralNetwork.CostFunction(Thetas, X, y, Lambda=1) print scipy.optimize.fmin_bfgs(decorated_cost, Thetas , maxiter=400)
I need to optimize both Theta1 and Theta2 but the API only takes 1 parameter as input.
Also, for some reason the above code (after a couple of iterations) the order of parameters of input to Cost function seem to change when operated by optimize function.