I am trying out the scipy Minimizer from the scipy.optimize package. I was just playing around to see how the optimization package works. But I cannot seem to get the Optimizer to work when I supply my own Gradient (Option jac). I Always get value Errors. Here is my example:

import numpy as np
from scipy import optimize

# Simulate own data
x = np.arange(-5,10,1)
theta = np.array([5,2])
noise = np.random.normal(0,1,len(x))
y = theta[0] + theta[1] * x + noise

# the cost function
def cost_function(theta, x, y):
    cost = 0
    theta0, theta1 = theta
    def cost_sum(x_i, y_i):
        cost_component = (y_i - theta0 - theta1*x_i)**2
        return cost_component
    for i in range(len(y)):
        cost += cost_sum(x[i], y[i])
    return cost

# the gradient
def cost_grad(theta, x, y):
    grad = np.zeros((1,2))
    theta_diff_1 = np.array([0.01,0])
    theta_diff_2 = np.array([0,0.01])
    grad[0,0] = ( cost_function(theta + theta_diff_1, x, y) - cost_function(theta - theta_diff_1, x, y) ) / 0.02
    grad[0,1] = ( cost_function(theta + theta_diff_2, x, y) - cost_function(theta - theta_diff_2, x, y) ) / 0.02
    return grad

# call Optimizer from scipy
res = optimize.minimize(cost_function, theta, args=(x,y), method = 'BFGS', jac=cost_grad)

This Code works without supplying the gradient. As far as I know, the jacobian is defined to be the Vector of first derivatives of the function (typically as a row vector). So this should have Dimension 1x2 in my example (which it does have). The description says it should have the same Dimension as Theta. However, if I try this, it still does not work.

I would be happy to hear hear your thoughts on this. Many thanks in Advance!


closed as off-topic by cpraveen, Anton Menshov, nicoguaro Jan 15 at 1:00

This question appears to be off-topic. The users who voted to close gave this specific reason:

If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ This is what the error message Looks like: ValueError: shapes (2,2) and (1,2) not aligned: 2 (dim 1) != 1 (dim 0) $\endgroup$ – SchaviHamburg Dec 26 '18 at 10:58
  • 1
    $\begingroup$ Could you please edit to format the code correctly? $\endgroup$ – Federico Poloni Dec 26 '18 at 20:43

It does not have the same dimension as theta. In Numpy, (2) and (1,2) are different shapes.


Not the answer you're looking for? Browse other questions tagged or ask your own question.