0
$\begingroup$

I am trying out the scipy Minimizer from the scipy.optimize package. I was just playing around to see how the optimization package works. But I cannot seem to get the Optimizer to work when I supply my own Gradient (Option jac). I Always get value Errors. Here is my example:

import numpy as np
from scipy import optimize

# Simulate own data
x = np.arange(-5,10,1)
theta = np.array([5,2])
noise = np.random.normal(0,1,len(x))
y = theta[0] + theta[1] * x + noise

# the cost function
def cost_function(theta, x, y):
    cost = 0
    theta0, theta1 = theta
    def cost_sum(x_i, y_i):
        cost_component = (y_i - theta0 - theta1*x_i)**2
        return cost_component
    for i in range(len(y)):
        cost += cost_sum(x[i], y[i])
    return cost

# the gradient
def cost_grad(theta, x, y):
    grad = np.zeros((1,2))
    theta_diff_1 = np.array([0.01,0])
    theta_diff_2 = np.array([0,0.01])
    grad[0,0] = ( cost_function(theta + theta_diff_1, x, y) - cost_function(theta - theta_diff_1, x, y) ) / 0.02
    grad[0,1] = ( cost_function(theta + theta_diff_2, x, y) - cost_function(theta - theta_diff_2, x, y) ) / 0.02
    return grad

# call Optimizer from scipy
res = optimize.minimize(cost_function, theta, args=(x,y), method = 'BFGS', jac=cost_grad)
print(res)

This Code works without supplying the gradient. As far as I know, the jacobian is defined to be the Vector of first derivatives of the function (typically as a row vector). So this should have Dimension 1x2 in my example (which it does have). The description says it should have the same Dimension as Theta. However, if I try this, it still does not work.

I would be happy to hear hear your thoughts on this. Many thanks in Advance!

$\endgroup$

closed as off-topic by cpraveen, Anton Menshov, nicoguaro Jan 15 at 1:00

This question appears to be off-topic. The users who voted to close gave this specific reason:

If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ This is what the error message Looks like: ValueError: shapes (2,2) and (1,2) not aligned: 2 (dim 1) != 1 (dim 0) $\endgroup$ – SchaviHamburg Dec 26 '18 at 10:58
  • 1
    $\begingroup$ Could you please edit to format the code correctly? $\endgroup$ – Federico Poloni Dec 26 '18 at 20:43
0
$\begingroup$

It does not have the same dimension as theta. In Numpy, (2) and (1,2) are different shapes.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.