I have been trying for 2-3 days now to get L2 regularized logistric regression to work in Matlab (CVX) and Python(CVXPY) but no success. I am fairly new to convex optimization so I am quite frustrated. Following is the equation that I am trying to solve using CVX/CVXPY. I have taken this equation from the paper https://intentmedia.github.io/assets/2013-10-09-presenting-at-ieee-big-data/pld_js_ieee_bigdata_2013_admm.pdf

In the case of L2 regularized logistic regression the problem becomes: $$ \text{minimize} \frac{1}{m}\sum_{i=1}^{m}\log[1 + \exp(-b_i\mathbf{A}_i^Tx)] + \lambda\Vert x\Vert_2^2$$ where $\lambda$ is the regularization factor.

My Matlab (CVX) code is

function L2
m = 800; N = 5;
lambda =0.000001;

A = load('/path/to/training/file'); 
b= A(:,6); //Label Matrix (800x1)
A = A(:,1:5); //Feature matrix (800x5)

    variable x(N)
    minimize( (1/m * sum( log(1+ exp(-1* A' * (b * x')) ) ) ) + lambda*(norm(x,2)))


CVX returns an error saying which makes sense but the paper mentions the above equation. How can I solve it ?

Your objective function is not a scalar.

After trying on Matlab, I tried on CVXPY. Here is the python code

from cvxopt import solvers, matrix,log, exp,mul
from cvxopt.modeling import op,variable
import numpy as np

n = 5
data = np.ndarray(shape=(m,n), dtype=float,)
bArray = []

file = open('/path/to/training/file')

i = 0;
for line in file:
    for num in line.split():
            data[i][j] = num
            j = j + 1

    i = i + 1

A = matrix(data)
b_mat= matrix(bArray)
m, n = A.size

lamb_default = 0.000001


b = -1*b_mat
w = exp(A.T*b*x)
f = (1/m) + sum(log(1+w)) + lamb_default*mul(x,x)

lp1 = op(f)

I get the error

TypeError: incompatible dimensions

So, my question is: What am I doing wrong in the code for calculation of L2 problem in CVX/CVXPY ?

  • $\begingroup$ In your MATLAB /CVX code, A'(bx') is a matrix rather than a scalar. You'll need to rewrite this to get the objective function that you want. $\endgroup$ May 16, 2015 at 17:03
  • $\begingroup$ Yes, after searching on google I realized that I need to only out a scalar value. I am very new to convex and don't know how to rewrite :( $\endgroup$ May 16, 2015 at 17:07
  • 4
    $\begingroup$ It seems that you have misunderstood the linear algebra in the objective function you have written down. This has nothing to do with programming. The first thing to do is write down the dimensions of each matrix and vector involved. That will help you see what you've misunderstood. $\endgroup$ May 16, 2015 at 17:32
  • $\begingroup$ @DavidKetcheson I actually did the dimension calculation on paper but whatever I did I was not getting a scaler value. The code that I posted was after the frustration :D that something might work :D. It is pretty bad on my part but still :D $\endgroup$ May 17, 2015 at 19:50

1 Answer 1


David Ketcheson has already indicated the problem in his comment. I will flesh it out here.

Note the form of the argument of the log in the logistic regression objective:


where $b_i$ and $x$ are vectors and $A_i$ is a matrix. Only in the above order of matrix-vector multiplications will you get a scalar as an exponent.

The Matlab code should be (thanks for the correction OP):

log(1 + exp(-1*b.*(A * x)))      

This will be a vector which will work with the following sum:

sum(log(1 + exp(-1*b.*(A * x))))

The Python code could be (updated after OP's comment):

log_vec = [log(1+ exp(b[i]*A[i,:].T*x)) for i in range(n)] 
f = (1/m) + sum(log_vec) + lamb_default*mul(x,x)

This list comprehension will work for numpy matrix type. Make sure it works for matrix type of cvxopt.

  • $\begingroup$ thanks for the answer. Python code didn't work and returned "TypeError: incompatible dimensions". Matlab code has a little problem and following fixed it "(1/m * sum( log(1+ exp(-b.* (A * x)) ) ) )" . See the ".*" in the equation. You can update your answer and i will mark it as correct. $\endgroup$ May 17, 2015 at 19:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.