I am trying to use the numpy polyfit method to add regularization to my solution. My non-regularized solution is

coefficients = np.polynomial.polynomial.polyfit(x,y,5)
ypred = np.polynomial.polynomial.polyval(x,coefficients)

How would I modify this to add L2-regularization?


Here is a piece of code I developed in order to fit polinomials using regularization:

/* x_tr and y_tr is trainig data, x_te and y_te test data */
degree = 4
identity_size = degree + 1
x_tilda = np.array([x_tr, np.ones(len(x_tr))]).transpose()

for degree in range(2, degree + 1): x_tilda = np.append(np.array(np.power(x_tr, degree))[np.newaxis].transpose(), x_tilda, axis=1)

y_reg = y_tr

identidad = np.identity(identity_size)

def get_theta(rho): mul1 = np.linalg.inv( np.add( np.matmul(x_tilda.transpose(), x_tilda), np.dot(identidad, rho))) mul2 = np.matmul(x_tilda.transpose(), y_reg) return np.matmul(mul1, mul2)

def poly(x, rho): theta = get_theta(rho) print("Theta polyfit: ", np.polyfit(x_tr,y_reg, degree)) print("Theta formula: ", get_theta(rho))

p = np.poly1d(theta) return p(x) rho = 0 // this is the regularization param plt.figure() plt.scatter(x_te,poly(x_te, rho), c ='r') plt.scatter(x_te,y_te, c ='b')<code>

Above solution based on the following formula for polynomials (instead of X, it is used poly(X) for a general analytical solution with a squared-error loss.

$$ \hat{\beta} = (X^TX)^{-1}X^TY $$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.