Questions tagged [regression]

Regression analysis is the process of measuring and establishing a relationship between a dependent variable and one or more independent variables.

Filter by
Sorted by
Tagged with
-1
votes
0answers
16 views

Can Catboost Regressor be leveraged for ordinal regression?

Customer satisfaction is generally an ordinal variable on a scale of 1 - 5, where 5 is the best score. Ordinal regression is used to regress such variables. Can Catboost Regressor, handle such a ...
2
votes
1answer
121 views

Preconditioning vs. regularization

I used to be more of a numerical linear algebra and computational science person, but recently, I've crossed into stats and machine learning. For this discussion, let's focus on matrices that are not ...
2
votes
1answer
66 views

Linearization of Remez algorithm rational case

In the rational case, we are interested to find polynomials $P(x)$ and $Q(x)$ s.t. $f(x_k)-P(x_k)/Q(x_k)=(-1)^kE$ for $k=1,2,\ldots, N$ where $N=deg(P)+deg(Q)+2$ This can be rewritten as $$ (1)~~~~~~(...
0
votes
3answers
112 views

How can we solve the normal equations with limited memory?

I was asked this open ended question in an interview once: How would you find a solution to the normal equations with limited memory? Unlike Solving sparse least squares system with limited memory, ...
3
votes
1answer
727 views

Clever ways to update LU factorization for ridge regression [duplicate]

Ridge regression can be posed as minimizing the following objective function (over $x$): $$\frac{1}{2} \lVert Ax - b \lVert_2^2 ~+ \frac{\lambda}{2} \lVert x \lVert_2^2 $$ Which has a closed form ...
1
vote
0answers
65 views

Error curve does not oscillate in between reference points using Remez

Using the Remez algorithm, implemented using multi-precision library, in certain functions that I want to approximate, the error curve does not oscillate in between reference points, and so no roots ...
15
votes
3answers
882 views

Fitting Implicit Surfaces to Oriented Point Sets

I have a question regarding quadric fit to a set of points and corresponding normals (or equivalently, tangents). Fitting quadric surfaces to point data is well explored. Some works are as follows: ...
2
votes
0answers
92 views

Remez algorithm convergence

I have implemented the Remez algorithm in Python where all calculations were done with the Python mpmath library. I have noticed that sometimes the $|E_{max}|$ and $|E_{min}|$ do not monotonically ...
4
votes
2answers
8k views

Tikhonov regularization in the non-negative least square - NNLS (python:scipy)

I am working on a project that I need to add a regularization into the NNLS algorithm. Is there a way to add the Tikhonov regularization into the NNLS implementation of scipy [1]? [2] talks about it, ...
0
votes
0answers
32 views

Comparing custom linear regression solver to SciPy equivalent in Python

From a given data set, I set out to complete a task which is below Fit the data of the previous exercise to fit Eq. (8.18) using the SciPy function ...
3
votes
0answers
52 views

Least-squares fit of explicit parabolic sheet to data points

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
2
votes
1answer
58 views

Using linear regression to find the ideal point given a set of trajectory's data

I have a set of points in 2D obtained from a pendular movement with some noise. I want to determine where is equilibrium point ($x_0$, $y_0$) from which the rope is fixed. There are at least two ...
10
votes
6answers
4k views

Seeking a free symbolic regression software

Now that Formulize / Eureqa started charging $2500 a year for using it and having crippled the trial version, does anyone know of any replacements that can do similar things like find an equation ...
2
votes
0answers
111 views

Proving convexity of Frobenius norm and correlation function formulations of an optimization problem

I have been working on formulating my requirements in the form of an optimization problem in a multi-output regression setting. Firstly, I would like to make the variables I used in the problem and ...
0
votes
0answers
66 views

Efficient inversion of multidimensional non linear function

I have a function $f:x\mapsto \vec{y}$ with $x \in [0,1]$ and $\vec{y}=(y_1,...,y_n) \in \mathbb{R}^n$. $n$ is a small integer e.g. $n=8$. Each of the component functions in $y_i(x)$ "oscilate" up and ...
2
votes
0answers
41 views

Good 3D surface fits for multiscale oscillatory surfaces

I have a 3D surface in $x$, $y$, and $z$. where $z$ is a function of $x$ and $y$ and my points are on a structured grid in $x$ and $y$. My function $z$ is highly oscillatory and irregular with ...
1
vote
0answers
29 views

Fitting a multivariate PDE (using Java)

I'm doing simulations of 2 coupled PDE's with Comsol Multiphysics. I want to fit some data (using the Application method, whose language is Java) to those simulations. In order to answer my question ...
0
votes
1answer
86 views

Spline regularization

I am fitting some B-splines to data, but the data has a "gap" region where the spline is less constrained by the data. I want to devise a regularization scheme to help prevent the spline from ...
7
votes
1answer
554 views

Problems Implementing the Remez Algorithm

So first off: *** This code is not being used in production software. It is a personal project of mine, trying to understand approximation theory and advanced curve fitting. In other words, I'm ...
0
votes
1answer
572 views

Power series regression linear fit in VBA excel

I wrote a program that calculates the best fit in VBA excel for the following model $$ y_k=c_1x_k+c_0+c_{-1}(x_k)^{-1} $$ solving for the best fit parameters $c_1$, $c_0$, and $c_{-1}$. However I ...
3
votes
1answer
175 views

Recover curves from noisy collection of points

Background: I'm trying to make a system that tracks a number of bubbles in a video I'm implementing the bubble detection in the single image case using the Circular Hough Transform. Due to occlusion,...
12
votes
2answers
648 views

Quickly finding rough lines in sets of points

In a particular class of detectors, our data comes out as pairs of points in two dimensions, and we want to string these points into lines. The data is noisy, and is binned in one direction but not ...
1
vote
1answer
3k views
-1
votes
1answer
33 views

Minimize squared error of linear function

Let $M$ be a $m \times n$ matrix, $x$ a $n$-vector, $y$ a $m$-vector, and $\|\cdot\|_2$ represent the $L_2$ norm (i.e., Euclidean norm). Given $M,y$, the goal is to find $x$ that minimizes the ...
1
vote
1answer
69 views

Correct weighting in least squares fitting

I am trying to fit some data points $d_i$ to a non-linear model function $m_i$, which depends on a number of fit parameters $f_k$ (I want to determine these) and also on some known, constant values $...
2
votes
2answers
390 views

What equation should I fit this set of data points to?

I have done an experiment Estimation of silver nitrate by potentiometric titration with standard KCl solution. A plot of $\dfrac{\Delta E}{\Delta V}$ versus Volume of KCl solution gives the ...
1
vote
1answer
240 views

Data Analysis - Cooling Efficiency

I have a question as I am starting my dive into computational analysis. I have a large set of data (~2 months) which includes the room temperature, HVAC status (heat/cool/off) and the location of ...
1
vote
1answer
298 views

Compressed sensing: $\ell_0$ “norm” vs $\ell_1$ norm

Suppose we have a very efficient way to perform $\ell_0$ "norm" compressed vs $\ell_1$ norm compressed sensing. Specifically, $\ell_0$ "norm" compressed sensing is $$\eqalign{ & \min \quad {x^T}...
7
votes
1answer
208 views

Methods to Estimate Optimal Distance Measure for Multidimensional Data Set

My problem at hand pertains to choosing a distance measure for use in locally weighted regression. In my particular problem, I have a data set that is upwards of 10 dimensions, where the variables ...
0
votes
1answer
2k views

Numpy.polyfit with regularization

I am trying to use the numpy polyfit method to add regularization to my solution. My non-regularized solution is ...
1
vote
0answers
35 views

fitting exponential versus exponential w/ power

I have two models which I would like to investigate for my data. One form is: \begin{equation} \label{one} f(r) = A e^{-B r} \end{equation} and the second is: \begin{equation} \label{two} g(r)...
2
votes
2answers
48 views

Fitting 2D mapping data from multiple measurements

Given a set of points in a plane, and series of measurements of the distances between those points, how would I go about generating a best-fit model of the position of the points? For example, given ...
1
vote
2answers
327 views

Polynomial approximation

Is there any universal method to fill this matrix for any $n$ value: $\textbf{A} = \left[ \matrix{n & \sum x_i & \sum x_i^2 & \cdots & \sum x_i^n \cr \sum x_i & \sum ...
0
votes
2answers
63 views

Finding parameters numerically

I suspect that a function $f(x,y)$ is of the form $f(x,y)=a(bx+c)^{dy+e}$. I have access to several values of $f(x,y)$. How do I proceed numerically to find the parameters $\{a,b,c,d,e\}$? By ...
0
votes
1answer
31 views

Propagation of error in fitting two sets of data to each other

I have two sets of experimental data: $\phi(t)$ and $I(t)$. In theory they are related to each other as: $\phi(t) = nI(t) $. By fitting these curves together I can find the value of $n$ (which is a ...
0
votes
2answers
61 views

will this methodology end up giving me a nonsense regression equation.

I'm wondering if this is a valid methodology to find the best regression equation for a given data set. User provides a rang of estimated value for some set of variables. Th algorithm uses the ...
1
vote
0answers
49 views

Update model parameter with new data, discarding old data

I have this dataset, and I am using y = (a * x^n) / (b + x^n) Hill function as the model, where a is the limit of the Hill curve,...
1
vote
1answer
34 views

Parameter identification for regression model

Consider the following regression model : Y = AX + BU where the size of Y is $N \times n$, A is $N \times n$, X is $n \times n$, B is $N \times n$ and U is $n \times 1$. The matrices X,Y and U are ...
1
vote
1answer
1k views

Curve fitting for oscillating data

This is my first question. I have the following data that I'd like to approximate as a parametric function: \begin{align} y = a + (bx_1 + cx_2 + dx_3 + ex_1x_2 + fx_1x_3 + gx_2x_3 + hx_1x_2x_3 + i)*(...
1
vote
0answers
47 views

Best way to to find fitting parameters for time series of decaying-growing oscillator type

I have discrete time series emerging from the numerical simulations. It means that the time series can be slightly noisy. The time series should "obey" to the following formula: $$ \psi(t) = \sum_{i=1}...
4
votes
0answers
79 views

Calculus of Variations with unknown cost function but some data

I have a problem that I've framed out in a particular way, but I don't know if I'm re-inventing the wheel here. Is there an existing literature base in this problem? Does it have a corresponding term ...
2
votes
1answer
2k views

Fitting a rectangle to a point set

I have an ordered list of (2d-)points that are forming a (not axis aligned) rectangle and I'd like to recover that rectangle. Approximations like a minimal enclosing rectangle can't be used so that I'...
0
votes
1answer
75 views

For finding the track of an object through space(3d) over time, what is the correct slope equation to use in the algorithm?

I am working on a program that tracks a flying object through space and predicts the future position of said object. I was given some equations to use, but some of them do not look right, mainly the ...
1
vote
2answers
2k views

Finding rate of convergence by curve fitting in Matlab

I have some data: number of nodes $N$ and error in energy norm corresponing to it. I have seen in some references that the rate of convergence is reported by $$\| u-u_h\| _E=CN^{\alpha} $$ How can ...
1
vote
0answers
199 views

Oscillating convergence in my Resilient BackPropagation (RPROP) implementation

I have implemented in matlab a neural network that uses rprop's algorithm to update its weights. Strangely the error on the training set does not converge to a local minimum, but oscillates. Here is ...
0
votes
1answer
138 views

Can Box-Cox transformation be applied for data of this form?

I have data of the form: X Y 3.53 0 4.93 50 5.53 60 6.21 70 7.37 80 9.98 90 16.56 100 And I want to find out $n$ so that this can be ...
1
vote
1answer
507 views

Constrained linear least squares matrix equation

It has been a while since I have done linear least squares, so forgive the simple question, but here goes: I am attempting to find the best fit coefficients, $\{c_i\}$, of a linear combination of ...
0
votes
1answer
2k views

How do I correctly multiply vectors and matrices in Python and MATLAB?

I have been trying for 2-3 days now to get L2 regularized logistric regression to work in Matlab (CVX) and Python(CVXPY) but no success. I am fairly new to convex optimization so I am quite frustrated....
4
votes
0answers
940 views

Neural network performs worse when using more input variables

This question is based more on the theory of neural networks than my particular implementation. Therefore I will leave out my code unless requested. I'm working on a project in C# which can create ...
1
vote
1answer
126 views

What are good parametrizations of rational functions for response surface models?

For fitting a response surface model to a physical process, I have 3-4 relevant "signals", like a feature density, a signal based on a feature width, or a signal based on a distance to the next ...