Questions tagged [regression]

Regression analysis is the process of measuring and establishing a relationship between a dependent variable and one or more independent variables.

Filter by
Sorted by
Tagged with
15
votes
3answers
738 views

Fitting Implicit Surfaces to Oriented Point Sets

I have a question regarding quadric fit to a set of points and corresponding normals (or equivalently, tangents). Fitting quadric surfaces to point data is well explored. Some works are as follows: ...
12
votes
2answers
647 views

Quickly finding rough lines in sets of points

In a particular class of detectors, our data comes out as pairs of points in two dimensions, and we want to string these points into lines. The data is noisy, and is binned in one direction but not ...
11
votes
2answers
1k views

Reporting curve-fit results in a scientific paper

(I hope this question fits this site; if not, accept my apologies). I ran a certain simulation, and got a time series y(t), t = 0, 1, ... 20. After trying some functions, I found that: ...
10
votes
2answers
134 views

Matching Similar Items from a Set

I'm trying to match items. Given a set of $n$ items I can rank on a scale from 0 to 100 of how similar they are to one another. For instance, if item $n_1$ is milk and item $n_2$ is also milk, then ...
9
votes
6answers
3k views

Seeking a free symbolic regression software

Now that Formulize / Eureqa started charging $2500 a year for using it and having crippled the trial version, does anyone know of any replacements that can do similar things like find an equation ...
8
votes
3answers
483 views

What should be the criteria for accepting/rejecting singular values?

I am solving a system using singular value decomposition. The singular values (before scaling) are: ...
8
votes
1answer
6k views

Fitting one set of points to another by a rigid motion

I'm not really sure how to explain this problem clearly, so please bear with me. I have a basis of 3 orthonormal unit vectors and a position, a standard 4x4 transform matrix in computer graphics. ...
8
votes
2answers
1k views

Kolmogorov–Smirnov test for multivariate data

I have a set of files consisting of randomly selected points from a dataset, each file belonging to a particular class. Each row in these files contains the coordinates in n-space of the point. I'd ...
7
votes
1answer
457 views

Problems Implementing the Remez Algorithm

So first off: *** This code is not being used in production software. It is a personal project of mine, trying to understand approximation theory and advanced curve fitting. In other words, I'm ...
7
votes
1answer
208 views

Methods to Estimate Optimal Distance Measure for Multidimensional Data Set

My problem at hand pertains to choosing a distance measure for use in locally weighted regression. In my particular problem, I have a data set that is upwards of 10 dimensions, where the variables ...
6
votes
4answers
2k views

parameters estimation

I have to estimate a parameter (K), but I don't know how I can do it. I think by a regression model (minimum least square?), but I'm not sure. The system is: ...
6
votes
4answers
3k views

Surface fitting

I do not need a complete answer but just some advice. I have a sparse matrix of points in a volume. I know a surface passing by these points exists and this surface is mostly flat and relatively ...
5
votes
2answers
355 views

Linear regression with quadratic constraints

What methods are suggested to solve problems of the form $\min || {A} x - y ||_k$, subject to $x^T P x \leq c$, and/or $x^T Q x = d$?
4
votes
2answers
8k views

Tikhonov regularization in the non-negative least square - NNLS (python:scipy)

I am working on a project that I need to add a regularization into the NNLS algorithm. Is there a way to add the Tikhonov regularization into the NNLS implementation of scipy [1]? [2] talks about it, ...
4
votes
2answers
367 views

Can the Levenberg-Marquardt algorithm be used for minimization and not fitting

Can the Levenberg-Marquardt algorithm be used for minimization and not fitting? Usually we input the derivative of the function we want to fit in the minimizer. Now if I assume I have an objective ...
4
votes
2answers
81 views

What are all of the different methods for parameterizing an amino acid (or other small molecule)?

What are all of the different ways to derive the partial charges, van der Waals interactions, bond lengths, etc. of an amino acid (in other words, all of the parameters that could be used in a ...
4
votes
2answers
340 views

best way to optimize a function with linear/non-linear parameters

I am trying to fit some raw data using a function of the form $f(r) = \sum_{i=1}^{K} d_kS_k(n_k,\alpha_k,r)$ where $S_k(n_k,\alpha_k,r) = \frac{\alpha_k ^{n_k+3}}{(n_k+2)!}r^{n_k}\exp(-\alpha_kr)$ ...
4
votes
0answers
78 views

Calculus of Variations with unknown cost function but some data

I have a problem that I've framed out in a particular way, but I don't know if I'm re-inventing the wheel here. Is there an existing literature base in this problem? Does it have a corresponding term ...
4
votes
0answers
818 views

Neural network performs worse when using more input variables

This question is based more on the theory of neural networks than my particular implementation. Therefore I will leave out my code unless requested. I'm working on a project in C# which can create ...
3
votes
2answers
837 views

Linear regression via SVD not producing best fit with escalating polynomial degree

I am using a basic singular value decomposition (via LAPACK) routine in FORTRAN to solve an overdetermined system in the form of $A\cdot X = B$ where $\mathrm{size}(A) = [m,n]$ with $m > n$. My ...
3
votes
1answer
644 views

Clever ways to update LU factorization for ridge regression [duplicate]

Ridge regression can be posed as minimizing the following objective function (over $x$): $$\frac{1}{2} \lVert Ax - b \lVert_2^2 ~+ \frac{\lambda}{2} \lVert x \lVert_2^2 $$ Which has a closed form ...
3
votes
3answers
2k views

Averaging scattered data

I have multiple sets of measured data that can easily be visualized using a scatter plot (red and black points in the figure). If my measurements were perfect, the red and black points should lie on a ...
3
votes
2answers
485 views

When fitting a Gaussian-like function, how does the amount of baseline datapoints affect the fit?

I am fitting a curve to some instrument data. The data is a pulse with a particular functional form, which starts from and returns to a constant (with noise) baseline level before and after the pulse. ...
3
votes
1answer
61 views

How to obtain the minimum set of variables required in a model to produce accurate estimation?

I have a system which I assume is linear. I have a matrix $A$ of which each row is a coefficient of a unknown variables in vector $x$. I have vector $B$ which contains the result of each $Ax$. ...
3
votes
2answers
972 views

How to compute the optimal ridge regression model

I found R function ridge.cv very useful. I would like to implement the equivalent function in MATLAB. As a starting point, I used MATLAB function ...
3
votes
1answer
535 views

Fitting a grid to an STM image

Suppose I have a scan from an STM image (very much like the things you see here). Suppose I have a simple square lattice with lattice parameter a. What I'd like to do is to numerically find the ...
3
votes
1answer
204 views

How to detect specific behavior in time series?

I was not quite sure what the right SE for this was, so I posted this also here on DSP. Please tell me which one to remove :) Problem statement I have a few hundred unrelated time series, say $P_i(t)...
3
votes
2answers
66 views

Parameter Fitting: Need measure of data 'support' for a parameter solution

I am estimating parameters on a dataset that would, for the most part, result in a weakly constrained solution. The dataset however also contains a few more data points that make the solution well-...
3
votes
1answer
173 views

Recover curves from noisy collection of points

Background: I'm trying to make a system that tracks a number of bubbles in a video I'm implementing the bubble detection in the single image case using the Circular Hough Transform. Due to occlusion,...
3
votes
0answers
48 views

Least-squares fit of explicit parabolic sheet to data points

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
3
votes
0answers
1k views

Polynomial Fitting with Least Squares using Numpy and Scipy

I am trying to fit data to a polynomial using Python - Numpy. The points, with lines sketched above them are as in the picture. I am trying to fit those points to a polynomial of 4. or 5. degree. ...
3
votes
0answers
73 views

Why not use this simpler variant of Stepwise Regression?

In stepwise regression, you step predictor by predictor, each time selecting the one with the greatest correlation with the measurement, subtracting greedily to leave a residual with no correlation to ...
3
votes
0answers
79 views

Partitioning Data for Multiple Regression Lines

We're all familiar with traditional least-squares method for constructing a straight line through a set of data points. The question is: suppose I show you a scatter plot which clearly is suggestive ...
2
votes
2answers
253 views

What equation should I fit this set of data points to?

I have done an experiment Estimation of silver nitrate by potentiometric titration with standard KCl solution. A plot of $\dfrac{\Delta E}{\Delta V}$ versus Volume of KCl solution gives the ...
2
votes
1answer
504 views

Polynomial Regression using Semidefinite Programming

I'm trying to design the frequency response function for a low-pass filter. I need the function to be polynomial and to fulfill the following constraints: the coefficients must sum to 1, the function ...
2
votes
2answers
44 views

Fitting 2D mapping data from multiple measurements

Given a set of points in a plane, and series of measurements of the distances between those points, how would I go about generating a best-fit model of the position of the points? For example, given ...
2
votes
1answer
55 views

Using linear regression to find the ideal point given a set of trajectory's data

I have a set of points in 2D obtained from a pendular movement with some noise. I want to determine where is equilibrium point ($x_0$, $y_0$) from which the rope is fixed. There are at least two ...
2
votes
1answer
2k views

Fitting a rectangle to a point set

I have an ordered list of (2d-)points that are forming a (not axis aligned) rectangle and I'd like to recover that rectangle. Approximations like a minimal enclosing rectangle can't be used so that I'...
2
votes
1answer
131 views

fitting a non-linear curve

I have an equation: $\ddot{x}+(\delta+\epsilon\cos{t})x=0$ known as the Mathieu equation.The $\delta-\epsilon$ parameter space of this equation looks something like The red lines in this diagram ...
2
votes
0answers
76 views

Remez algorithm convergence

I have implemented the Remez algorithm in Python where all calculations were done with the Python mpmath library. I have noticed that sometimes the $|E_{max}|$ and $|E_{min}|$ do not monotonically ...
2
votes
0answers
98 views

Proving convexity of Frobenius norm and correlation function formulations of an optimization problem

I have been working on formulating my requirements in the form of an optimization problem in a multi-output regression setting. Firstly, I would like to make the variables I used in the problem and ...
2
votes
0answers
41 views

Good 3D surface fits for multiscale oscillatory surfaces

I have a 3D surface in $x$, $y$, and $z$. where $z$ is a function of $x$ and $y$ and my points are on a structured grid in $x$ and $y$. My function $z$ is highly oscillatory and irregular with ...
2
votes
1answer
790 views

Some questions about MINPACK usage and messages

I am trying to use the nonlinear fitting routines of MINPACK for fitting a rather complicated equation of state to a set of experimental data. A subset of the data is fitted fairly well to a ...
1
vote
2answers
321 views

Polynomial approximation

Is there any universal method to fill this matrix for any $n$ value: $\textbf{A} = \left[ \matrix{n & \sum x_i & \sum x_i^2 & \cdots & \sum x_i^n \cr \sum x_i & \sum ...
1
vote
2answers
2k views

Finding rate of convergence by curve fitting in Matlab

I have some data: number of nodes $N$ and error in energy norm corresponing to it. I have seen in some references that the rate of convergence is reported by $$\| u-u_h\| _E=CN^{\alpha} $$ How can ...
1
vote
1answer
929 views

Curve fitting for oscillating data

This is my first question. I have the following data that I'd like to approximate as a parametric function: \begin{align} y = a + (bx_1 + cx_2 + dx_3 + ex_1x_2 + fx_1x_3 + gx_2x_3 + hx_1x_2x_3 + i)*(...
1
vote
1answer
391 views

Constrained linear least squares matrix equation

It has been a while since I have done linear least squares, so forgive the simple question, but here goes: I am attempting to find the best fit coefficients, $\{c_i\}$, of a linear combination of ...
1
vote
1answer
66 views

Correct weighting in least squares fitting

I am trying to fit some data points $d_i$ to a non-linear model function $m_i$, which depends on a number of fit parameters $f_k$ (I want to determine these) and also on some known, constant values $...
1
vote
1answer
124 views

What are good parametrizations of rational functions for response surface models?

For fitting a response surface model to a physical process, I have 3-4 relevant "signals", like a feature density, a signal based on a feature width, or a signal based on a distance to the next ...
1
vote
1answer
72 views

Least Angle when $\textbf{A}^T\textbf{A}$ is singular

I'm teaching myself this regression stuff, so forgive me if this is a basic question. I can't seem to find a discussion of my particular problem. So I'm least-squares-ing this overdetermined system $\...