The study of collection, organization, analysis, and interpretation of data.

learn more… | top users | synonyms

1
vote
2answers
35 views

How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions out of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the ...
4
votes
0answers
39 views

Is resampling more accurate than block average for statistical analysis of data?

I'm working in laboratories where molecular dynamics data are almost always analysed usign block average as stated in the famous Allen and Tildesley book. We divide the datas in blocks of size $M$ on ...
0
votes
0answers
17 views

Impulse Response Function - Negative Shocks on R

I have two questions on how to produce impulse responses using R (1) Impulse responses to a negative shock in the independent variable (money supply) (2) Impulse responses at 2 standard deviations ...
0
votes
0answers
49 views

Generating Random Numbers in Fortran for Metropolis method

I am having a really hard time getting any kind of reliable / consistent result from my Metropolis code. I have torn it apart and am now examining just the randomness in my random number generator. I ...
0
votes
1answer
34 views

Force a line through the origin

What does it mean: to force a line through the origin? I interpret this to be making (forcing) the intercept as (0,0) in the regression procedure of an x-y scatterplot, but am not sure.
0
votes
1answer
49 views

Normalize data so that the sum of squares = 1

In presenting geochemical data, I would like to try a statistical method that presents the data in an ISOCON diagram. This method requires scaling all the data to be the same distance from the origin (...
1
vote
0answers
44 views

How to get mathematical model from a data set with MATLAB

I have some values imported from excel about the annual sell of a product, with 3 var: month, price, sold items I have seen how to plot a chart using curve fitting tool but I need to define a ...
6
votes
2answers
90 views

Sum over very small exponentials: Underflow

I am trying to compute (in C) a sum like $S = \sum_i \exp( - a_i )$, where $10^{4} < a_i < 10^{5}$ are approximately normal distributed. So even if I do the Log-Sum-Exp trick $S = \exp(\...
0
votes
1answer
37 views

Conditional Independence (Bayesian Network)

suppose you have this bayesian network: p(a,b,c) = p(c|b) p(b|a) p(a) a -> b -> c are a and c conditionally independent given b? If yes, why are they independent? How can I show that?
2
votes
1answer
55 views

Is there a relationship between the covariance matrix and the partial derivative?

Suppose that there are $N$ pieces of data, each of which contains $M$ parameter values such that $N >> M$. If we put this information into matrix form ($N$ rows, $M$ columns) and then compute ...
1
vote
2answers
232 views

Polynomial approximation

Is there any universal method to fill this matrix for any $n$ value: $\textbf{A} = \left[ \matrix{n & \sum x_i & \sum x_i^2 & \cdots & \sum x_i^n \cr \sum x_i & \sum ...
2
votes
1answer
81 views

CFD turbulence modelling mean pressures vs peak pressures

I have been experimenting with using Autodesk CFD to investigate facade/ cladding pressures on a (rectangular) building, comparing results with cladding/ facades pressures pressures from design codes ...
-1
votes
1answer
88 views

Problem using MATLAB `fminunc` [closed]

I am trying to find the minimum of this function. But I receive the following error when I run the script. What am I doing wrong: ...
1
vote
0answers
22 views

Efficient calculation for L-Kurtosis?

I am doing some statistical signal analysis and was wondering if there are any C/Java packages that do L-moment calculations, specifically L-Kurtosis as I am wanting to do things such as ...
0
votes
1answer
58 views

Can Box-Cox transformation be applied for data of this form?

I have data of the form: X Y 3.53 0 4.93 50 5.53 60 6.21 70 7.37 80 9.98 90 16.56 100 And I want to find out $n$ so that this can be ...
0
votes
0answers
14 views

Calculating win statistics for tennis teams

A tennis club has four teams in three different divisions (two of the teams play in the same division). Players can play for one or more of the teams in a season. In terms of data I have the ...
0
votes
0answers
65 views

How to curve fit an unknown function?

I have data which can be described by $y=f(x,z)$ where $z$ varies from 170 ~ 154. Now values given by $ks$ are known sample values that equals value given in the table header, $uks$ are unknown ...
0
votes
1answer
56 views

Best path for estimation

I have a Cartesian grid (100x100) in which some of the points are known (30 out of 10,000) and the rest are unknown. I want to use the known points and estimate the other cells. Is there any ...
1
vote
1answer
192 views

How to prevent overflow and underflow in the Euclidean distance and Mahalanobis distance

I was working in my project when I was struck by the question of whether it would be necessary, or at least cautious, prevent overflow and underflow in the calculation of these two distances. I ...
0
votes
0answers
97 views

How to choose (and many) threshold in Eigenface?

I edited my question trying to make it as short and precise. I am developing a prototype of a facial recognition system for my Graduation Project. I use Eigenface and my main source is the document ...
0
votes
2answers
129 views

Statistical analysis of optimization algorithms

If we optimize some parameter using 4 optimization algorithms, 2 of which are population based (say A and B) and 2 trajectory methods (single point search)(say C and D); what statistical test can be ...
2
votes
2answers
101 views

fastest and most efficent way to count all combinations in many sets and sum them together

I am a Java programmer who has reached the limits of brute computer power. My relational database (and non relational databases) is not producing results quick enough and I have hit a bottleneck in ...
1
vote
1answer
166 views

What is the algorithm that matlab used in its built-in function 'pca'?

Do anyone know what is the algorithm that MATLAB used in its built-in function "pca"? I have the following data set: 148.9820 55.8438 210.2150 149.3030 56.8891 208.4280 151.4400 ...
0
votes
2answers
123 views

What is 'SOLVER' in R and Statistics/Analytics ?

**strong text**a) I tried to research on what exactly is a SOLVER only to find a not clear-cut simple answer. My doubts still remain after going through several sites full of discussions about it. I ...
2
votes
2answers
140 views

Python: What is a good way to generate a 1D particle field with a gaussian distribution?

If I have N particles how do I assign their x values so that the end result is Gaussian distribution. i.e. particles near the ends are more spread out than particles near the center.
0
votes
0answers
16 views

Model estimation using ACF and PACF

CAN ANYONE HELP IN MODEL ESTIMATION? The following are the ACF,PACF and the plot of the sample respectively.
0
votes
0answers
25 views

Mapping protein sequences on an abscissa?

I have asked this question in the stack forum but I think here, it's more appropriate. What would you do to represent the distribution of small sequences (<=40 residues) according to their ...
1
vote
1answer
249 views

Coding complex equations into C++

I have here equations from a paper by E. Bradlow. They're for counting the events in a Weibull-distributed data set. $$\begin{align} \Pr(N(t)=n) &= \sum_{j=n}^\infty{\frac{(-1)^{j+n}(\lambda t^c)^...
12
votes
3answers
134 views

Correct statistics for reporting speedup results

Say I have slow and fast versions of some code, and want to report a speedup number comparing the two. I run the slow version $n$ times and the fast version $m$ times, producing times $(s_1, \ldots, ...
1
vote
3answers
109 views

Quality of randomness in generated random number

I have generated a list of 3 random number where each summed to 1. I would like to assess the quality of randomness. What is the best mechanism to assess this randomness? E.g my random numbers are. ...
4
votes
1answer
71 views

Is it possible to compute quantiles of a set of numbers, without first sorting those numbers?

I apologize in advances if this is the wrong place to be asking this question, (I had considered putting it on CV first). I am studying financial (S&P, Dow Jones, etc) data, and would like to ...
1
vote
1answer
124 views

Difference powerlaw, lognormal and streteched exponential (Weibull) function

I am currently fitting above mentioned functions to my data and I can observe, that both lognormal and Weibull are better fits than powerlaw. In literature, it is often suggested, that it is hard to ...
5
votes
1answer
150 views

Generation of variable with given auto-correlation function

How can I generate realizations of random complex variable x(t) with given auto-correlation function C(s), defined by C(s) = < x(s)x(0) > and obeying the condition C(-s) = C^*(s) ? Any link ...
1
vote
1answer
618 views

Information from residuals in polynomial regression

I have two physical properties which are exact (no noise). If I perform a polynomial regression of any degree, all coefficients are statistically significant. The problem is that the residuals are ...
1
vote
0answers
68 views

Differential equation - Model Fitting

My question is a quite simple one about statistics. I need to know the following thing. I have a differential equation from my model which I can solve. Now I can compare this solution with the ...
3
votes
0answers
155 views

How to estimate if a velocity field is statistically homogeneous?

For a 3D velocity field $\mathbf{u}$ obtained by direct numerical simulation. Assuming that the field is defined on a periodic domain $\mathcal{P}$ of periodicity $L_x$ in the $x$ direction, $L_y$ ...
1
vote
1answer
2k views

Autocorrelation problems

I have written some code, to produce some data. In what I have shown below, the output is just one-dimensional array of numbers. I have an analytical expression for what the autocorrelation function ...
2
votes
0answers
1k views

What is average pooling

I was reading an article where they have a set of pages. For every page they extract a feature vector of dimention 300. Then they fuse these vectors into one vector by using a method called average ...
6
votes
1answer
167 views

What computational methods would allow me to rank 2D surfaces (with examples)

I have a program which compares the similarity of two images for different positions, so my surface consists of points which correspond to X and Y translations each with a value (mutual information). ...
1
vote
1answer
60 views

Extract clusters from a graph of absolute-distance edge

I don't know if I have formulated this problem right: I have tons of items and the distances between each pair of them. Feeding this data into some visualization tool, I am able to create a nice ...
0
votes
0answers
388 views

matlab program to calculate Maxwell-Boltzmann distribution

I am trying run a Matlab program to number density of Maxwell-Boltzmann statistics (non relativistic limit) $n= \frac{m^2}{2}\pi^2 \exp(\frac{-1}{T})K2(\frac{m}{T})$ where $K2$ is the Bessel ...
7
votes
2answers
2k views

Is there a fast way to compute histograms for high-dimensional large datasets?

Currently the way I compute histograms for data is by generating grid in $N$ dimensions (where $N$ is the dimension of the data) and searching through the $M$ data points in each dimension to see in ...
4
votes
2answers
142 views

Hardware random number generator Vs. Pseudo random number generator in the battlefield of Markov Chain Monte Carlo processes

I'm implementing a Markov Chain Monte Carlo process for a Quantum Monte Carlo routine, in every book and paper I've read so far the success of the routine and quality of the results strongly depends ...
0
votes
0answers
136 views

How to avoid over fitting? An application of GBM boosting package

I distributed my data into 90% training and 10% testing, then I build a boosting regression model using GBM boosting package in R which is very similar to the famous Adaboost regression model. There ...
4
votes
1answer
1k views

TypeError from scipy.optimize.curve_fit

I am trying to fit a data set to an exponential model using scipy. However, the covariance matrix that is returned is 'inf' and I receive the following error: Traceback (most recent call last): ...
3
votes
1answer
46 views

How to obtain the minimum set of variables required in a model to produce accurate estimation?

I have a system which I assume is linear. I have a matrix $A$ of which each row is a coefficient of a unknown variables in vector $x$. I have vector $B$ which contains the result of each $Ax$. ...
-1
votes
1answer
80 views

quantifying interaction between variables in an equation

What do I need to measure interaction between variables in a particular equation? For e.g. Me just taking 50 grams of protein everyday will help me health wise. Me just doing exercise for 1 hour ...
2
votes
0answers
31 views

An asynchronous version of the Covariance Matrix Adaption (CMA)?

Does anyone know of a variant of the CMA algorithm that is suitable for an asynchronous parallel implementation? The conventional version of the algorithm allows one to evaluate the fitness function ...
10
votes
1answer
198 views

Statistical models for local memory/compute, network latency, and bandwidth jitter in HPC

Parallel computation is frequently modeled using a deterministic local rate of computation, latency overhead, and network bandwidth. In reality, these are spatially variable and non-deterministic. ...
1
vote
1answer
223 views

Clustering data based on the radius of a sphere

I have a data-set that has four columns [X Y Z C]. I would like to find all the C values that are in a given sphere centered at [X, Y, Z] with a radius r. What is the best approach to address this ...