# Questions tagged [statistics]

The study of collection, organization, analysis, and interpretation of data.

93 questions
Filter by
Sorted by
Tagged with
2answers
26k views

### Define custom probability density function in Python

Is there a way, using some established Python package (e.g. SciPy) to define my own probability density function (without any prior data, just $f(x) = a x + b$), so I can then make calculations with ...
4answers
8k views

### How to add large exponential terms reliably without overflow errors?

A very common problem in Markov Chain Monte Carlo involves computing probabilities that are sum of large exponential terms, $e^{a_1} + e^{a_2} + ...$ where the components of $a$ can range from ...
3answers
221 views

0answers
159 views

1answer
170 views

### Computing square root of diag(u)-uu'?

I need an efficient way to take square root of a matrix which is a sum of diagonal matrix and rank-1 matrix. More specifically it's the following matrix $$A=D-uu'=\text{diag}(u)-uu'$$ Where entries ...
1answer
213 views

### What computational methods would allow me to rank 2D surfaces (with examples)

I have a program which compares the similarity of two images for different positions, so my surface consists of points which correspond to X and Y translations each with a value (mutual information). ...
1answer
361 views

### Is resampling more accurate than block average for statistical analysis of data?

I'm working in laboratories where molecular dynamics data are almost always analysed usign block average as stated in the famous Allen and Tildesley book. We divide the datas in blocks of size $M$ on ...
4answers
562 views

### How can I reduce the error of the sample average?

I want to use the sample average $(X_1 + .... X_n)/n$ as a substitute for the expectation $\mathbb{E}(X)$. As claimed by the weak law of large numbers, as n increases the sample average should ...
3answers
913 views

### Computing the PDF of a quadratic function of two random variables

Given the function $\mathcal{M} = g + Ah + Bh^2$ where $A$ and $B$ are constants and $g$ and $h$ are random variables with their distributions $f_G(g)$ and $f_H(h)$ known, is it possible to compute ...
0answers
135 views

### Probabilistic algorithms for matrix approximation

Considering regular matrix approximation inequality || $A - QQ^TA$|| < e where we try to approximate matrix $A$ by a lower rank orthonormal matrix $Q$. I've read an article on probabilistic ...
4answers
358 views

### Approximate a distribution function from a finite sample

Say I have a simulation that produces a single floating point number as a result, and a different number is produced each time the simulation runs. These numbers are randomly distributed according to ...
2answers
240 views

### Hardware random number generator Vs. Pseudo random number generator in the battlefield of Markov Chain Monte Carlo processes

I'm implementing a Markov Chain Monte Carlo process for a Quantum Monte Carlo routine, in every book and paper I've read so far the success of the routine and quality of the results strongly depends ...
1answer
147 views

### How to define a dimensionless Objective function for determining how peaked a curve is?

I have attached 2 plots for FFT spectra. One is considered good and one is bad. The good one is classified on the basis of how closely spaced the frequencies and the bad is based on how multiple ...
1answer
2k views

### Is there a relationship between the covariance matrix and the partial derivative?

Suppose that there are $N$ pieces of data, each of which contains $M$ parameter values such that $N >> M$. If we put this information into matrix form ($N$ rows, $M$ columns) and then compute ...
1answer
189 views

### Is it possible to compute quantiles of a set of numbers, without first sorting those numbers?

I apologize in advances if this is the wrong place to be asking this question, (I had considered putting it on CV first). I am studying financial (S&P, Dow Jones, etc) data, and would like to ...
1answer
918 views

### Sampling from posterior predictive distribution

First post. I'm working on this problem using Bayesian methods. In desperation I'm considering using p-values (shock horror), specifically posterior predictive p-values. So I need to simulate from the ...
1answer
157 views

### Using the PAST algorithm to find eigenvectors

I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book: Large pdf. I don't understand the ...
1answer
2k views

### TypeError from scipy.optimize.curve_fit

I am trying to fit a data set to an exponential model using scipy. However, the covariance matrix that is returned is 'inf' and I receive the following error: Traceback (most recent call last): ...
0answers
58 views

### Efficient computation of marginalized multivariate normal likelihood

In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms: p(\textbf{x}) = \...
1answer
160 views

### How to add perturbation to a base state in physical grid space?

For study 3D instabilities problem i.e a base state (velocity field and temperature field) and an additional perturbation. The method must be independent of the type of instabilities considered but ...
1answer
793 views

### Computing a rolling quantile

An algorithm I'm writing needs to compute rolling quantiles of a time series. Currently I do this in the naive way: for a window of size W and a vector ...
1answer
56 views

1answer
120 views

### CFD turbulence modelling mean pressures vs peak pressures

I have been experimenting with using Autodesk CFD to investigate facade/ cladding pressures on a (rectangular) building, comparing results with cladding/ facades pressures pressures from design codes ...
1answer
64 views

### Efficient calculation for L-Kurtosis?

I am doing some statistical signal analysis and was wondering if there are any C/Java packages that do L-moment calculations, specifically L-Kurtosis as I am wanting to do things such as ...
1answer
542 views

### How to prevent overflow and underflow in the Euclidean distance and Mahalanobis distance

I was working in my project when I was struck by the question of whether it would be necessary, or at least cautious, prevent overflow and underflow in the calculation of these two distances. I ...