Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [statistics]

The study of collection, organization, analysis, and interpretation of data.

Filter by
Sorted by
Tagged with
0 votes
0 answers
22 views

Unable to decipher the error in Anderson-Darling estimation code in R

I just ran the code in R, which estimates some parameters using Anderson-Darling estimation method. The code is given in the appendix of the Habib-Khalil paper. ...
vidyarthi's user avatar
  • 111
0 votes
0 answers
25 views

Sampling from a very high dimensional Gaussian with covariance in block form

I'm interested in sampling from a Gaussian with zero-mean and covariance given by: $$ \Sigma = \begin{bmatrix} \Sigma_{11} & \Sigma_{12} & \cdots &\Sigma_{1,100}\\ \Sigma_{21} & \...
WeakLearner's user avatar
0 votes
0 answers
39 views

What is the minimum error achievable using gaussian process emulation?

I am interested in using Gaussian processes as emulators for other computational models, and I would like to characterize the expected numerical precision of the emulator. Specifically, how small can $...
user9794's user avatar
  • 465
2 votes
1 answer
130 views

Tools to compare two matrices with same dimensions

Context: I have two 3D non-random matrices that have the same dimensions. These matrices represent satellite images with 1 band, so their values are strictly positive. They both present areas that ...
Nihilum's user avatar
  • 121
1 vote
0 answers
30 views

SageMath: How do I plot the sum of a deterministic variable and a random variable

Suppose I have the following function $$f(x,y,z1,z2)=g(x-z1,y-z2)$$ where $g(x,y)$ is a deterministic function from the $X\times Y\to S$ and $z1,z2$ are random variables following a known probability ...
vidyarthi's user avatar
  • 111
0 votes
0 answers
128 views

what is the proper way to update the XY model for a Metropolis MC simulation

I am trying to do a 2D simulation of the classical XY model in order to observe vortexes in the system. I am not really interested at the moment in calculating variables such as Magnetization because ...
Mephistopheles Faust's user avatar
1 vote
0 answers
46 views

Monte Carlo simulation of classical Heisenberg model doesn't represent Curie curve

I've created a JavaScript file to execute and log average energy and magnetization values of 2D lattice classical Heisenberg model. I run the simulation with parameters, ...
M. Çağlar TUFAN's user avatar
-1 votes
1 answer
30 views

How to generate p Sample of GGM of dimension m, for parameter : the weight, the means and the covariance?

after searching in the python numpy, scipy and sklearn module, there is no function who can generate p samples of a gmm (gaussian mixture model) for parameter means, covariances and the weight of each ...
Loca's user avatar
  • 1
0 votes
0 answers
34 views

Algorithm for Probabilities in a "Random" Integers Runs test

Suppose you have a set of N random integers. ( aka bytes 0..255 ) You then consider each subsequent neighbor as "ascending" or "descending", counting the number of consecutive ...
David GSM's user avatar
0 votes
0 answers
34 views

How to model data that is repeated over time within the same group, from which outcome variable is categorical and explanatory variables are linked?

I have 1 dependent variable that is categorical. And I have 3 explanatory variables, the two continuous variables are likely interrelated and the 1 categorical variable is not interlinked. The sample ...
Elly's user avatar
  • 1
1 vote
0 answers
41 views

Where to find datasets to research the optimal parameter search space for HPC systems? [closed]

I am a student researcher who is new to the HPC domain. I have recently taken a project where I am working on optimizing the parameter search space (both application level and hardware level ...
Abrar Hossain's user avatar
1 vote
1 answer
693 views

Beta function and integral value

I have two values $a$ and $b$ where $a \ge 0$ and $b \ge 0$ and I have to calculate the formula below. $$ \frac{1}{2}\int_0^1\text{abs}\left[\left( \frac{p_i^{(a - 1)} \times (1 - p_i)^{(b - 1)}}{\...
Bhavana Reddy's user avatar
2 votes
1 answer
159 views

SciPy ODR "Ordinary" Least Squares?

Scipy.odr has a setting for "fit types", including one for ordinary least-squares. This matches with the documentation of ODRPACK (see p. 31, Computational method). However, the package ...
Bernd's user avatar
  • 21
-1 votes
1 answer
34 views

Parameter explained by many distributions

If we had, for example, labeled data, where for each entry (label) we have several data distributions associated to it, how can I get something meaningful from them? Is this a solvable problem? Is ...
plastico's user avatar
-1 votes
1 answer
113 views

Generating Rvs for a given PDf in python

Two random variables $X$ and $Y$ are distributed according to \begin{align} f_{XY}(x,y)= \begin{cases} x+y & 0\leq x \leq 1, 0\leq y \leq 1 \\ 0 & otherwise \end{cases} \end{align} I ...
leo's user avatar
  • 1
4 votes
0 answers
76 views

Efficient computation of marginalized multivariate normal likelihood

In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms: $$p(\textbf{x}) = \...
nwknoblauch's user avatar
1 vote
1 answer
166 views

Define continuous, non-analytical pdfs in python

I am planning to do some basic algebra on continuous, non-analytical random variabels. I want to define their probability density functions as arrays x and f(x). Yet, I was surprised to find out that ...
Yann's user avatar
  • 159
4 votes
1 answer
211 views

How to define a dimensionless Objective function for determining how peaked a curve is?

I have attached 2 plots for FFT spectra. One is considered good and one is bad. The good one is classified on the basis of how closely spaced the frequencies and the bad is based on how multiple ...
Edwin Rajeev's user avatar
2 votes
0 answers
205 views

Whi are chi-squared distributions in (C++) boost::random and C++ STL faster than in boost::math?

I am trying to generate random chi-squared numbers in C++, according to some degree of freedom (which can be a float). Several libraries can be used for that purpose, among which the C++11 Standard ...
Clej's user avatar
  • 153
2 votes
1 answer
259 views

Best way to convert a sparse (containing zeros) covariance matrix into a correlation matrix?

I have a $100$x$100$ covariance matrix that looks like this. Some rows/cols are all-zero because those corresponding elements are not present in the sample from which covariance is calculated. I'm ...
rtviii's user avatar
  • 151
2 votes
0 answers
149 views

Partial/Extended/Truncated Template Matching

So template matching using correlation is available in a lot of computational packages; OpenCV matchTemplate(), scipy.signal.correlate2d(), IPP CrossCorrNorm, etc. But they all either evaluate ...
RubeRad's user avatar
  • 121
3 votes
1 answer
63 views

Reconstructing statistics of $x\otimes y$ from E[XX'], E[YY'] and E[XY']

I'm looking at random vectors $z$ of size $d^2$ which can be written as $z=x\otimes y$ where $x$,$y$ are random vectors in $\mathbb{R}^d$ with following second moments known -- $E[XX']$, $E[YY']$ and $...
Yaroslav Bulatov's user avatar
7 votes
1 answer
341 views

Computing square root of diag(u)-uu'?

I need an efficient way to take square root of a matrix which is a sum of diagonal matrix and rank-1 matrix. More specifically it's the following matrix $$A=D-uu'=\text{diag}(u)-uu'$$ Where entries ...
Yaroslav Bulatov's user avatar
2 votes
1 answer
807 views

Weighted moving variance

i have a time-series and, in analogy with exponentially weighted moving average, i would like to compute the exponentially weighted moving standard deviation or variance in an efficient, numerically ...
bshanks's user avatar
  • 141
1 vote
0 answers
56 views

Fast convergence of smoothing of periodic noise

I have essentially periodic data from a simulation (not exactly periodic but is qualitatively fairly periodic), and I'd like to take an average or noise filter of some sort that I can get a well ...
EMP's user avatar
  • 2,089
3 votes
2 answers
224 views

How to optimize sampling for parameter estimation

I have a computer model with a number of parameters that need to be calibrated based on experimental results. It's also important to understand the sensitivity of the results to each parameter ...
JNo's user avatar
  • 41
2 votes
0 answers
133 views

difference between PLSA and LDA

What is the difference between Probabilistic Latent Semantic Analysis(pLSA) and Latent Dirichlet Allocation (LDA)?
MIK's user avatar
  • 21
0 votes
0 answers
97 views

how to estimate the number of people on a street within an hour?

I tried to use opencv to analyze a video filmed on the street. But the problem is the performance is not enough. I think the number of people must follow the poisson distribution. So I want to ...
Etronie DENG's user avatar
1 vote
0 answers
147 views

Why does the correlation function of this stochastic differential equation starts at different points?

I am working with the following differential equation: The equation is $$x=\beta +\sqrt{2D} \xi(t)$$ where $\xi(t)$ is a white noise term, with a reflecting wall boundary conditions. After solving ...
George Farah's user avatar
2 votes
1 answer
288 views

Which statistical method should I use for comparing machine run-time of two algorithms?

I am comparing the run-time of two algorithm by solving different instance of the problems. Sample of my data: ...
wahidd's user avatar
  • 23
2 votes
0 answers
32 views

Identifying the time window at which a meteorological factor influences tree growth the most

This is a problem I am encountering in a research project. I have thought of a naive approach to solve it which relies on many assumptions, is not guaranteed to work, and would be tedious and ...
Lance's user avatar
  • 121
0 votes
1 answer
67 views

Is there any robust criteria for this kind of outlier?

I have this data, and I need to detect the outliers. The outliers are clearly visible on the borders (start and end, in red). And I only care for outliers present at the start or end. I could easily ...
yoxota's user avatar
  • 3
1 vote
0 answers
50 views

What is a performant clustering algorithm for approx 10,000 vectors of approx 30 dimension?

I have a set of real-valed vectors, for example $S = \{v_1, v_2, ..., v_k\}$ $v_i = \begin{pmatrix} age_i \\ height_i \\ weight_i \\ ... \end{pmatrix}$ or whatever. Each vector has on the order of ...
spraff's user avatar
  • 111
1 vote
0 answers
23 views

Uniform distribution in 3D space [duplicate]

Posted this at math stack exchange as well, but alas no replies! So, I have been trying to find ways of distributing particles of spherical or other shape in 3D space, e.g. rectangular space. Random() ...
nabber's user avatar
  • 45
4 votes
1 answer
164 views

Using the PAST algorithm to find eigenvectors

I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book: Large pdf. I don't understand the ...
knajp's user avatar
  • 41
2 votes
1 answer
54 views

Verifying that ODE integration generates Theoretical Stationary distribution

I am trying to simulate an ODE, like $ \dot{x} = \xi(x) $ that should have a stationary distribution (a la Stat Mech). Assuming that my ODE algorithm generates time samples of my system state $ x $ ...
user avatar
0 votes
2 answers
64 views

will this methodology end up giving me a nonsense regression equation.

I'm wondering if this is a valid methodology to find the best regression equation for a given data set. User provides a rang of estimated value for some set of variables. Th algorithm uses the ...
Dan Anderson's user avatar
1 vote
2 answers
627 views

Data for Tracy-Widom distribution

I have data for a random variable and I wish to test whether it conforms to the Tracy-Widom distribution. However, the T-W distribution is hard to compute. Is there a readly available table of its ...
Marcel's user avatar
  • 113
2 votes
1 answer
1k views

1-D turbulent energy spectra in homogeneous direction (non-isotropic)

I am trying to compute the one-dimensional energy spectra for my channel-flow simulation. I have already written a post-processing script to achieve this; however, I need to validate my code before ...
debronee101's user avatar
2 votes
1 answer
100 views

Legendre expansion of $r(x) = f(x)/g(x)$ using a finite number of samples from $f(x)$ and $g(x)$

I have two finite sets of events $\{x_1, ..., x_N\}$ and $\{y_1, ..., y_N\}$ that are sampled from the PDFs $f(x)$ and $g(x)$, respectively, where $x \in [-1,+1]$. I want to estimate the Legendre ...
Sam's user avatar
  • 23
1 vote
1 answer
244 views

Data Analysis - Cooling Efficiency

I have a question as I am starting my dive into computational analysis. I have a large set of data (~2 months) which includes the room temperature, HVAC status (heat/cool/off) and the location of ...
Caoder's user avatar
  • 11
1 vote
1 answer
47 views

Parameter identification for regression model

Consider the following regression model : Y = AX + BU where the size of Y is $N \times n$, A is $N \times n$, X is $n \times n$, B is $N \times n$ and U is $n \times 1$. The matrices X,Y and U are ...
user41037's user avatar
2 votes
1 answer
76 views

Using physical parameter as a Gaussian random variable in a simple Poisson problem

I want to vary the input parameter of a physical dynamic mechanics problem, as a Gaussian Random variable and view the resulting Probability Density Function (PDF). I used the Finite Element Method to ...
CRG's user avatar
  • 347
2 votes
1 answer
447 views

Optimize custom probability distribution in Python [closed]

Consider random variables $X$ and $Y$, their distributions are given. $Z = f_a(X, Y)$ where $f(\cdot, \cdot)$ is a deterministic, not random function $f_a: \mathbb{R}^2 \to \mathbb{R}$ depending on a ...
Denis  Korzhenkov's user avatar
7 votes
1 answer
339 views

Tikhonov (Ridge) Regression and Normalization

For a typical Ridge Regression method for solving an inverse problem $$ \min_x ||A~x - b||^2 + \lambda^2||\Gamma~x||^2 $$ Which has an analytical solution of $$ \hat{x}_{est}=(A^TA+\lambda^2 \Gamma^T\...
abnowack's user avatar
  • 255
0 votes
2 answers
43 views

How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions out of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the ...
mhdella's user avatar
  • 111
6 votes
1 answer
518 views

Is resampling more accurate than block average for statistical analysis of data?

I'm working in laboratories where molecular dynamics data are almost always analysed usign block average as stated in the famous Allen and Tildesley book. We divide the datas in blocks of size $M$ on ...
G.Clavier's user avatar
  • 323
1 vote
0 answers
946 views

Generating Random Numbers in Fortran for Metropolis method

I am having a really hard time getting any kind of reliable / consistent result from my Metropolis code. I have torn it apart and am now examining just the randomness in my random number generator. I ...
Joseph's user avatar
  • 131
0 votes
1 answer
190 views

Force a line through the origin

What does it mean: to force a line through the origin? I interpret this to be making (forcing) the intercept as (0,0) in the regression procedure of an x-y scatterplot, but am not sure.
eam's user avatar
  • 1
0 votes
1 answer
6k views

Normalize data so that the sum of squares = 1

In presenting geochemical data, I would like to try a statistical method that presents the data in an ISOCON diagram. This method requires scaling all the data to be the same distance from the origin (...
eam's user avatar
  • 1