1
$\begingroup$

I have generated a list of 3 random number where each summed to 1. I would like to assess the quality of randomness. What is the best mechanism to assess this randomness? E.g my random numbers are. Any idea what tools can I use excel for this?

0.4 0.5 0.1
0.2 0.3 0.5
0.6 0.2 0.2
0.5 0.2 0.3
0.2 0.4 0.4
0.2 0.1 0.7
0.3 0.3 0.4
0.8 0.1 0.1
0.1 0.5 0.4
0.4 0.4 0.2
$\endgroup$
  • $\begingroup$ Related: scicomp.stackexchange.com/questions/1922/… $\endgroup$ – Paul Feb 20 '14 at 18:00
  • $\begingroup$ Since the obvious questions haven't been asked yet: where did these numbers come from? Is it possible to generate more of them? 10 data points isn't a lot, so it would be good to know why only 10 data points are available. $\endgroup$ – Geoffrey Irving Feb 21 '14 at 1:50
  • $\begingroup$ Assuming that they sum to one means that there is only 2/3 the possible entropy as before. ie knowing one of the numbers tells you quite a bit about the distribution of the remaining two. Depending on how this assurance was made, it may be more or less concentrated. The additional constraint that they are all positive breaks a lot of assumptions you can make about "normal" distributions, so you have to likely fall back on CDF fitting techniques. I would start by testing the columns individually, then the sums of two columns renormed over the uniform space reamining imposed by the third. $\endgroup$ – meawoppl Feb 23 '14 at 3:08
-1
$\begingroup$

Lets say that you have your numbers in a matrix

x = [...
0.4 0.5 0.1
0.2 0.3 0.5
0.6 0.2 0.2
0.5 0.2 0.3
0.2 0.4 0.4
0.2 0.1 0.7
0.3 0.3 0.4
0.8 0.1 0.1
0.1 0.5 0.4
0.4 0.4 0.2];

where x(i,:) is the i-th row (you 3D random variable). You have to calculate the mean row (I am using matlab notation)

x_mean = mean(x);
disp(x_mean);

Then you can calculate the residual matrix (the difference between your rows and the mean one)

res = x - repmat(x_mean,10,1);

and this matrix res should have the rows distributed as a 3D gaussian variable with zero mean (or uniform, or whatever it is the random distribution you have used to generate your sample).

You have some dependence between your variables, and I dont know how to deal with that. It is probably a good idea to ask this question here CrossValidated.

$\endgroup$
  • $\begingroup$ ok I am moving to cross validated so can I use any mechnanism. I need clarification when you say mean for every row in my case it will be 0.333 cause the total for each row is 1. $\endgroup$ – biz14 Feb 20 '14 at 10:39
  • $\begingroup$ @biz14 I said the mean of the distribution of the residuals (res)should be zero. For sure they will help you much more in cross validated. Good luck. $\endgroup$ – sebas Feb 20 '14 at 11:38
  • $\begingroup$ Why are you assuming that: 1) the variables are uncorrelated, and 2) the distribution is supposed to be Gaussian? Across a row, samples have to be correlated, because they sum to 1! Also, a standard random number generator, like numpy.random.rand generates samples from a uniform distribution, not a Gaussian distribution. $\endgroup$ – Geoff Oxberry Feb 20 '14 at 20:31
  • $\begingroup$ Hi @GeoffOxberry, you can read the answer again. I said: as a 3D gaussian variable with zero mean (or uniform, or whatever it is the random distribution you have used to generate your sample)., and I also said: You have some dependence between your variables, and I dont know how to deal with that. $\endgroup$ – sebas Feb 20 '14 at 21:17
  • $\begingroup$ Actually, I think that you have 2 degrees of freedom, because you can generate the three random numbers independently, and then normalize them (3 dof and 1 constrain). But depending on the way you calculate them it could be even more complicated. I was then suggesting to as in CrossValidated. $\endgroup$ – sebas Feb 20 '14 at 21:19
3
$\begingroup$

Testing whether or not the mean is correct, or even if the histogram of your generated random variants "looks" like a certain distribution is not sufficient. Stick with much more rigorous test suites such as TestU01 or Diehard.

Also, you really only have TWO random numbers in each row, because of the constraint that they sum to 1.

This requires more thought, but I think you may want to test the randomness of the first column, then test the randomness of the second column conditioned on the numbers in the first column.

$\endgroup$
  • $\begingroup$ Agreed here. What really needs to be done is more sophisticated hypothesis testing, which is what TestU01 and Diehard do. I am not sure how to incorporate the constraint that the rows sum to 1. $\endgroup$ – Geoff Oxberry Feb 20 '14 at 20:37
  • $\begingroup$ Here for the constraint: stats.stackexchange.com/questions/87271/… $\endgroup$ – Quartz Feb 26 '14 at 18:16
-1
$\begingroup$

What do you mean by "Quality of Randomness"? In simple terms, if you want to see if the generated set of numbers is random enough, their mean should be close to 0, i.e. a pure gaussian distribution. Note that this is a good option only when you have sufficiently large no. of random samples, in which case getting a mean close to zero indicates that the rand generator has done a pretty good job.

In other words, purely random numbers are uncorrelated with each other, although it almost never happens in practice.

There may be other mathematically rigorous ways to test it, but for simple tasks this would suffice.

$\endgroup$
  • $\begingroup$ I mean quality in terms of saying that some of the first column are high followed by some of the center column are high and last the column. I want to test for even distribution $\endgroup$ – biz14 Feb 20 '14 at 9:47
  • $\begingroup$ If the sum of the numbers has to be 1, then by definition, the random samples are correlated across a row. Also, the OP has not described the distribution of the random variables he is sampling from; it may not be Gaussian. $\endgroup$ – Geoff Oxberry Feb 20 '14 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.