The numbers in the column to the right of each chart refer to the number of bits in the field being tested, and the numbers in the row at the bottom give the position of the most significant bit of each field. Green indicates that the results were "consistent" with the premise that random() produces a sequence of independent, uniformly distributed random numbers. Yellow and light blue results are marginal. Dark blue means that the generator NEVER FAILED the test (out of 1000 tries), whereas it should fail 5% of the time, on average. Orange and especially pink results are clearly unacceptable, while red means that the generator NEVER PASSED the test.

This is a good test for finding undesired structure in random number generator outputs. For 8 bytes of state array, random() performs poorly. With a sequence length of 5, we expect to see the 5! = 120 possible permutations occurring equally often. However, using the 3-bit fields consisting of bits 2-0 and 3-1, we find only 6 of the 120 permutations ever occur, while the field consisting of bits 4, 3, and 2 produces only 4 permutations. Even the 11 permutations found using bits 5, 4, and 3 falls far short of 120. Also, while the 4 permutations in bits 4-2 occur equally often, in the other cases mentioned above, some permutations occurred twice as often as others.

As the field size increases from 3 bits, the number of permutations that occur also increases. However, even for the field consisting of bits 7-0, over 13% of the possible permutations never occurred (with an expected count of 100 for each permutation), and the frequencies of those permutations which did occur varied over a 6-to-1 range. As the field size gets larger or the index of the m.s.b. of the field gets higher, the number of permutations that occur generally increases, and the relative frequencies of those permutations which do occur tend to equalize.

Increasing the size of the state array has an effect similar to increasing the field size or the index of the m.s.b. of the field. In all of the cases for which random() always passed, the results do not appear predictable.

For an explanation of the permutation test, click here.

Click here for more results (larger state arrays).

Return to the testing home page.