A Random Number Generator Test Suite for the C++ ... - ETH Zürich

¡Institute for Theoretical Physics Winter 2003–2004ETH ZürichDiploma ThesisA Random Number Generator Test Suitefor the C++ StandardMario RüttiMarch 10, 2004Supervisor: Prof. M. Troyer †maruetti@comp-phys.org† troyer@phys.ethz.ch

I am grateful to my diploma professor Prof. Matthias Troyer for giving me the opportunityto write this instructive and inspiring diploma thesis. To say nothing of the time he spenthelping me to resolve my (and my computer’s) problems, and his effort to find new andunconventional solutions.My special thanks also go to my office co-worker Manuel Gil for the motivating and amusingdiscussions about our work and his pleasant companionship.I am grateful to Frank Moser who was acting as editor and assisted me in correcting andpolishing my English sentences.I want to apologize to Ariana about lackluster evenings with a friend lost in thought. Thankyou for your support and understanding during this time.Finally, I am grateful to my parents for the tremendous support they gave me during myyears of studies which enabled me to achieve my goals.

to my parentsUrs and Heidi

AbstractThe heart of every Monte Carlo simulation is a source of high quality random numbers andthe generator has to be picked carefully. Since the “Ferrenberg affair” it is known to a broadcommunity that statistical tests alone do not suffice to determine the quality of a generator,but also application-based tests are needed. With the inclusion of an extensible randomnumber library and the definition of a generic interface into the revised C++ standard it willbe important to have access to an extensive C++ random number test suite. Most currentlyavailable test suites are limited to a subset of tests are written in Fortran or C and cannoteasily be used with the C++ random number generator library.In this paper we will present a generic random number test suite written in C++. Theframework is based on the Boost reference implementation of the forthcoming C++ standardrandom number generator library. The Boost implementation so far contains most modernrandom number generators. Employing generic programming techniques the test suite isflexible, easily extensible and can be used with any random number generator library, includingthose written in C and Fortran. Test results are produced in an XML format, whichthrough the use of XSLT transformations allows extraction of summaries or detailed reports,and conversion to HTML, PDF, PostScript or any other format.At this time, the test suite contains a wide range of different test, including the standardtests described by Knuth, Vattulainen’s physical tests, parts of Marsaglia’s Diehard test suite,and a number of number of newer tests.

Contents1. Introduction 12. What are random numbers? 22.1. Types of random numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . 23. Analyzing Statistics 43.1. χ 2 test (“Chi-square” test) . . . . . . . . . . . . . . . . . . . . . . . . . . 43.2. Kolmogorov-Smirnov test (KS test) . . . . . . . . . . . . . . . . . . . . . 63.3. Gaussian Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84. Using the “Random Number Generator Test Suite” 104.1. How to run a test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.2. The rng_test_suite environment . . . . . . . . . . . . . . . . . . . . 114.2.1. Template Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2.2. Confidence Level . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2.3. Seeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.2.4. Random Number Generators . . . . . . . . . . . . . . . . . . . . . 124.2.5. Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.2.6. Running. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.3. Testing Parallel Random Number Generators . . . . . . . . . . . . . . . . 144.4. Iterating a test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.5. Count failings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.6. Bit Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.7. Bit extract test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.8. The XML output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175. Tests for Studying Random Data 185.1. Equidistribution test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.2. Run test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.2.1. Runs up and down . . . . . . . . . . . . . . . . . . . . . . . . . . 195.2.2. Runs above and below mean . . . . . . . . . . . . . . . . . . . . . 225.2.3. Length of runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.3. Gap test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245.4. Poker test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.5. Coupon-collectors test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275.6. Permutation test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295.7. Maximum of t test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305.8. Birthday Spacings test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.9. Collision test (Hash test) . . . . . . . . . . . . . . . . . . . . . . . . . . . 325.10. Serial correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33i

5.11. Serial test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.12. Blocking test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.13. Repeating Time Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.14. gcd test (greatest common divisor) . . . . . . . . . . . . . . . . . . . . . . 365.15. Gorilla test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365.16. Ising-model test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365.17. Random-walk test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375.18. n-block test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.19. Random Walker on a line (S n test) . . . . . . . . . . . . . . . . . . . . . . 405.20. 2D Intersection test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.21. 2D Height Correlation test . . . . . . . . . . . . . . . . . . . . . . . . . . 405.22. Sum of independent distributions test . . . . . . . . . . . . . . . . . . . . . 405.23. Fourier transform test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.24. Universal statistical test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.25. The Diehard Test Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.25.1. Birthday Spacings test . . . . . . . . . . . . . . . . . . . . . . . . 425.25.2. The overlapping 5-permutation test . . . . . . . . . . . . . . . . . 425.25.3. Ranks of binary matrices . . . . . . . . . . . . . . . . . . . . . . . 425.25.4. The bitstream test . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.25.5. The OPSO, OQSO and DNA tests . . . . . . . . . . . . . . . . . . 445.25.6. The count-the-1’s test . . . . . . . . . . . . . . . . . . . . . . . . . 455.25.7. The parking lot test . . . . . . . . . . . . . . . . . . . . . . . . . . 455.25.8. The overlapping sums test . . . . . . . . . . . . . . . . . . . . . . 465.25.9. Squeeze test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.25.10.The Minimum Distance test . . . . . . . . . . . . . . . . . . . . . 475.25.11.Random Sphere test . . . . . . . . . . . . . . . . . . . . . . . . . 475.25.12.The runs test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.25.13.Craps test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486. Extending the Random Number Generator Test Suite 496.1. How to implement a test . . . . . . . . . . . . . . . . . . . . . . . . . . . 496.1.1. Implementing a χ 2 , Kolmogorov-Smirnov or a Gaussian test . . . . 516.1.2. χ 2 test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526.1.3. Kolmogorov-Smirnov test . . . . . . . . . . . . . . . . . . . . . . 526.1.4. Gaussian test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.2. The multiple_test wrapper . . . . . . . . . . . . . . . . . . . . . . . 546.3. Useful sequence diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . 576.4. Demands on Random Number Generators . . . . . . . . . . . . . . . . . . 576.5. Foreign Random Number Generators . . . . . . . . . . . . . . . . . . . . . 576.6. The XML Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61A. Collection of Test Parameters 64B. Examples 66C. Compiling the Test Suite 67ii

1. IntroductionHow random is random?In this diploma thesis a generic random number test suite (RNGTS) is developed. The testsuite framework is written in C++ with attention to modern generic programming paradigm.It is based on the Boost reference implementation of the forthcoming C++ standard randomnumber generator library. The aim of RNGTS is to assist in finding a suitable random numbergenerator for a specific purpose and in deciding between good and bad random numbergenerators. Through a generic interface the RNGTS makes a variety of different tests availableand provides the possibility to extending the suite with user defined tests. The test resultsare produced in XML format, which allows the transformation into summaries or detailedreports through the use of XSLT style sheets. The main purpose is to support the user in hisdecision about a random number generator, and in the question how random the numbersproduced by the random number generators are.In the second part of this paper there is a short discussion about the different types ofrandom numbers and their applications.Then, in the third part the involved statistical methods and their pertaining programminginterface are presented.The fourth part contains the handling of RNGTS. This part is a “must” for the user whowants to perform any tests. It also describes the core of the whole test suite.In the fifth part there is a presentation of the most popular random number generatortests, their parameters and programming interfaces. These tests are collected fromdifferent sources and authors.The sixth part is for “advanced” users who want to extend RNGTS and add new testsor different extensions.Finally, the appendix contains a collection of different lists with test parameters andother useful stuff.DownloadThe RNGTS framework is located on the www.comp-phys.org web server and may bedownloaded there. There are also some installation hints, some examples and the full documentationwith additional interface descriptions and the XSL schema.1

2. What are random numbers?Random numbers are characterised by the fact that their value can not be predicted. Or, inother words, if one constructs a sequence of random numbers, the probability distribution ofthe following random numbers have to be completely independent of all the other generatednumbers.A more sophisticated mathematical definition and discussion can be found in [6].2.1. Types of random numbersThere are three types of random numbers, quasi-, pseudo- and true- random numbers. Thesedifferent types of random numbers have different applications. (It is philosophical questionwhat we can call random or not, but here, we use the following descriptions, its simpler. . . )True Random Number The most often used example for “truly” random numbers is the decay ofa radioactive material. If a Geiger counter is put in front of such a radioactive source, theintervals between the decay events are truly random. True random numbers are gained fromphysical processes like radioactive decay or also rolling a dice. But rolling a dice is difficult,perhaps someone could control the dice so well to determine the outcome.Pseudo Random Number These numbers are generated by a computer or that is to say, by an algorithmand because of this not truly random. Every new number is generated from the previousones by an algorithm. This means that the new value is fully determined by the previous ones.But, depending on the algorithm, they often have properties making them very suitable forsimulations.Quasi Random Number A good description quoted from [25], Chapter 7.7Sequences of n-tuples that fill n-space more uniformly than uncorrelated randompoints are called quasi-random sequences. That term is somewhat of a misnomer,since there is nothing random about quasi-random sequences: They arecleverly crafted to be, in fact, sub-random. The sample points in a quasi-randomsequence are, in a precise sense, maximally avoiding each other.Quasi random numbers are not designed to appear random, rather to be uniformly distributed.One aim of such numbers is to reduce and control errors in Monte Carlo simulations.A picture is always a good way to illustrate the difference between this two types. In figure2.1 1 and 2.2 2 we have plots with different numbers of pseudo- and quasi-random numbers.This is a good demonstration to show the structure of quasi-random numbers, but it is also1 This plot was generated with the Matlab 6 rand generator, a combination of a lagged Fibonacci generator,with a cache of 32 floating point numbers and a shift register random integer generator.2 This plot was generated with the sobol.m routine for Matlab from http://www.csit.fsu.edu/~burkardt/m_src/sobol/sobol.html. This web-site includes also a variety of references for Sobolsequences and some implementations in different programming languages.2

2.1. Types of random numberspossible to see that quasi-random numbers fill continuously the hole plane, while pseudorandomnumbers may build clusters and holes. If we are talking about random numbers inthe following parts, we mean pseudo random numbers.10.80.60.40.200 0.2 0.4 0.6 0.8 1100 Points10.80.60.40.200 0.2 0.4 0.6 0.8 1250 Points10.80.60.40.200 0.2 0.4 0.6 0.8 1500 Points10.80.60.40.200 0.2 0.4 0.6 0.8 11000 PointsFigure 2.1.: Pseudo Random Numbers10.80.60.40.200 0.2 0.4 0.6 0.8 1100 Points10.80.60.40.200 0.2 0.4 0.6 0.8 1250 Points10.80.60.40.200 0.2 0.4 0.6 0.8 1500 Points10.80.60.40.200 0.2 0.4 0.6 0.8 11000 PointsFigure 2.2.: Quasi Random Numbers3

n3. Analyzing StatisticsIn this section we describe the χ 2 test and the Kolmogorov-Smirnov test. Both are designedto check if the measured distribution is similar to the expected distribution. So we cancompare different distributions. Later on we describe the gaussian test which is based on thegaussian normal distribution.A detailed description for the outlined C++ classes can be found in the section aboutimplementing additional tests 6.1.3.1. χ 2 test (“Chi-square” test 1 )The χ 2 -Test is perhaps the best known statistical test. It is based on a comparison betweenthe empirical distribution function and the theoretically expected distribution. The empiricaldistribution is based on the results of the random process.The n measured random values must be divided in k classes I 1 I 2contain N 1¤ 2¤ k¥ ¢¡¢¡¢¡£values.k . The classesIN N NFor each class, the expected number of values must be calculated with the expecteddistribution¡¢¡¢¡¤function p i for a given p i (p p§i¨) i¥ N¦Considering the squares of the differences between the measured values and the expectedvalues gives the χ 2 valuei¥ Nkχ ∑i©1§n i 2¥npnp i¥i¨21nk∑i©12 inp i(3.1)With k classes, there are 1 degrees of freedom in the χ 2 distribution. Lookingup for χ 2 and ν in “χ 2 distribution” tables, which can be found in [16], [3], theprobability being above or below the given χ 2 can be found.k ν¥Calculating the probability of a χ 2 value is not such an easy task, but there is analgorithm published by Hill and Pike, which can be used, see [11], [12], [14].Example: Throwing a dieAfter throwing a die 120 times we get the following resultsvalue 1 2 3 4 5 6# observed 15 19 22 21 17 261 Sometimes the “χ 2 test” stands for the Equidistribution test.4

¥3.1. χ 2 test (“Chi-square” test)There is no reason to change the 6 natural classes I 1 I 2 I 6 . The number ofvalues is 120. For a true die we expect a probability of p for each die-numberi¥ n¥ ¢¡¢¡¢¡£ k¥The expected number of values is np n¦i¥20 i¥The χ 2 value is calculated by the following sum.χ 2k∑i©1§n i ¥npnp i20¨22076§15 ¤3¡8020¥ ¥i¨220¨220¤ §1920¨220¤ §220¨220¤ §17 §211620¨220¤20¨220§26Here we have classes. This means that the number of degrees of freedom is5. Looking up for χ 3¡80 in a table, the value lies between 50% and 75%.This means that we will have a χ 2 3¡80 between 25% and 50% of the time. Therandomness observed in this experiment is satisfactory in this test.2¥ ν¥ k¥6Available codeTo handle the χ 2 statistics there is the chisquare_test class, which provides differentmethods used for the calculation. Some important methods are listed in the declarationbelow. The class is defined in the chisquare_test.h file.class chisquare_test{void prepare_statistics(std::size_t count_size,uint64_t runs,std::size_t degOfFreedom = 0);templatevoid calculate_chisquare_value(ForwardIterator first,ForwardIterator last,std::size_t degOfFreedom);templatevoid calculate_chisquare_value(ForwardIterator first, ForwardIterator last);void set_chisquare_value(double chiSquareValue, std::size_t degOfFreedom);chiSqr_stat_type get_chisquare_value();double get_chisquare_prob();}In the same file there is also a function to calculate the χ 2 value without class stuff.templatedouble calc_chisquare_value(ForwardIterator first,ForwardIterator last,UnaryFunction probability,std::size_t degOfFreedom)To calculate the probability from a χ 2 value in the file chisqr_prob.h file there is afunction managing this task.double chi_probability(double chisqr, int dof)5

ii¨3. Analyzing Statistics3.2. Kolmogorov-Smirnov test (KS test)As we have seen, the χ 2 test be applied when observations can fall into a finite number ofcategories. But normally one will consider random quantities which may assume an infinitenumber of values. In this test, the random number generators distribution function F iscompared to the expected distribution function F§x¨. In [16], Knuth defined this functionsas follows:n§x¨F§x¨ ¥ probabilityF n§x¨that§X of X 1 X 2 ¢¡¢¡¢¡£ x¨nn which are xXThe n measured random values must be sorted in ascending order, X 1 X 2 ¡¢¡¢¡n¥numberTo make the test, we form the following statistics:K¡n ¥n max£ ∞¤ x¤ ∞§F ¢ n§x¨F§x¨¢¨ ¥¥¢ n max1¤ i¤ nK £ n ¥n max£ ∞¤ x¤ ¢ F∞§F§x¨n§x¨¢¨¢ n max1¤ i¤ n¥F§X i¨1n iXn F§XLike in the χ 2 -test, we may now look up the values K¡n , K £ n in a table [16] to determineif they are significantly high or low. An other way is to calculate the probabilities bythe algorithm given in [1] and in chapter 3.3.1, “C. History, bibliography, and theory”of [16]In [16] there is also formula given to calculate the probability exactlyprob§K¦ntttnn∑ nk©0k ¥ n¨¢§k t¨k§t¤ n k¨n£ k£ 1(3.2)Example: 10 random numbersWe got 10 numbers from a random number generator. These are {0.809, 0.465, 0.151, 0.628,0.318, 0.824, 0.394, 0.968, 0.179, 0.458}First we sort the random numbers X i ascending orderCalculate the K¡ quantities i and K i£and find the maximum of these quantities6

3.2. Kolmogorov-Smirnov test (KS test)i X i K¡£i K i1 0.151 0.051 0.1512 0.179 0.021 0.0793 0.318 0.018 0.1184 0.394 0.006 0.0945 0.458 0.042 0.0586 0.465 0.135 0.0357 0.628 0.072 0.0288 0.809 0.009 0.1099 0.824 0.076 0.02410 0.968 0.032 0.068With these values we calculate K¡10 and 10as followsK£K £ 10¥ ¢ n maxi¨ ¥ ¢ 10 0¡151¥ 0¡4781¤ i¤ n§K¡K¡10¥ ¢ n maxi¨ ¥ ¢ 10 0¡135¥ 0¡427If we look up these values in an appropriate table for 10, we find that the chanceto get a K¦10 greater then 0¡427 or 0¡478 lies between 50% and 75%.n¥1¤ i¤ n§K £Available codeTo calculate the Kolmogorov-Smirnov statistics there is a class which supports the requiredroutines. The definition of this class called ks_test is found in ks_test.h. Someimportant methods are listed belowclass ks_test{void prepare_statistics(uint64_t runs);templatevoid calculate_ks_value(ForwardIterator first, ForwardIterator last);templatevoid ks_value(ForwardIterator first,ForwardIterator last,UnaryFunction integratedProbDistr);ks_stat_type get_ks_value();ks_prob_type get_ks_prob();}There is also a function to calculate the KS values.templatestd::paircalc_ks_value(ForwardIterator first,ForwardIterator last,UnaryFunction integratedProbDistr)To calculate the probability for a KS value the following function is defined in the fileks_prob.h.boost::tuple ks_probability(int n, std::pair ksPair)7

3. Analyzing Statisticspercent factor10.40.895%5%0.30.20.195%0.650% 5%0.40.2-3 -2 -1 1 2 3Σ-3 -2 -1 1 2 3meanΣFigure 3.1.: Gaussian distributionFigure 3.2.: Percentage function3.3. Gaussian TestThe Gaussian test is a little different from the χ 2 or the Kolmogorov-Smirnov test. In thesetwo tests the expected distribution function is compared with the measured distribution functionand based on the difference some indicators are calculated.In the Gaussian test a physical view is used. If a measurement is done, it is known that,even if the best tools are used, the result depends on a number of ruleless and uncontrolledparameters. These measurement errors are random and a combination of different singleerrors.The central limit theorem predicates that the measured value behaves like a normal distributedrandom variable (This is valid in the normal case). The normalized density functionis written as1 1 µ2§xσ¨2 2πσ§ e£ ¢¥ f§x¨¡ x ¡ ∞¨ (3.3)∞where µ is the mean of expected value and σ the standard deviation.To make a classification of measured values one can compare the deviation from the expectedvalue with the standard deviation. It can be calculated that in the interval σ µ¤ µ ¢68.3 % of all measured values are expected. If the interval is expanded to 3σ µ µ¤ 3σ£σ£we expect 99.7% of all measured values in this range. Based on this theory it is possible togive a possibility for a measured value. The assumption is that the expected value and thedeviation are known. The deviation factor is calculated with the following ¢ formula:1 1 1erf perc§x¨2where erf denotes the¥2“error ¤ function”2z t 2π 0 dt. In this formula we define themean value (expected value) as 50 %, if the deviation is positive a percentage value biggererf§z¨¥than 50 % results or if the deviation is negative, a percentage value smaller than 50 % results.The function is shown in e£ figure 3.2.¥(3.4)¢ 2x8

3.3. Gaussian TestExample: Ising model test statisticWe run the Ising model test described later on and check the result. From the simulationwe get a specific energy of 1.45183 whereas a value of 1.45306 is expected. The standarddeviation is calculated as 0.0037. This results in a deviation from the mean of -0.3324σ.This result can be converted in percent and one gets 36.98 % from the mean value. Thatmeans that only 36.98 % of the measured values will be smaller than this value.Available codeTo get some support calculating the gaussian statistics there is a class called gaussian_testin the file gaussian_test.h. The declarations of the most important methods are listedbelow.class gaussian_test{void prepare_statistics(double deviation, double stat_value, double mean);void calc_gaussian_value();double get_gaussian_prob();}9

4. Using the “Random Number Generator Test Suite”This section describes how to use the “Random Number Generator Test Suit” (RNGTS) withthe available wrappers and helpers.The aim was to supply a simple but enough powerful interface to build a flexible systemto test different types of random number generators with different tests. But also to allowthe generation of various kind of result representation through using a universal XML outputformat.4.1. How to run a testTesting a random number generator is simple, the only requirement for the generator is that itfulfils the Boost Pseudo-random number engine requirements. This can be found in http://www.boost.org/libs/random/wg21-proposal.html written by Jens Maurer.The listing below shows a exemplary test program.// include Boosts random number generator#include // definition to show progress during the test#define PRINT_STATUS// include the test suite environment#include "rng_test_suite.h"// include all header of used tests#include "poker_test.h"#include "ising_model_test.h"int main(){// import random number generator from Boostusing boost::lagged_fibonacci44497;using boost::mt19937;// create a ’TestSuite’ using uint32_t seedsrng_test_suite testSuite;// add desired confidence leveltestSuite.add_confidence_level(0.05);testSuite.add_confidence_level(0.95);testSuite.add_confidence_level(0.1);testSuite.add_confidence_level(0.9);// add desired seedstestSuite.add_seed(314159265);testSuite.add_seed(236598);testSuite.add_seed(1237);// register the random number generator to test10

4.2. The rng_test_suite environmenttestSuite.register_rng("Lagged Fibonacci 44497");testSuite.register_rng("mt19937, Mersenne Twister", 10000);// create the test objectpoker_test pokerTest(100000, 5);ising_model_test isingTest(1000000, 16);// register the teststestSuite.register_test(isingTest);testSuite.register_test(pokerTest);// run tests...// specify destination for writing the XML output, write output into a filestd::ofstream file_out("test_output.xml");// runs all tests and catches possible exceptionstry{// catch possible logic_error exceptionstestSuite.run_test(file_out, true);} catch (std::exception& e){std::cout

4. Using the “Random Number Generator Test Suite”4.2.3. SeedsAdding seeds is not such an easy task, because the Pseudo-random number engine requirementsdoes only specify the iterator based seeding, nothing else. But most generators supportalso a seed(seedType) method. So it is possible to add multiple seeds to use with thegenerators. If a generator does not support the seed(seedType) method the test suiteuses a pseudo-DES algorithm (see [25], sec. 7.5) to create a set of numbers and feeds thesenumbers into the generator with the mandatory iterator based seed method.void add_seed(uint32_t seed)The other way to seed the generator is filling its buffer with values. To do this thereis the seed(iterator, iterator) method. This method must be supported by allgenerators from Boost. The user has to check himself if there are enough values between thetwo iterators to fill the buffer. If there are insufficient values an exception might be thrown.template void add_seed_iterators(const seedIter begin, const seedIter end) type of iteratorbegin iterator to the begin of the buffer with seedsend iterator to the end of the buffer with seeds starting the testBefore a random number generator is seeded, it is reset to the initial state. This means tothe same state as it was while adding to the test suite. This guarantees the repeatability fordifferent seeding. If no seeds are added, the tests run with the initial state of the generator.If a generator has to be tested in a special state, e. g. with a special seeded buffer, there isthe method register_seeded_rng to handle this case.4.2.4. Random Number GeneratorsTo register the random number generators which have to be tested, the test suite provides thefollowing two methods. (The requirements of a random number generator are described insection 6.4)template void register_rng(std::string rng_name, uint64_t warmup = 0) type of the random number generator to testrng_name name of the generator, should be uniquewarmup number of random numbers to produce with the generator before starting the testThis method takes the type of the random number generator as a template parameter. Theconcrete generator object is created inside the test suite with the default constructor. Theseed calls are done in this initial state.template void register_seeded_rng(T mrng,std::string rng_name,std::string description,uint64_t warmup = 0)12

4.2. The rng_test_suite environment type of the random number generator to testmrng object of the random number generator of type Trng_name name of the generator, should be uniquedescription a description of the seed-statewarmup number of random numbers to produce with the generator before starting the testThis method takes an object of a random number generator as a parameter. So it is possibleto use pre-seeded generators. For most generators, all further operations are done on this stateof the generator. This is valid if and only if the generator class does not have external links,e. g. function pointers. If a foreign random number generator (see 6.5) is used, the generatorwill not be seeded before the test - it remains in the previous state.4.2.5. TestsAdding a test is really simple, just create an object of the test class and add it to the test suite.This is done with the following method.template void register_test(T test) type of the testtest test to add to collection of tests to performIt is important to note that the test must be in a ’ready to run’ state when it is added to thetest suit, because the test suit calls only the run method and nothing before.4.2.6. Running. . .If all desired generators, seeds and test are added to the test suite the test can be run by callingthe run_test method. One has to specify where to write the XML output to. Writing tothe terminal is as simple as using a file as target to write to. The second argument specifiesif logic errors, thrown by a test, are caught or not. As an example, an exception may bethrown if one tries to make a binary rank test for matrices bigger than the number of bits ofthe random number generator. If the exception is not caught, the test suite stops and does notfinish the other tests. If the exception is caught, the test is omitted and the test suit continueits work.void run_test(std::ostream& out, bool catch_logic_errors = true)out ostream to write the XML output tocatch_logic_errors specifies if logic_errors thrown by tests are caught or notThe run_test method should be in a try-block, there are sources which throws exceptions.The order of testing all seeds is the following:user seeded generatorsseed a generator with seed(s)seed a generator with seed(it, it)13

4. Using the “Random Number Generator Test Suite”4.3. Testing Parallel Random Number GeneratorsTo test a parallel application using different random number generators in different threads,there is a class called parallel_rng_imitator(from parallel_rng_imitator.h)which simulates such an application. The class contains a collection of definable generatorsand calls one after another. This generator fulfils the Boost specification and can be used ina normal way.There are some preconditions to keep in mind when using such a random number generator.All random number generators used in this parallel random number generator musthave the same result_type. Unfortunately the boost::uniform_01type doesnot support an default constructor, so it is not possible to map the result type to an othertype. To do this, a converter which fulfils the specified interface for generators has tobe written.All random number generators must have the same maximum and minimum value.Pre-seeded random number generators should be favoured because of a better controlover seeding the particular generators.// include Boosts RNGs#include // include parallel generator#include "parallel_rng_imitator.h"// import RNGs from Boostusing boost::minstd_rand0;using boost::lagged_fibonacci19937;using boost::lagged_fibonacci23209;using boost::lagged_fibonacci44497;using boost::mt19937;using boost::ecuyer1988;// make a RNG from two different Lagged Fibonacci RNGsparallel_rng_imitator parallelRNG;// does not compile, because the enlisted generators do not// have same result_typeparallel_rng_imitator parallelRNG_error_compile;// does compile, but throws an exception because the RNGs// does not have same min() or max() valueparallel_rng_imitator

4.4. Iterating a testminstd_rand // max = 2147483646> parallelRNG_error_runtime;4.4. Iterating a testThe idiom says that “Once doesn’t count”. So, we have to repeat a test multiple times andmake a statistic over all results. (Probably we also like to repeat this repetition. . . )This class iterates a given test n times and calculates a Kolmogorov-Smirnov statistic overall results. This is only possible if the test to iterate is derived from the chisquare_test,ks_test or gaussian_test base class. The iteration of a χ 2 or a gaussian test give anormal K-S statistic. But if we have to do this for a K-S test itself, we get four values, K¡and K £ for the original K¡ and the same for the original K £ .The iterate_test fulfils the test interface and acts like a normal test.template< class Test >iterate_test(Test test, std::size_t iterations) type of the testtest test to iterateiterations number of times to iterate the test4.5. Count failingsAnother way to decide about success or failure is to count the failings of each test andcompare with a maximal number of failures.This class iterates a given test n times and count the number of failings. If the test failsmore than the failLimit allows, then it will fail, else the test is passed. (Mathematically,failings failLimit) This is only possible if the test to iterate is derived from thechisquare_test, ks_test or gaussian_test base class. The iteration of a χ 2 ora gaussian test gives one value for failings, the K-S test variation results in two values, onefor K¡ and one for K £ .The count_fails_test fulfils the test interface and acts like a normal test.template< class Test >count_fails_test(Test test, std::size_t iterations, std::size_t failLimit) type of the testtest test to iterateiterations number of times to iterate the testfailLimit Limit deciding between failure or success4.6. Bit TestsIn some kind of tests, like in the “count-the-1’s” test from the Diehard test suite, overlappingranges of bits are tested. From each random number some new particular numbers are15

4. Using the “Random Number Generator Test Suite”built. This is done by masking the bit representation of the number with a specific maskwhich is shift from the least significant bit to the most significant bit. An example of splittingup a number in overlapping sub-numbers is given in figure 4.1. This class is calledoriginal = 180Bits 3..0 = 4Bits 4..1 = 10Bits 5..2 = 13Bits 6..3 = 6Bits 7..4 = 111 0 1 1 0 1 0 001 00 11 0 11 1 0 11 1 01 1000Figure 4.1.: Bit Test, Example of Bit Concatenationrng_bit_test and is located in the same denominated header file. This wrapper canonly be used if the test is derived from one of the given base classes (chisquare_test,ks_test or gaussian_test). The interface istemplaterng_bit_test(TEST test) type of the test number of bits for each random numbertest test to use for bit testExampleAs an example we want to know if a sequence of each 10 bits is uniformly distributed in aχ 2 sense. We have to create a test object, pass this to the wrapper an register the test.chisqr_uniformity_test chi_uni_test(200000, 10);rng_bit_test bit_chi_uni_test(chi_uni_test);rngTest.register_test(bit_chi_uni_test);4.7. Bit extract testAnother way to test a generator is to extract only a specific range of bits from each generatedrandom number and interpret this bits as a new number. In figure 4.2 bits 2¡¢¡¢¡5 are used tomake a new number. Or, we take a specific bit of a number of random numbers and interpretthis bits as a new number. In figure 4.3 this is done with bit 5. To build a new random numberbit five of six consecutive random numbers are used.This tests are supported by two wrappers in rng_bit_extract.h.templatebit_extract(std::size_t b=10240) type of random number generator16

4.8. The XML outputOriginal1801948913411Bit5 Bit 2 Selected0 1 1 0 1 0 01 0 0 0 0 1 00 1 0 1 1 0 0 11 1 1 0 1 0 1 0Figure 4.2.: Extracting subsequencesas nextrandom numbers130610Original180194891341952111Bit 50 1 1 0 1 0 01 0 0 0 0 1 00 1 0 1 1 0 0 11 1 1 0 1 0 1 01 1 0 0 0 0 1 10 0 0 1 0 1 0 1Selected 36Figure 4.3.: Concatenating singlebits to the next randomnumber first bit of new random number number of bits of new random numberb buffer size of random number generatortemplatebit_sequence(std::size_t b=10240) type of random number generator bit to use for random number number of bits for each random numberb test to use for bit test4.8. The XML outputThe result of every test is written out on a specific stream. This stream may be defined in therun_Test(std::ostream, bool catch_logic_errors = true) method. Theoutput may be written onto the console via std::cout or, better for further processing, toa file. To write the output in a file, one has to create a file like this:#include std::ofstream fileOut("results.xml");For a more detailed description about the XML-schema see 6.6.17

5. Tests for Studying Random DataIn this section we present different tests to study the behavior of random number generators.We can distinguish two different sorts of tests, statistical tests and physical test 1 .The only difference is the motivation to do the test. In the first case, we want to know thebehavior of some statistical properties, in the second case, we simulate a physical system.(Strictly speaking there are some more tests like “visual tests” or “theoretical test”. But wedo not look at them because of lack of automatism). Each of these tests checks a specialproperty of the generated numbers against the theoretically expected behavior. These testsare not my invention, I only collected them and add examples of usage to it. A reference tothe source (not source code) is mentioned with each test.Table 5.1 lists many known random number generator tests and its occurrence in oftencited test-benches. It is impossible to list all tests, there are an infinite number of them, sowe mention the most popular ones.A more interesting table for testers is table 5.2. It shows all available 2 tests in the test suiteand their class names. (The name of the header file is the concatenation of class name and.h).5.1. Equidistribution testIn this test we check if the generated numbers are equally distributed. See [16].The N measured random values in the interval ¢ α;β£ must be divided in k classesI 1 I 2 ¢¡¢¡¢¡£k . The classes contains N N 1¤N ¡¢¡¢¡¤ 2¤k¥values.For each class, the expected number is calculated with the assumption that all valuesappear with the same probability k Np¥INCheck the probability with the χ 2 test for the classes and use the KS test to check thewhole data.α¨ §β1 A nice description of physical tests is given in [29]Passing several tests does not prove the randomness of any sequence, however. This is dueto the fact that proving randomness requires that the sequence fulfils an actual definition forrandomness. An unfortunate fact is, however, that there is no unique definition for randomness.[...] Therefore, passing many tests is never a sufficient condition for the use of any pseudo randomnumber generator in all applications. In other words, in addition to standard tests, efficientapplication specific tests of randomness are also needed. This need is emphasized by recentsimulations, in which some physical models combined with special algorithms have been foundwhich are very sensitive to the quality of random numbers.2 I hope that by the time this paper is published the list will already be updated with further implementations18

5.2. Run testExample: Throwing a dieAn example for the χ 2 part of this test is given in section 3.1 and the example for the KS testcan be found in section 3.2.Constructor in chisqr_uniformity_test.hchisqr_uniformity_test(uint64_t n, std::size_t classes)n number of numbers to countclasses number of classes to range in random numbersConstructor in ks_uniformity_testks_uniformity_test(std::size_t n)n number of random numbers to count5.2. Run testIn this test, we are looking for monotone subsequences of the original sequence, which arecalled runs. There are three different sorts of tests. We can count “runs up” and “runs down”,“runs above” and “runs below” the mean or the “length of runs”.As an example of a run, consider the sequence of eleven numbers {3 1 4 1 5 9 2 6 5 3 5}.To show the “runs up” we put a vertical line at the left and right and between X i and X i¡ 1whenever X i X i¡ 1. Here we get | 3 | 1 4 | 1 5 9 | 2 6 | 5 | 3 5 |5.2.1. Runs up and downSplit the sequence of random numbers into increasing and decreasing subsequencesand count the sequences n inc¤n dec¥ NIf N has an adequate size, the mean and variance are given byσ16N 2¥µ2N a¥132990For N 20, the distribution of a is reasonably approximated by a normal distribution,N§µ a σa¨.Converting 2 to a standardized normal distribution byZ 0¥σ aa390Failure to reject the hypothesis of independence occurs when z α¡ 2 Z 0 z α¡ 2,where α is the level of significancea µ¥ a §2N1¨§16N 29¨19

5. Tests for Studying Random DataTestAvailable in Test-BenchKnuth 1 Helsinki 2 Diehard 3 SPRNG 4Equidistribution Test (Frequency ¢ ¢ ¢Test)Gap ¢ ¢ ¢TestIsing Model ¢Testn-block testSerial ¢ ¢ ¢TestPoker Test (Partition ¢ ¢Test)Coupon collector’s ¢ ¢TestPermutation ¢ ¢TestRun ¢ ¢ ¢TestMaximum of t ¢ ¢ ¢TestCollision Test (Hash ¢ ¢ ¢Test)Serial correlation ¢TestBirthday-Spacing’s ¢ ¢TestOverlapping Permutations ¢TestRanks of 31 31 and 32 32 matrices ¢TestRanks of 6 8 Matrices ¢TestMonkey Tests on 20-bit ¢WordsMonkey Tests OPSO, OQSO, ¢DNACount the 1‘s in a Stream of ¢BytesCount the 1‘s in Specific ¢BytesParking Lot ¢TestMinimum Distance ¢TestRandom Spheres ¢TestThe Sqeeze ¢TestOverlapping Sums ¢TestThe Craps ¢TestSum of distributions (for parallel ¢streams)¢FFTBlocking ¢Test2-d Random ¢WalkRandom Walkers on a line (S n Test)2D Intersection Test2D Height Correlation TestRepeating Time TestGorilla Testgcd TestMaurers Universal Test¢ ¨ § ¢ ¨ §1 [16]2 [29]3 [18], [19]4 [21]Figure 5.1.: Compilation of known tests20

5.2. Run testTest Class Name DescriptionEquidistribution Test (Frequency Test) ks_uniformity_test 5.1chisqr_uniformity_test 5.1Gap Test gap_test 5.3Ising Model Test ising_model_test 5.16n-block test n_block_test 5.18Serial Test serial_test 5.11Poker Test (Partition Test) poker_test 5.4Coupon collector’s Test coupon_collector_test 5.5Permutation Test permutation_test 5.6Run Test runs_test 5.2.3Maximum of t Test max_of_t_test 5.7Collision Test (Hash Test) collision_test 5.9Serial correlation Test serial_correlation_test 5.10Birthday-Spacing’s Test birthday_spacing_test 5.8Overlapping Permutations Test 5.25.2Ranks of 31 31 and 32 32 matrices Test bin_rank_chisqr_test 5.25.3Ranks of 6 8 Matrices Test bin_rank_ks_test 5.25.3Monkey Tests on 20-bit Words 5.25.4Monkey Tests OPSO,OQSO,DNA 5.25.5Count the 1‘s in a Stream of Bytes 5.25.6Count the 1‘s in Specific Bytes 5.25.6Parking Lot Test 5.25.7Minimum Distance Test minimum_distance_test 5.25.10Random Spheres Test random_sphere_test 5.25.11The Sqeeze Test squeeze_test 5.25.9Overlapping Sums Test 5.25.8The Craps Test craps_test 5.25.13Sum of distributions (for parallel streams) 5.22FFT 5.23Blocking Test 5.122-d Random Walk random_walk_test 5.17Random Walkers on a line (S_n Test) 5.192D Intersection Test 5.202D Height Correlation Test height_corr2d_test 5.21Repeating Time Test 5.13Gorilla Test 5.15GCD Test 5.14Maurers Universal Test 5.24Figure 5.2.: Available tests in the RNGTS framework21

¡2n a n b¢ 2n a n b£ N£5. Tests for Studying Random DataExample:If a sequence of numbers has to few runs, it is unlikely that it is a real random sequence.If we look at the following sequence, {0.12, 0.35, 0.38, 0.45, 0.51, 0.69, 0.77, 0.78,0.90, 0.93} we can only find one run up. It is not likely to be a random sequence.If a sequence of numbers has too many runs, it is unlikely to be a real random sequence.Look at the sequence {0.08, 0.93, 0.15, 0.96, 0.26, 0.84, 0.28, 0.79, 0.36, 0.57}. If wesplit this sequence into “runs up” and “runs down”, we will find the following.0¡08 0¡93 0¡15 0¡96 0¡26 0¡84 0¡28 0¡79 0¡36 0¡57 five runs up0¡08 0¡93 0¡15 0¡96 0¡26 0¡84 0¡28 0¡79 0¡36 0¡57 four runs downIt has nine runs, five up and four down.5.2.2. Runs above and below meanThis test is an addition to the “Runs up and down” test (5.2.1). It’s easy to build a sequence,with the first 20 numbers above mean while the following 20 numbers are below the mean,which does not fail the “Runs up and down” test. So we have to check the behaviour of theruns above and below the mean.Calculate the mean of the sequence of random numbersSplit this sequence into subsequences above and below the mean and count the numberof runs below n b and above n a . r is the total number of runs.The mean and variance of r can be expressed asσ 2 2nr¥a n b2nµNa n b§2n a nN¤ r¥ b 2§N12For either n a or n b greater than 20, r is approximately normally distributed1¨ N¨§2nZb 0¥a n b N¨2N 2 ¢ N£ 1£Failure to reject the hypothesis of independence occurs when z α¡ 2 Z 0 z α¡ 2,where α is the level of significance1Example:We have the following sequence of random numbers. {0.78, 0.49, 0.41, 0.58, 0.82, 0.26,0.30, 0.06, 0.36, 0.01}.Calculating the mean gives 0¡408 µ¥22

5.2. Run testSplitting up in subsequences above and below the mean gives the following situation:| 0.78 0.49 0.41 0.58 0.82 | 0.26 0.30 0.06 0.36 0.01 |In this case one run is above, one below the mean. It is not likely to be a randomsequence.5.2.3. Length of runsThis test is an addition to the last two tests. It’s still possible to create a sequence of numberswhich passes the last two tests, but the probability that this sequence is truly random is verysmall. Such a sequence may be a run of two numbers below the mean, then a run of twonumbers above the mean and so on. So we need to test the randomness of the length of runs.Split the sequence into subsequences in one of the given manner above where N is thenumber of samplesStore the number of runs of length i into RUN[i]Here, we should not apply a χ 2 -test to the data stored in RUN. This is because adjacentruns are not independent. A long run will tend to be followed by a short run, andvice-versa. So, the statistic should be computed as following61N j©1§RUN[i] ∑Nb i¨§RUN[j] nb j¨a i j (5.1)iThe coefficients a i j and b i can be found in [16], there is also a method shown tocalculate the coefficients for arbitrary maximal run length.Example: Length of “runs up”We have a random sequence: {3 1 4 1 5 9 2 6 5 3 5}Marking the “runs up” in the sequence produces | 3 | 1 4 | 1 5 9 | 2 6 | 5 | 3 5 |We get the following “statistic”– 1 run of length 3– 3 runs of length 2– 2 run of length 1Constructor in runs_test.hruns_test(uint64_t n, std::size_t maxRunLength)n number of random numbers to check for runsmaxRunLength run length above this length are cumulated23

5. Tests for Studying Random DataInternally, this test has to invert a matrix. This functionality is supported by the LAPACKlibrary and the matrix handling is covered with routines from BLAS. The Boost interface forthis two libraries is not yet in the official release, but available in the “Boost-Sandbox”. Touse this test, the “Boost-Sandbox” must be installed which is also available at [2] at “SandboxCVS”.5.3. Gap testThis test is used to examine the length of “gaps” between occurrences of samples in a certainrange. It determines the length of consecutive subsequences with samples not in a specificrange. The algorithm to count the gap length is found in [16].Define an interval ¢ α;β£ with 0 α ¡ β 1Define a list to save the number of occurrence of gaps with length l, where 0 l t.This is easily done with a structure like COUNT[l]. With every occurrence of asequence of length l, do COUNT[l] = COUNT[l]+1. If l is bigger than t, increaseCOUNT[t].Search a subsequence X i X i¡ 1 i¡ l of the random sequence X 0 X 1X i¡ l lies in but the other X’s do not. This subsequence of l¤ α;β¨ ¢ ¢¡¢¡¢¡£ ¢¡¢¡¢¡£N in whichnumbers representsa gap of length l. This increases the number in COUNT[l]After enough samples are tested, the χ 2 -test is applied to the 1 values ofCOUNT[0], COUNT[1], . . . COUNT[t], using the following probabilities:t¤ k¥XX1p p 0¥p p p¨ p§1 1¥p¨2 p§1 2¥p t £ 1¥ p§1 p¨t£ 1p t¥¡¢¡¢¡§1 p¨tHere p¥the probability that α X i¡β.The gap test can be applied with 0 or 0 to facilitate the test-procedure. The special1case§α §0 of§12give rise to the “runs above mean” or “runs below mean” test.This is not the same implementation of the test as used in [29]. They use n randomnumbers and count the number of gaps, this algorithm produces random numbers until ngaps were counted. An approximative conversion from one test to the other is possible withan estimation for the number of gaps within n random numbers. gaps1¨ 2¨ ¥ β¨ β¥ α¥β α,Example:We have the following sequence: {0.11, 0.83, 0.56, 0.95, 0.88, 0.73, 0.91, 0.01, 0.75, 0.67,0.23, 0.38}α¨ n§βIn this case we would take the first two numbers to determine the interval. This meansor ¢ 0¡11;0¡83£ .β¥The sequence to check is {0.56, 0.95, 0.88, 0.73, 0.91, 0.01, 0.75, 0.67, 0.23, 0.38}24α¥ 0¡110¡83

5.4. Poker testThe first value lies in the interval, the next two values not. This means that the gaplengthis 2. Marked in the sequence, with bold letters for values in the interval andnumbers to count the gap-length, the sequence looks as following:0¡56 0 0¡95 1 0¡88 2 0¡73 0 0¡91 1 0¡01 2 0¡75 0 0¡67 0 0¡23 0 0¡38Calculating the probabilities with 0¡83 p¥0¡11¥ 0¡72 and a total of five gapst p t expected # of gaps counted # of gaps0 0.72 3.60 31 0.2016 1.01 02 0.0564 0.28 23 0.0158 0.08 0Constructor in gap_test.hgap_test(std::size_t n,double lowerGapLimit,double upperGapLimit,std::size_t maxGapCount)n number of random numbers to countlowerGapLimit start of gap (α)upperGapLimit end of gap (β)maxGapCount number of steps counted until they are cumulated5.4. Poker testThe “original” poker test considers n groups of five successive integers, denoted by§X 5i X 5i¡ 10 i n. We observe which of the following seven patterns each quintuple matches:¡ ¢¡¢¡¢¡£5i¡ 4¨,All different: abcde Full house: aaabbOne pair: aabcd Four of a kind: aaaabTwo pairs: aabbc Five of a kind: aaaaaThree of a kind: aaabcXA χ 2 -test is based on the number of quintuples in each category.To get a simpler version of this test, a good compromise [16] would be to simply countthe number of distinct values in the set of five. So we would have five categories:5 different = all different4 different = one pair3 different = two pairs, or three of a kind2 different = full house, or four of a kind1 different = five of a kindThis breakdown is easier to determine systematically, and the test is nearly as good.25

5. Tests for Studying Random DataGenerate n groups of k successive numbersCount the numbers of k-tuples with r different valuesA χ 2 -test can be made using the following probability:pd§d 1¨r¥kr¡(5.2)d1¨kr¤ §dThe Stirling number 3 k(of second is the number of ways to partition a set ofk elements into exactly r parts. For this test we use 5, sokind)¢the Stirling numbers canr 1 2 3 4 5ber£written in a little table. 515 25 10 1k¥¢1Example: Throwing a dieLets throw a die until there are one hundred values between one and five. If a six occurs,ignore it.r£The sequence looks like this: { 5 2 4 4 5 1 2 4 2 3 3 4 2 3 5 5 4 4 1 4 1 2 1 5 1 3 1 1 52 3 5 4 2 4 3 3 5 2 4 3 4 3 5 2 2 5 5 1 5 3 1 1 4 5 2 1 1 3 1 2 5 5 5 2 3 2 4 3 4 3 1 3 5 54 2 4 4 2 4 1 3 5 5 2 2 5 2 4 5 3 4 5 3 5 2 5 4 5 }Arrange the sequence into 20 n¥groups of 5 k¥numbers:5 2 4 4 5 | 1 2 4 2 3 | 3 4 2 3 5 | 5 4 4 1 4 | 1 2 1 5 1 | 3 1 1 5 2 | 3 5 4 2 4 |3 3 5 2 4 | 3 4 3 5 2 | 2 5 5 1 5 | 3 1 1 4 5 | 2 1 1 3 1 | 2 5 5 5 2 | 3 2 4 3 4 |3 1 3 5 5 | 4 2 4 4 2 | 4 1 3 5 5 | 2 2 5 2 4 | 5 3 4 5 3 | 5 2 5 4 5Count the number of different valuessequence r sequence r sequence r sequence r5 2 4 4 5 3 1 2 4 2 3 4 3 4 2 3 5 4 5 4 4 1 4 31 2 1 5 1 3 3 1 1 5 2 4 3 5 4 2 4 4 3 3 5 2 4 43 4 3 5 2 4 2 5 5 1 5 3 3 1 1 4 5 4 2 1 1 3 1 32 5 5 5 2 2 3 2 4 3 4 3 3 1 3 5 5 3 4 2 4 4 2 24 1 3 5 5 4 2 2 5 2 4 3 5 3 4 5 3 3 5 2 5 4 5 3We get the following “statistic”r 1 2 3 4 5# r 0 2 10 8 03 The Stirling number can be written in a closed as¤ form m 1n¥m! ∑ m 1mkmkknk¦0§©¨26

5.5. Coupon-collectors testTo calculate the expected values we use equation (5.2) with d¥and 5. k¥p 1 ¥5 55 51625¥ 0¡0016 ¥ 1¡5p 2p 3p 1p 555 5 1¨ 5§5 ¥552¨5 1¨§5 5§5 ¥12125¥ 0¡096 ¥ 2¡552¨§5 3¨5 1¨§5 5§5 ¥1225¥ 0¡48 ¥ 3¡552¨§5 3¨§5 4¨5 1¨§5 5§5 ¥48125¥ 0¡384 ¥ 4¡24625¥ 0¡0384 ¥ 5¡It is now possible to make a table with the expected number of special quintuples andthe measured number.20¥r # expected # measured1 0¡0016 0¡032 02 0¡096 1¡92 23 0¡48 9¡6 104 0¡384 7¡68 85 0¡0384 0¡768 0Constructor in poker_test.hpoker_test(uint64_t n, std::size_t different_cards)n number of poker gamesdifferent_cards number of different poker cards in the game5.5. Coupon-collectors testThis test is similar to the poker test 5.4. We observe the sequence X 1 X 2 ¢¡¢¡¢¡and countlength r of the subsequence X i¡ 1 X i¡ 2 ¢¡¢¡¢¡£theX i¡ r required to get a “complete set” of integersfrom 0 to 1. Obviously, the minimal length of r is d, the maximum length is not boundedso we define a t which gives an upper bound. So it follows d r t. This test is describedin [16].¡ dWe run this test until we get n “complete sets” of integers from 0 to 1 and storethe quantity of each length r in a list like COUNT[r] where d r t. All sequenceslonger then t are accumulated in COUNT[t].¡ d27

5. Tests for Studying Random DataTo perform a χ 2 -test, with 1 degrees of freedom, we have to know theexpected probabilities for each length. This are calculated by the following formulas:d¤ t k¥p r ¥d!dr r1d r ¡ t (5.3)p t ¥d!d t £ 11d 1¡for1d ¡ t(5.4)Once more the termrd¡ denotes the Stirling number of second kind.Example: Throwing a dieWe use the same data as in 5.4. We have integers from 1 to 5 and we define 13We first split up the sequence into “complete sets” and count the lengtht¥5¡¢¡¢¡complete setlength5 2 4 4 5 1 2 4 2 3 103 4 2 3 5 5 4 4 1 94 1 2 1 5 1 3 71 1 5 2 3 5 4 72 4 3 3 5 2 4 3 4 3 5 2 2 5 5 1 165 3 1 1 4 5 2 71 1 3 1 2 5 5 5 2 3 2 4 123 4 3 1 3 5 5 4 2 94 4 2 4 1 3 5 75 2 2 5 2 4 5 3 4 5 3 5 2 5 4 16We can calculate the expected length of sequences with equation (5.3), (5.4) and compareto the measured length.r p r # expected # counted245625=0.0384 0.38 0486625=0.0768 0.77 031273125=0.0998 1.00 433683125=0.1075 1.08 0408249390625=0.1045 1.05 2=0.0955 0.95 1=0.0838 0.84 0=0.0716 0.72 1=0.3220 3.22 210372963906251116370419531251227984390625>136289011953125Constructor in coupon_collector_test.hcoupon_collector_test(uint64_t n,std::size_t different_coupons,std::size_t maxSeq)28

5.6. Permutation testn number of coupon setsdifferent_coupons number of different couponsmaxSeq sequence length above this value will be cumulated5.6. Permutation testThe sequence of numbers is divided into n groups of t elements each, denoted as the vector§X it X ¢ t£ it¡ 1 it¡ for 0 i n. The elements in each group of t values can have t!possible orderings. The number of times each ordering appears is counted and a χ 2 -test witht! degrees of freedom and with probability 1 t! for each ordering. The theory and analgorithm may be found in [16].¢¡¢¡¢¡£ ¡ 1£¨k¥XDivide the input sequence into n groups of t elements eachCount the occurrence of each possible ordering in the groupDo a χ 2 -test with t! k¥degrees of freedom and with probability 1 t! for each orderingExample:We get 50 n¥groups of data with each three values out of 1 2 3.1-10 3 1 2 1 2 3 3 1 2 1 3 2 2 1 3 2 3 1 2 1 3 2 1 3 3 1 2 1 3 211-20 2 1 3 3 1 2 3 1 2 3 2 1 2 3 1 3 2 1 1 2 3 1 2 3 1 2 3 3 2 121-30 1 3 2 3 1 2 3 1 2 1 3 2 2 3 1 1 3 2 2 1 3 2 3 1 2 1 3 3 1 231-40 2 3 1 2 3 1 1 2 3 3 1 2 2 3 1 1 2 3 2 3 1 3 2 1 3 2 1 1 2 341-50 3 2 1 2 3 1 1 2 3 1 2 3 1 3 2 2 1 3 1 2 3 3 2 1 3 2 1 1 3 2For 3 there are d!¥ 6 different combinations. We count the occurrence of eachcombinationd¥sequence 1 2 3 1 3 2 2 1 3 2 3 1 3 1 2 3 2 1# sequences 10 7 7 9 9 8The expected value for every combination is n 508 6¥ d!¥1 3Constructor in permutation_test.hpermutation_test(uint64_t n, std::size_t nrOfElements)n number of permutations to generatenrOfElements number of elements to permute29

n £ 1¨5. Tests for Studying Random Data5.7. Maximum of t testThe sequence of numbers is divided into n groups of t elements each, denoted as the vector§X it X it¡ ¢ t£ 1 it¡ for 0 i n. Then determine the maximum of each group. Thedistribution of the maxima should follow x t . This test is described in [16].¡ 1£¨ ¢¡¢¡¢¡£Divide the input sequence into n groups of t elements each, denoted by Vfor 0 i n.¡ i¥§X it X it¡ 1 X it¡ ¢ t£ 1£¨ ¢¡¢¡¢¡£XGenerate a new sequence max§V max§V 0¨max§V ¢¡¢¡¢¡ 1¨We apply the Kolmogorov-Smirnov test to the sequence of maxima with the distributionfunction t ,§0 x 1¨.F§x¨xWe make k equidistant bins between ¢ 0;1£ . To get the expected number of values ineach bin we have to subtract the probability for the lower bin from the probability forthe actual bin.¥We see that the percentage in bin 1¡¡k k¡i£ i 1is ¡tk . To get the expected valueswe multiply the value per bin with the number of groups n.t i¥Example:We have 50 random floating point numbers in 0;1¨. This numbers are alreadygrouped in sequences of elements, the maxima per group are printed bold.t¥ ¢ n¥5seq.random numbers1 0.911647 0.79844 0.783099 0.394383 0.8401882 0.55397 0.277775 0.76823 0.335223 0.1975513 0.95223 0.513401 0.364784 0.628871 0.4773974 0.606969 0.141603 0.717297 0.635712 0.9161955 0.156679 0.804177 0.137232 0.242887 0.01630066 0.218257 0.998925 0.108809 0.12979 0.4009447 0.637552 0.296032 0.61264 0.839112 0.5129328 0.771358 0.292517 0.972775 0.493583 0.5242879 0.283315 0.891529 0.400229 0.769914 0.52674510 0.949327 0.0697553 0.919026 0.807725 0.352458If we make a binning of the interval ¢ 0;1¨ into 10 subintervals we should know, howmany values we expect for each bin.30

£ bin range percentage # expected # measured1 0.0 - 0.1 1¡00 1060.00 02 0.1 - 0.2 6¡30 1050.00 03 0.2 - 0.3 6¡65 1040.02 04 0.3 - 0.4 3¡37 1030.08 05 0.4 - 0.5 1¡15 1020.21 06 0.5 - 0.6 3¡10 1020.47 07 0.6 - 0.7 7¡10 1020.90 08 0.7 - 0.8 1¡44 1011.60 19 0.8 - 0.9 2¡69 1012.63 310 0.9 - 1.0 4¡69 1014.10 65.8. Birthday Spacings testConstructor in max_of_t_test.hmax_of_t_test(uint64_t n, std::size_t t, std::size_t bins)n number of groups to check for maximumt number of elements per groupbins number of classes for statistic5.8. Birthday Spacings testIn this test we check how random “birthdays” are distributed over a “year”. To do this, wehave a look at the spacings between two successive birthdays. This test was first implementedin Marsaglias Diehard test suite [18]. The theoretical background was presented in [17]. Astronger version is described in [20] and its implementation can be found at http://www.jstatsoft.org/v07/i03/. In the latest version of [16] this test is also included.Choose a number of m “birthdays” in a “year” of n daysSort the birthdays in ascending order and calculate the space between two successivebirthdaysCount the number of collisionsThe expected number of collisions should approximately be Poisson distributed withmean 3 4n. This distribution is tested with a χ 2 test.µ¥mExample:Lets assume a “year” with n¥“days” and m¥“birthdays”Sort the birthdays, calculate and sort the spacings.3136515

5. Tests for Studying Random Databirthday birthdays sorted spacings spacings sorted305 71143 101 30 5285 122 21 6290 132 10 10331 143 11 1171 173 30 13122 186 13 15279 201 15 15101 228 27 15201 279 51 21173 285 6 26228 290 5 27132 305 15 30186 331 26 30346 346 15 51The collisions of “birthday spacings” are printed bold. We got three collisions.The mean of the Poisson distribution is µ¥3 15 34 365¥ 2¡3124n¥Constructor in birthday_spacing_test.hmbirthday_spacing_test(std::size_t runs,std::size_t birthdays,uint64_t days,std::size_t maxCollisions)runs number of birthday experiments to runbirthdays number of birthdays in a yeardays number of days in a yearmaxCollisions collision counts above this number are cumulated5.9. Collision test (Hash test)The χ 2 test statistic is meaningful only when each interval has more than, lets say, 5 samples.But this test is designed such that the number of intervals is much larger than the number ofsamples.Suppose we throw n balls randomly into m empty urns with m n. If a ball falls intoa nonempty urn we get a collision. This is the 1-dimensional collision test. To get the¡2-dimensional version, we have to sort the urns on a 2-dimensional array.Sometime this test is also called “Hash test”. The test can be interpreted as building anenormous hash table and generating an appropriate index. Some theory can be found in [16]or in [4]Select the number of “balls” n and the number of “urns” m. To do this make a listBALLS[n] and insert for ball i the urn in which it falls.32

N £ 2X N X 2 £N£1¤1¨§X5.10. Serial correlationCheck the list BALLS[i], 1¡¢¡¢¡nfor collisions (check if there are any numberstwice in BALLS[i])i¥Theoretically, the probability that an urn receives exactly k balls isnso the expected number of collisions isk ¥ p§k¨1mk11mn£ k∑k 0§k C¥If m ¡ n, then Cn 22m .1¨p§k¨Example: 1-dimensional testWe take “urns” and n¥ 8¥ m¥32 5¥balls256nThe expected number of collisions is C 232 22 256¥ 22m¥The number of “urns” in which the ball flies are listed bellow 214, 100, 199, 203, 232,50, 85, 195, 70, 141, 121, 160, 93, 130, 242, 233, 162, 182, 36, 154, 4, 61, 34, 205,39, 102, 33, 27, 254, 55, 130, 21322We can see that one collision occurs, in urn 130 will be two balls.Constructor in collision_test.hcollision_test(std::size_t runs,uint64_t balls,std::size_t edge_length,std::size_t dim)runs number of experiments to runballs number of balls to throw in urnsedge_length edge length of the “urns field”dim dimension of the “urns field”5.10. Serial correlationMost random numbers are generated by algorithms and not produced by physical processes.Because of this we must assume that there are dependences between two successive numbers.A way to represent this fact is the “serial correlation coefficient”.The “serial correlation coefficient” C from a sequence X 0 X 1 N £ 1 of N random numbersis calculated by the following formula given in [16]:¢¡¢¡¢¡£0 X X 1¤1 Xn§X 2 0¤ 2¤N 1X§X£0¤ 0¨XC¥n§X¤X0¤X2 1¤N £ 1¨21¤¤XN £ 1¨2X¤X1¤X¤X(5.5)33

0¡1111 0¡672;0¡449£5. Tests for Studying Random DataA correlation coefficient between always and¤lies 1 1. When it is zero or very small, itindicates that X i and X j are independent of each other. A “good” value of C will be betweenµ N and µ N which means 95%, whereN¤ Nµ N¥11σ1 N N¥ N 2 (5.6)2σGenerate a sequence of N random numbers X 0 X 1 ¢¡¢¡¢¡N £ 12σ1 13¨ N¤Calculate the “serial correlation coefficient” C with formula (5.5)N§N NCalculate the mean and the standard deviation and check if C lies between the two σlimit (µ N¡ 2σ N ) which denotes the 95% limitXExample:Lets take the same sequence of ten random number as in section 3.2 {0.809, 0.465, 0.151,0.628, 0.318, 0.824, 0.394, 0.968, 0.179, 0.458}.We have to calculate C as shown in equation (5.5)10 0¡465¤§0¡809 0¡465 0¡151¡¢¡¢¡¨20¡465¤10 §0¡809 0¡465 2¤ 2¤ 2¡¢¡¢¡¨ §0¡809¤0¡465¤ 0¡5153760¡151¡¢¡¢¡¨2 §0¡809¤To check if the calculated C coefficient lies between the 2σ bounds. Use the formulasgiven in (5.6) 0¡151¡¢¡£¡¨ C¥¥11 70µ 10¥ σ 10¥9 11¥ 0¡28039We see that the valid interval for this test is . The calculated coefficientC lies between these bounds. (The interval is so large because we only tested tennumbers)¢ ¥Constructor in serial_correlation_test.hserial_correlation_test(uint64_t n)n number of random numbers to calculate correlation for5.11. Serial testThis test checks if not only particular numbers are uniformly distributed but also two, three ord-dimensional points. To make this test, count the number of times the tuple§X di X di¡ 1 di¡ ¢ d£ 1£¨ ¢¡¢¡¢¡£Xthe test is the same as the “Equidis-occurs, for 0 i n and the dimension d 0. If 1tribution test”. The tuples should be χ 2 distributed.d¥ ¡Generate n d-tuples§X di X di¡ 1 X ¢¡¢¡¢¡di¡ ¢ d£ 1£¨, where d0 and 0 X j¡k34

§25.12. Blocking testApply a χ 2 test to these k d categories with probability 1 k d in each category.To get a valid χ 2 test n should be large compared to k d ,say nk dA more detailed description is given in [16].Example:Lets generateare:§2 §2 2¨ 1¨ n¥pairs of random numbers (d¥ between zero and two 3), the pairs§1 §0 §1 §1 §2 §1 §0 1¨ 2¨ 2¨ 1¨1¨ 1¨ 2¨ 0¨Then we count the appearing of each tuple10pairs §0count 0 2 0 0 2 2 1 1 22¨ 1¨ 0¨ 2¨ 1¨ 0¨ 2¨ 1¨ 0¨§0§0§1§1§1§2§2§2The expected number of pairs for each class is 1 d 3 n¥9¥1¡11Constructor in serial_test.hserial_test(uint64_t n, std::size_t gridSize, std::size_t dimension)10n number of random numbers to placegridSize edge length of the griddimension dimension of the grid5.12. Blocking testFor this test I found only a really scanty description in [26] . So, the only way to find outhow this test has to be implemented is looking into existing source code.The Blocking test tests a proposition of the central limit theorem. This says that, the thesum of k independent variables with zero mean and unit variance approaches the normaldistribution with mean zero and variance equal to k. To test the proposition n sums of suchgroups or blocks will be built and checked for normality.5.13. Repeating Time TestThis test checks if a uniform§0 random number generator starts to repeat its sequencewhen it is expected to. If the repetition occurs to soon, the test fails because the generatordoes not generate all possible number but only a subset of all values. If the first repetitionoccurs to late after the expected value, this means that the numbers are unusually uniformlyspread. The implementation and included description can be found in [10]1¨35

We get gcd§216 256¨ ¥ 1,815225. Tests for Studying Random Data5.14. gcd test (greatest common divisor)This test calculates the greatest common divisor of two random numbers using Euclid’salgorithm. Now the number of steps to complete Euclid’s algorithm and the resulting gcdwhere checked against their expected probability.The idea of this test is described in [20] and some theory may be found in [16] in section4.5.2, in the exercises and in the accordingly answers.The problem is that the expected distribution of the number of steps and the gcd is unknown,so the comparison must be done with simulated values.Example:We calculate the gcd of 216 u¥and 256 v¥The algorithms gives:25621640¥16¥¥ ¥4040¤16216¤8816¤and the number of steps 4 k¥5.15. Gorilla testThis is a strong version of the monkey test from the Diehard test suite [17] . The test countsthe number of missing 26-bit “words” and compares it with the expected value. UnfortunatelyMarsaglia’s version is hard-wired so a more flexible implementation has to developthe associated statistic. The theory of the test and its implementation can be found in [20]5.16. Ising-model testThe Ising model [15] is one of the simplest and most fundamental models of statistical mechanics.It describes the properties resulting from interacting spins on a lattice.The system considered is an array of N fixed points called lattice sites that form an n-dimensional periodic lattice. Associated with each lattice site is a spin variable s1 which is a either¤number or that is 1 1. If s 1, the ith site is said to havespin up, and if s it is said to have spin down. A given set of numbers s i specifies aconfiguration of the whole system. The energy of the system in the configuration specifiedby si§i¥i is defined by the Hamiltonian¤ i¥ N¨ ¢¡¢¡¢¡ i¥∑ ¢ i j£s i s jJwhere J is the coupling energy. The sum is over pairs of nearest-neighbour sites on thelattice. H¥36

– Generate n random number in X¢0;1¨ i¥5.17. Random-walk testTo perform a Monte Carlo simulation of the Ising model, we use the Wolff cluster-flippingalgorithm. This algorithm generates large clusters on a lattice by connecting bonds fromstarting point to nearest neighbours with the same spin with the following probability:2Jk B Tewhere J is, like above, the coupling energy and T the temperature. The model is simulatedat the critical temperature T c which can be calculated via1 p¥2T c¥log§1¤¢ 2¨ ¥ 2¡26918531421302But simulations performed with the Wolff algorithm [33] are very sensitive to the propertiesof the used random number generator. This effect is published in [8]. Ferrenberg, Landau andWong denoted aggravating discrepancies between the expected and the simulated energiesfor some random number generators.A standard model size in literature is a 1616 square lattice. For this size, at the criticaltemperature, we know the Ising models exact solution for the energy average E1¡45306. The result we are interested in, is the deviation in σ’s (standard deviation) of thesimulation result from the exact result. To calculate the exact energies we used the exact partitionfunctions computed by Häggkvist and Lundow [13]. A similar implementation for a¡16 16 lattice is given in [28] and may be found at http://www.physics.helsinki.¥fi/~vattulai/codes/acorrtiw.fConstructor in ising_model_test.hising_model_test(uint64_t n, std::size_t lattice_size = 16)n number of Wolff steps in simulationlattice_size edge length of lattice5.17. Random-walk testIn the random walk test [28] we consider random walks, something like brownian motions,on a two dimensional lattice. This is divided into four equal blocks, each of which has anequal probability to contain the random walker after a walk of length n (or n steps). The testis performed N times, and the number of occurrences in each of the four blocks is comparedwith the expected value of N 4, using the χ 2 test with three degrees of freedom. Vattulainensimplementation [28] in Fortran can be found at http://www.physics.helsinki.fi/~vattulai/codes/2drwtest.fRepeat the following procedure N times– Set the x-, y-coordinates zero37

5.18. n-block test5.18. n-block testThis test checks the average of subsequences, so called blocks. This is done by calculatingthe average of many sequences of uniformly distributed random numbers (0 x i¡1) andincreasing a counter if the average of the sequence ¯x 1 2. This test is described in [28] anda implementation in Fortran can be found at http://www.physics.helsinki.fi/~vattulai/codes/nblocktest.fGenerate a sequence of n random numbers x 1 x 2 x ¢¡¢¡¢¡n where 0 x i¡1Calculate the average ¯x over the sequence, if ¯x 1 2 increase y 1Repeat the last two steps N timesCalculate the measured probability for ¯x 1 2 as y ¡with one degree of freedom on the y i2¥1 and perform a χ 2 testyVattulainens criterion for failing: Each test is repeated 3 times, and the generator failsfor fixed n if at least two out of three χ 2 failed, which should occur with a probabilityof about 3 400NExample:Lets generate 10 N¥sequences of n¥numbers8x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 ¯x0.84 0.394 0.783 0.798 0.912 0.198 0.335 0.768 0.6290.278 0.554 0.477 0.629 0.365 0.513 0.952 0.916 0.5860.636 0.717 0.142 0.607 0.0163 0.243 0.137 0.804 0.4130.157 0.401 0.13 0.109 0.999 0.218 0.513 0.839 0.4210.613 0.296 0.638 0.524 0.494 0.973 0.293 0.771 0.5750.527 0.77 0.4 0.892 0.283 0.352 0.808 0.919 0.6190.0698 0.949 0.526 0.0861 0.192 0.663 0.89 0.349 0.4660.0642 0.02 0.458 0.0631 0.238 0.971 0.902 0.851 0.4460.267 0.54 0.375 0.76 0.513 0.668 0.532 0.0393 0.4620.438 0.932 0.931 0.721 0.284 0.739 0.64 0.354 0.63Now we check the averages ¯x and get y 5 1¥and y y N 2¥5 1¥Constructor in n_block_test.hn_block_test(std::size_t n, std::size_t block_size)n number of blocksblock_size size of each block39

5. Tests for Studying Random Data5.19. Random Walker on a line (S n test)This test uses different random walkers in one dimension. This N random walker movesimultaneously without any interaction. At each step in a walk, they can jump left or rightwith the same probability. After t 1 steps for each walker, the number of visited sitesS ¡n t has an asymptotic form S n t f§N¨tγ where the scaling function §ln 2 and N¨1¡1 2 is the expected exponent as based on theory. The value of the exponent γ observedfrom simulations serves as a measure of correlations.A description is available in [27] and the appropriate implementation can be found at http://www.physics.helsinki.fi/~vattulai/codes/sn1d_test.fγ¥ ¥ f§N¨5.20. 2D Intersection testIn this test we use two random walkers in two dimensions. Their paths are given by twodifferent sequences of random numbers. After n steps of each random walker we calculatethe probability that they never meet the same place in plane (at the same time or not) exceptat their common starting point. For a random process it is known that the number of intersectionsI§n¨behave asymptotically like a power law α with an exponent 8.A description is available in [27] and the appropriate implementation can be found at http://www.physics.helsinki.fi/~vattulai/codes/intersections.fα¥ £ I§n¨5.21. 2D Height Correlation testThe Height Correlation test observes again the behavior of one-dimensional random walker.Here the correlation between the heights of two walkers are measured, where each randomwalker represents a stream of random numbers. To do this we construct two sequences ofrandom steps on a line (x 1 i x 2 i ), then the height is defined as h x 1 xt 2 .£¢The correspondingcorrelation function H h 0 t φ is known to decay asymptotically as a power lawwith a exponent 1 2.t¥A description is available in [27] and the appropriate implementation can be found at http://www.physics.helsinki.fi/~vattulai/codes/height.ft t¥ tφ¥n5Constructor in height_corr2d_testheight_corr2d_test(std::size_t n, std::size_t steps)¡hn number of samples with each two walkssteps number of steps per walk5.22. Sum of independent distributions testThis test is used in the SPRNG test suite [26]. It is designed to check multiple streams forindependence. This test builds n sums of groupsize random numbers from each stream andtests the distribution with a K-S statistic.40

5.23. Fourier transform test5.23. Fourier transform testFor this test I found only a really scanty description [26]. So, the only way to find out howthis test has to be implemented is looking into existing source code.For a short description the following can be said. It is a test for multiple streams, butmultiple streams can be built of multiple subsequences. A two-dimensional array has to befilled with random numbers, each row with n numbers from a different stream. Then thetwo-dimensional Fourier coefficients were calculated and compared with the expected ones.A related article can be found in [7].5.24. Universal statistical testThis test was designed to detect any significant deviation of a devices output statistics fromthe statistic of a truly random bit source. This test is done by measuring a parameter closelyrelated to the devices per-bit entropy. The fully description is in [23].5.25. The Diehard Test SuiteDiehard is the name of a battery of tests for random number sequences which was developedby George Marsaglia in 1995 [18]. The original Code was written in FORTRAN, but thereare two new implementations in C [19], [22]. A useful paper may also be [24].The tests contained in the Diehard battery are listed bellow.Birthday Spacings testOverlapping PermutationsRanks of 31 31 and 32 32 matrices testRanks of 68 matrices testCount the 1‘s in a Stream of BytesCount the 1‘s in Specific BytesMonkey tests on 20-bit WordsMonkey tests OPSO,OQSO,DNAParking Lot testOverlapping Sums testSqeeze testMinimum Distance testRandom Spheres test41

5. Tests for Studying Random DataRuns testCraps testThe following sections describes tests from the Diehard battery. In most cases there is theoriginal (converted to LATEX style) test description quoted.5.25.1. Birthday Spacings testThis test is described in 5.8. The parameters used in Diehard areruns 500birthdays 2 9 512days 2 24 16777216maxCollisions = “not used” 45.25.2. The overlapping 5-permutation testThe following description is the original text from the Diehard test suite.This is the OPERM5 test. It looks at a sequence of one million 32-bit randomintegers. Each set of five consecutive integers can be in one of 120 states, forthe 5! possible orderings of five numbers. Thus the 5th, 6th, 7th,¡¢¡¢¡numberseach provide a state. As many thousands of state transitions are observed, cumulativecounts are made of the number of occurrences of each state. Then thequadratic form in the weak inverse of the 120 120 covariance matrix yieldsa test equivalent to the likelihood ratio test that the 120 cell counts came fromthe specified (asymptotically) normal distribution with the specified 120 120covariance matrix (with rank 99). This version uses 1 000 000 integers, twice.5.25.3. Ranks of binary matricesThe Diehard test suite implements three binary matrix tests for different matrix dimensions.The aim of all these test are the same, namely to check the rank of the constructed randommatrix against the expected rank.This implemented binary rank test is more flexible, it is possible to specify the dimensionof the matrix to construct from random numbers. In the test each random number is splitinto bits 0¡¢¡¢¡n, and so on, until n reaches the bit length of the random number.Matrices are constructed from each of these sequences and over each sequence we performa χ 2 test. At the end we make a K-S test over all χ 2 values. The probabilities for rank k in am n matrix is given in [32].In the Diehard test suite the result of the 6 8 and the 31 31 or 32 32 matrices test1¡¢¡¢¡§n¤are analysed in a different way. For the 6 8 matrix the χ 2 probability of all sub-matrices1¨is calculated and then a Kolmogorov-Smirnov test is performed over the values. For the4 In section 5.8 we wrote the expected value as birthdays days. The maxCollisions valueshould be much bigger than this value. In this example we get so 3¡§4¢ we can choose 16.µ¥ µ¥422,

5.25. The Diehard Test Suitebigger matrices, only the χ 2 value for matrix is reported. To accommodate to this differentmeanings, there are two different tests implemented, a bin_rank_ks_test and abin_rank_chisqr_test. The usage of both classes is exactly the same, only the statisticsare different.Constructors in bin_rank_test.hbin_rank_ks_test(uint64_t n,std::size_t rows,std::size_t columns,std::size_t minRankCount)bin_rank_chisqr_test(uint64_t n,std::size_t rows,std::size_t columns,std::size_t minRankCount)n number of matrices to buildrows number of rows in matrixcolumns number of columns in matrixminRankCount count ranks down to this rank, if a rank is smaller cumulate itIn the original implementation the following parameters were used:Ranks of 31 31 matrices testbin_rank_chisqr_testn 40000rows 31columns 31minRankCount 28Ranks of 32 32 matrices testbin_rank_chisqr_testn 40000rows 32columns 32minRankCount 29Ranks of 6 8 matrices testbin_rank_ks_testn 100000rows 6columns 8minRankCount 45.25.4. The bitstream testThe following description is the original text from the Diehard test suite.43

5. Tests for Studying Random DataThe file under test is viewed as a stream of bits. Call them b 1 b 2 ¢¡¢¡¢¡. Consideran alphabet with two “letters”, 0 and 1 and think of the stream of bits as asuccession of 20-letter "words", overlapping. Thus the first word is b 1 b 2¡¢¡¢¡b ,the second is b 2 b 3¡¢¡¢¡b 2120, and so on. The bitstream test counts the number ofmissing 20-letter (20-bit) words in a string of 2 21 overlapping 20-letter words.There are 2 20 possible 20 letter words. For a truly random string of 2 19 bits,the number of missing words j should be (very close to) normally distributed21¤with mean 141 909 and 428. Thus j £ 141909428should be a standard normalvariate (z score) that leads to a uniform 0 p value. The test is repeatedtwenty times.1¨ ¢ σ¥5.25.5. The OPSO, OQSO and DNA testsThe text of the following sections is from the original description of the Diehard test suite.OPSO means Overlapping-Pairs-Sparse-OccupancyThe OPSO test considers 2-letter words from an alphabet of 1024 letters.Each letter is determined by a specified ten bits from a 32-bit integer in the sequenceto be tested. OPSO generates 2 21 (overlapping) 2-letter words (from2 1 "keystrokes") and counts the number of missing words—that is 2-letterwords which do not appear in the entire sequence. That count should be veryclose to normally distributed with mean 141 909, 290. Thus missingwords £ 141909290should be a standard normal variable. The OPSO test takes 32 bits at a timefrom the test file and uses a designated set of ten consecutive bits. It then restartsthe file for the next designated 10 bits, and so on.σ¥ 21¤OQSO means Overlapping-Quadruples-Sparse-OccupancyThe test OQSO is similar, except that it considers 4-letter words from an alphabetof 32 letters, each letter determined by a designated string of 5 consecutivebits from the test file, elements of which are assumed 32-bit randomintegers. The mean number of missing words in a sequence of 2 21 four-letterwords, (2 3 “keystrokes"), is again 141909, with 295. The mean isbased on theory; σ comes from extensive simulation.σ¥ 21¤The DNA testThe DNA test considers an alphabet of 4 letters C,G,A,T, determined by twodesignated bits in the sequence of random integers being tested. It considers10-letter words, so that as in OPSO and OQSO, there are 2 20 possible words,and the mean number of missing words from a string of 2 21 (overlapping) 10-letter words (2 9 “keystrokes") is 141909. The standard deviation 339was determined as for OQSO by simulation. (Sigma for OPSO, 290, is the truevalue (to three places), not determined by simulation.σ¥ 21¤44

5.25. The Diehard Test Suite5.25.6. The count-the-1’s testThe text of the following sections is from the original description of the Diehard test suite.A stream of bytesThis is the “count-the-1’s” test on a stream of bytes. Consider the file undertest as a stream of bytes (four per 32 bit integer). Each byte can containfrom 0 to 8 1’s, with probabilities 1,8,28,56,70,56,28,8,1 over 256. Now let thestream of bytes provide a string of overlapping 5-letter words, each “letter”taking values A,B,C,D,E. The letters are determined by the number of 1’s in abyte 0,1, or 2 yield A, 3 yields B, 4 yields C, 5 yields D and 6,7 or 8 yield E.Thus we have a monkey at a typewriter hitting five keys with various probabilities(37,56,70,56,37 over 256). There are 5 5 possible 5-letter words, and from astring of 256,000 (overlapping) 5-letter words, counts are made on the frequenciesfor each word. The quadratic form in the weak inverse of the covariancematrix of the cell counts provides a χ 2 test Q5-Q4, the difference of the naivePearson sums ¢ OBS£ EXP£ 2ofEXPon counts for 5- and 4-letter cell counts.Specific bytesThis is the “count-the-1’s” test for specific bytes. Consider the file undertest as a stream of 32-bit integers. From each integer, a specific byte is chosen,say the left-most bits 1 to 8. Each byte can contain from 0 to 8 1’s, withprobabilities 1,8,28,56,70,56,28,8,1 over 256. Now let the specified bytes fromsuccessive integers provide a string of (overlapping) 5-letter words, each “letter”taking values A,B,C,D,E. The letters are determined by the number of 1’s,in that byte 0 1 or 2 A, 3 B, 4 C, 5 D, and 6 7 or 8 E. Thuswe have a monkey at a typewriter hitting five keys with various probabilities37,56,70,56,37 over 256. There are 5 5 possible 5-letter words, and from a stringof 256 000 (overlapping) 5-letter words, counts are made on the frequencies foreach word. The quadratic form in the weak inverse of the covariance matrixof the cell counts provides a χ 2 test Q5-Q4, the difference of the naive Pearsonsums ¢ OBS£ EXP£ 2ofEXPon counts for 5- and 4-letter cell counts.5.25.7. The parking lot testThe following description is the original text from the Diehard test suite.In a square of side 100, randomly “park” a car-a circle of radius 1. Thentry to park a 2nd, a 3rd, and so on, each time parking “by ear". That is, if anattempt to park a car causes a crash with one already parked, try again at a newrandom location. (To avoid path problems, consider parking helicopters ratherthan cars.) Each attempt leads to either a crash or a success, the latter followedby an increment to the list of cars already parked. If we plot n: the number of45

5. Tests for Studying Random Dataattempts, versus k: the number successfully parked, we get a curve that shouldbe similar to those provided by a perfect random number generator. Theoryfor the behavior of such a random curve seems beyond reach, and as graphicsdisplays are not available for this battery of tests, a simple characterization ofthe random experiment is used: k, the number of cars successfully parked after000 attempts. Simulation shows that k should average 3523 with σ¥1221¡9 and is very close to normally distributed. Thus k £ 352321 9should be a standardnormal variable, which, converted to a uniform variable, provides input to a KStestbased on a sample of 10.n¥5.25.8. The overlapping sums testThe following description is the original text from the Diehard test suite.Integers are floated to get a sequence ¢¡¢¡¢¡. of uniform 0 variables.Then overlapping sums,¢¡¢¡¢¡are formed. The S’s are virtually normal with a certain covarianceU§100¨ S2¥matrix. A linear transformation of the S’s converts them to a sequenceU§2¨ ¢ 1¨U§101¨of independentstandard normals, which are converted to uniform variables for aKS-test. The p-values from ten¡¢¡¢¡¤ U§2¨¤KS-tests are¡¢¡¢¡¤ U§1¨¤ ¥ S§1¨given still another KS-test.5.25.9. Squeeze testThe following description is the original description from Diehard and can be found at [18],[19].Random integers are floated to get uniforms on 0 1¨.Starting with 2 k¥ ¢2147483647, the test finds j, the number of iterations necessary to reduce kto 1, using the reduction ceiling§kU¨, with U provided by floating integersfrom the file being tested. Such j’s are found 100 000 times, then counts for thenumber of times j was 6 7 48 are used to provide a χ 2 test for cellfrequencies.k¥¢¡¢¡¢¡31¥47Constructor in squeeze_test.hSqueezeTest(uint64_t n, uint64_t squeezeStart, std::size_t maxCount)n number of numbers to squeezesqueezeStart start value of squeezingmaxCount squeeze steps bigger then this number are cumulatedThe implemented version of the squeeze test is a bit more universal. In the original implementationthe parameters aren 100000squeezeStart 2 31 ¡ 1 2147483647maxCount 4846

5.25. The Diehard Test SuiteThe probability, used to perform the χ 2 test, for i squeeze steps is calculated by the followingformula11k§lnk¨i£p§i (5.7)Γ§i¨ ¥ k¨5.25.10. The Minimum Distance testThe following description is the original description from Diehard and can be found at [18],[19].The implemented version is based on [9], there the exact expectation values are given.It does this 100 times: choose 8000 random points in a square of siden10000. Find d, the minimum distance between the2pairs of points. If thepoints are truly independent uniform, then d 2 , the square of the minimum distanceshould be (very close to) exponentially distributed with mean 0¡995. Thusdn¥ 2 should be uniform on n2£ 0 and a KS-test on the resulting100 values serves as a test of uniformity for random points in the square. Test0¡995¨0 mod 5 are printed but the KS-test is based on the full set of 100random choices of 8000 points in the 10000 10000 square.1¨ ¢numbers¥Constructor in minimum_distance_test.h1 exp§minimum_distance_test(std::size_t runs, std::size_t n)runs number of experimentsn number of points to place in squareThe implemented version of the minimum distance test is a bit more universal. In the originalimplementation the parameters areruns 8000n 1005.25.11. Random Sphere testIn this implementation of the “Random Sphere” test the number of spheres to place in spaceis not fixed, it may be changed. The following description is quoted from the Diehard testsuite. To calculate the probabilities a report [9] is really helpful.Choose 4000 random points in a cube of edge 1000. At each point, centera sphere large enough to reach the next closest point. Then the volume of thesmallest such sphere is (very close to) exponentially distributed with mean 120π3 .Thus the radius cubed is exponential with mean 30. (The mean is obtained byextensive simulation). The “3D-spheres” test generates 4000 such spheres 20times. Each min radius cubed leads to a uniform variable by means of 1exp§ r 3 30¨, then a KS-test is done on the 20 p-values.47

5. Tests for Studying Random DataConstructor in random_sphere_test.hrandom_sphere_test(std::size_t runs, std::size_t n)runs number of experimentsn number of spheres to place in squareIn the original implementation the parameters shown next are used:runs 20n 40005.25.12. The runs testThe runs test is described in section 5.2. The parameters for the runs test used in the Diehardtest suite aren 10000maxRunLength 65.25.13. Craps testThis is one more test invented with the Diehard test suite. Marsaglia gives the followingdescription:This is the “craps test”. It plays 200 000 games of craps, finds the numberof wins and the number of throws necessary to end each game. The numberof wins should be (very close to) a normal with mean 200000 p and variance244200000 p¨,with495. Throws necessary to complete the game canvary from 1 to ∞, but counts for all 21 are lumped with 21. A χ 2 test is madeon the no.-of-throws cell counts. Each 32-bit integer from the test file providesthe value for the throw of a die, by floating to 0 1¨, multiplying by 6 and taking1 plus the integer part of the result.¢p¥ p§1Constructor in craps_test.hcraps_test(uint64_t n, std::size_t max_throws)n number of Craps games to playmax_throws maximal number of rolling the dice until the number is cumulated48

6. Extending the Random Number Generator Test SuiteThis chapter is, additional to the source code, the key to extend the RNGTS framework. Hereis shown how to implement further tests, by using the given base classes or by specifying therequirements of other random number generators. At the end there is also an overview overthe used XML-schema.6.1. How to implement a testIf one likes to write a new test for random number generators a specific interface needs to beimplemented. This allows the RNGTS framework to interact with the test, e. g. it executesthe test automatically.Unfortunately in C++ it is not possible to define interfaces which act only as specificationsfor the methods of implementation (like in Java). A way to make an interface is to buildabstract classes, but then we have virtual function calls. There are only few methods toimplement which are described below.The following listing shows the base of each test, containing all required methods.#include "buffered_random.h"// definition of "buffered_random_rumber_generator_base"#include "xml_helper.h" //XML output functionsclass the_new_test{public:the_new_test(...);void run(buffered_random_rumber_generator_base& rng);std::string test_name() const;template < class InputIterator >void analyze(xml_helper& out, InputIterator cl_begin, InputIterator cl_end) const;}void print_parameters(xml_helper& out) const;The constructor must be able to take all parameters which are needed to run a completetest, e. g. the number of runs.the_new_test(uint64_t runs, ...)The RNGTS framework calls the run method to execute the test. When run has finishedits work, the statistic must be calculated.void run(buffered_random_rumber_generator_base& rng)49

6. Extending the Random Number Generator Test Suiterng is the actual random number generator to test. It may be converted to a boost::uniform_realgenerator or an other boost typeThis method must return the name of the test.std::string test_name() constThe task of the analyze(...) method is to check the confidence level for the calculatedquantities. It also has to write the results in a XML structure to the output. Theavailable XML tags can be found in the XML Schema definition of the result file or in thelisting. Below a sample implementation is given.template < class InputIterator >void analyze(xml_helper& out, InputIterator cl_begin, InputIterator cl_end) const{// this implementation is given as a example}// helper to convert numeric values to stringsstd::ostringstream val;// tag marks the begin of the result section in the XML output// if it is not a χ 2 or a KS analyze one makes a ’RESULTS’ tag else one can// make a ’CHI_SQUARE’ or ’KOLMOGOROV_SMIRNOV’ tag, or better, one uses the// appropriate base class and this method is already implementedout.startTag("RESULTS");// write all relevant results as a tag to the XML stream// convert ’result’ to a streamval

6.1. How to implement a testThis method must write all required parameters to reproduce the test to the XML structure,below an example is given.void print_parameters(xml_helper& out) const{// this implementation is given as a example// helper to convert numeric values to stringsstd::ostringstream val;}}// converts the ’parameter_’ to a streamval class chisquare_tTest{void prepare_statistics(std::size_t count_size,uint64_t runs,std::size_t degOfFreedom = 0);inline std::size_t get_entry(buffered_random_rumber_generator_base& rng);double get_chisqr_probability(std::size_t i) const;}ks_testtemplate < class DerivedType >class ks_test{void prepare_statistics(uint64_t runs);inline double get_entry(buffered_random_rumber_generator_base& rng);}gaussian_testclass gaussian_test{void prepare_statistics(double deviation, double stat_value, double mean);void calc_gaussian_value();}51

6. Extending the Random Number Generator Test SuiteThe implementation of a χ 2 or Kolmogorov-Smirnov test are very similar. Implementinga Gaussian test is different because we could not support as much functionality as in theother two tests.Here, only a short overview over the most important methods of the test base class isgiven. More detailed and specific information is found in the class description and in thesource itself.6.1.2. χ 2 testTo get an overview of the involved methods and the order of method calls there is a sequencediagram in figure 6.1 which shows the events graphically.The prepare_statistics method has to be called before the underlying test is executedby the run method. This must be done in the constructor of the test class.void prepare_statistics(std::size_t count_size,uint64_t runs,std::size_t degOfFreedom = 0);count_size The number of classes used to make the statisticruns The number of invocations of the get_entry methoddegOfFreedom The degrees of freedom used for the statistical calculations, as default is takencount_size ¡ 1The base class invocates the get_entry method the chosen number of repetitions (runs).This method must return the index of the class which belongs to the calculated/measuredvalue. The appropriate class count will be increased. Keep in mind that this method must notchange the state of the class to one not equivalent to the state after the constructor was called.The RNGTS framework only calls get_entry so it is not possible to reset any variables fortesting a new generator.inline std::size_t get_entry(buffered_random_rumber_generator_base& rng);rng Random number generator to use in the testreturn Returns the index of the class appropriate to the calculated valueThe base class needs the probability for each class to calculate the χ 2 statistic. So, the testclass has to support such a method.double get_chisqr_probability(std::size_t i) const;i Class to get the probability for. 0 ¡£¢¥¤§¦©¨¨¨ ireturn Returns the probability for class i6.1.3. Kolmogorov-Smirnov testTo get an overview of the involved methods and the order of method calls the same figureas in the χ 2 test is usefull 6.1, one only has to replace chisquare by ks. Theprepare_statistics method has to be called before the underlying test is executed52

6.1. How to implement a test: concrete_test_runner: chisquare_testany chisquare test1: create2: createsets possiblestatistic name3: prepare_statistics4: test_name5: run6: get_entrysome time aftercreation...7: get_entry8: get_entryget_entry iscalled 'n' time9: get_entry10: calculate_chisquare_value11: get_chisqr_probability12: print_parameters13: analyzeFigure 6.1.: Sequence diagram for the χ 2 test53

6. Extending the Random Number Generator Test Suiteby the run method. This must be done in the constructor of the test class. The parametersare:void prepare_statistics(uint64_t runs);runs The number of invocations of the get_entry methodThe base class invocates the get_entry method the chosen number of repetitions (runs).This method must return a probability value for the K-S statistic. (The name “probability”already tells that the value must¢be ). Keep in mind not to change the internal state ofthe class, for the same reason as in the χ 2 test class.0¡¡1£inline double get_entry(buffered_random_rumber_generator_base& rng);rng Random number generator to use in the testreturn Returns a probability value for the K-S statistic6.1.4. Gaussian testThe main difference to the two base classes above is the fact that the test itself has to calculatesome statistical values. This values have to be passed to the base class to make some furthercalculation. The passing is done via the prepare_statisticsmethod, which obviouslyhas to be called after the test has run. Additionally one needs to implement the run methodinstead of a get_entry routine. The method calls are little different than in tests before.The exact sequence of calls can be viewed in the sequence diagram in figure 6.2.void prepare_statistics(double deviation, double stat_value, double mean);deviation The calculated/measured deviation in σ’sstat_value The calculated/measured value (the “result”)mean The expected mean valueAfter the statistic has been prepared with the method above, the gaussian value can becalculated. This method calculates the deviation from the mean value as a factor (may alsobe interpreted as percent).void calc_gaussian_value();A discussion about this method is given in section 3.3.6.2. The multiple_test wrapperThere are some cases in which a test has more than one statistic, e. g. the “runs” test. In suchcases it is not possible to derive the test class two times from the base class, we need an otherconcept.To permit the use of different statistical tests for a test, we provide the multiple_testclass as base class. This class takes a tuple of statistical test types and a tuple of as manystd::string types as template parameter.54

6.2. The multiple_test wrapper: concrete_test_runner: gaussian_testany gaussiantest1: create2: createsets possiblestatistic name3: test_namesome timeafter creation...4: run5: prepare_statistics6: calc_gaussian_valuecalled afterrun method7: print_parameters8: analyzeFigure 6.2.: Sequence diagram for the gaussian test55

6. Extending the Random Number Generator Test SuiteThe usage of the class is quite simple, the first thing to do is to derive the test class frommultiple_test base class. In the “run test” example (it contains two χ 2 tests) it lookslike:class runs_test : public multiple_test{ ... }or as a interface description:template< class T, class S >class multiple_testT boost::tuple containing the wanted statistical test typesS std::string containing as many strings as test types in T, this are used to store each testsindividual nameThe constructor of the derived class has to call the constructor of the base class resp. theconstructor of multiple_test to set the each statistical tests name. Our run example:runs_test(uint64_t n, std::size_t maxRunLength):multiple_test< boost::tuple,boost::tuple>(boost::make_tuple("Runs-Up", "Runs-Down")), ...The constructor is called with the two statistic names, “Runs-Up” and “Runs-Down”. Thefirst name in the S tuple is assigned to the first test in the T tuple and so on. The interface ofthe constructor is the following:multiple_test(S statistic_names)statistic_names a boost::tuple containing the name of each statisticAll statistical tests are stored in a member variable called multipleTest_ which isaccessible from the derived class. Getting access to each statistic is simple. E. g. a call of themethod boost::tuples::get(multipleTest_).prepare_statistics(...)prepares the first statistic in the tuple, where boost::tuples::get(multipleTest_)grants access to the first element in the test tuple. In general the following syntax can be used:boost::tuples::get(multipleTest_).method();n number of statistic to access, the order of statistics is given by the order used in the derivatingspecificationmethod the name of the method to call from statistic at position nIt must be denoted that the multiple_test base may be used if and only if all statisticsof the associated statistical tests must be written out. This wrapper calls the analyzemethod of each associated statistical test.56

6.3. Useful sequence diagrams6.3. Useful sequence diagramsDuring the implementation of new tests or other extensions to the test suite, it sometime isimportant to know the order of method calls. A graphical representation is given with UMLdiagrams 6.3, 6.4. This diagrams show only some special cases because of the vast varietyof different possible cases.6.4. Demands on Random Number GeneratorsTo use a random number generator with this test suit, it has to fulfil different properties.These are nearly the same as a boost “Pseudo-Random Number Generator” has to fulfil.Jens Maurer wrote a specification for the boost library called “Random Number GeneratorLibrary Concepts” which can be found in the Boost documentation [2] or a summaryin table 6.1. One also has to implement an appropriate traits class to allow using of aseed(value) method. This method is not requested by the standard but often implemented.If the generator supports the “single call” method, the traits class can be implemented,in the rng_traits.h header, as follows// from lagged_fibonacci.hpptemplatestruct has_single_call{BOOST_STATIC_CONSTANT(bool, value = true);};or, if there is no “single call” method, the value must be false.// example from additive_combine.hpptemplatestruct has_single_call{BOOST_STATIC_CONSTANT(bool, value = false);};6.5. Foreign Random Number GeneratorsIt is also possible to test “foreign” random number generators, as such from C or Fortran.To use such generators a simple wrapper class is delivered which encapsulates the call of thenext random number. This class supports all methods required for a pseudo random numbergenerator. The declaration is the following:templateclass rng_wrapperreturn_type type of generated random numbersRNG() function pointer to the random number functionThe constructor of the class has the signature:rng_wrapper(result_type min_value = 0, result_type max_value)57

6. Extending the Random Number Generator Test Suitetest_suite_main : rng_test_suite : buffered_random_: concrete_test_runner1: add_confidence_level2: add to set3: add_seed4: add to vector5: register_rng6: create7: create generator holder8: add to vector9: register_seeded_rng10: register_test11: create12: run_testsFigure 6.3.: Sequence diagram, initialization of the test suite58

6.5. Foreign Random Number Generatorstest_suite_main: rng_test_suite: xml_helper : buffered_random_ print test attributesnumber_generator : test_runner: concrete_test_runnerrun rng test1: run_tests2: print initial tag3: add initial attribute4: get rng from vector5: print rng specific tag6: add rng specific attributes7: print seed tag8: print seed9: get test from vector10: set_confidence_levelWhy first run the test andthen print the parameters?So, it is possible to printinteresting rng specificparameters, like the numberof bits per number11: seed12: warm_up13: run14: run15: print test tag16: print test attributes18: print parameter tag17: run rng test19: print parameter attributes20: start analyze tag21: analyzeFigure 6.4.: Sequence diagram, “run a test” part59

6. Extending the Random Number Generator Test SuitePseudoRandomNumberGenerator requirementsexpression return type descriptionX::result_type T type of random numbersoperator()() T returns next random numbermin() T lower bound of random numbersmax() T upper bound of random numbersX() – default constructorX(it1, it2) void creates an generator initialized with valuesbetween it1 and it2seed() void set same state like in X()seed(it1, it2) void seed generator with values between it1 andit2operator()() T returns next random numberx == y bool checks if generators have same statex != y bool checks if generators have not same stateoperator> std::istream& reads the generator from its textual representationTable 6.1.: Requirements for “Pseudo-Random Number Generators”There is no possibility to specify a seed function as a function pointer! Why not? Theproblem is the internal use of a clone method which duplicates the state of the generator. Inthis case we only have a function pointer onto the function delivering the next number. It isnot possible to copy the state of the generator. So, the seed function does not make sensebecause we can not seed from an initial state which is equal for all tests. – Using seededgenerators is possible via the register_seeded_rng method.To show the usage of the wrapper we give a short example. We assume that there is a filecalled mt199937ar.c implementing a variant of the “Mersenne twister”. We will generatenumbers of type double. To do this, the required C functions have to be declared in a C++file. This is done with the extern statement:extern "C"{/* generates a random number on [0,1)-real-interval */double genrand_real2(void)}Adding the generator to the test suit is not a great deal, one only has to specify the desiredtemplate parameters and its done.rng_wrapper mersenne_double;rngTest.register_seeded_rng

6.6. The XML Schema>>(mersenne_double, "C Mersenne (double)", "standard seed");To compile the whole thing the file containing the generator must be pre compiled into anobject file, which can be linked with the other parts of the test suite.6.6. The XML SchemaThe XML format was chosen in order to have a universal format with a simple structurewhich allows transformation to other formats like HTML or LATEX. Such transformations aredone with so called XSLT [5] (XML Stylesheet Language Translation) style sheets whichcontain rules to generate appropriate output. Here we cover the translation to HTML andLATEX.To view the results in HTML one only needs a “modern” web browser understandingXML and stylesheets. “Mozilla” and the “Internet Explorer” are capable to process theinstructions. The stylesheet is called xml2html.xslThere is also a stylesheet (xml2LaTeX.xsl) to translate the output to a LATEXsource file.To make this transformation, a XSLT processor is used. (A standard one is the “xsltproc”tool, available at [31] as a part of the “GNOME” project) The transformation delivers a LATEXsource file which simply can be processed to a Post-Script file or whatever.The structure, attributes and restrictions are defined in an XML schema. A graphicalrepresentation is shown in figure 6.5. The following list shows a short description of thedifferent tags and attributes, a detailed description of the whole schema is found in the source.RNG_TEST_SUITE_RESULTdate the tests starting dateRNGname the name of the random number generatorwarmup number of random numbers to throw away for warmupSEEDseed seed value or, if the generator was seeded by the user, the string user-seededdescription if the generator was seeded by the user, a description of the used seed (optional)TESTname the name of the random number generatorPARAMETERSANALYZEPARAMETERname name of the parameter61

6. Extending the Random Number Generator Test Suitevalue value of the parameterCHI_SQUAREname the name of the statistic (optional)chi2 the χ 2 valueprobability the probability for the χ 2 valuedof the degrees of freedom of the statisticKOLMOGOROV_SMIRNOVname the name of the statistic (optional)ksPlus the Kolmogorov-Smirnov K valueprobPlus the probability for the K valueksMinus the K¡ Kolmogorov-Smirnov valueprobMinus the probability for K¡ the valuedof the degrees of freedom of the statisticRESULTSname the name of the statistic (optional)PASSEDconfidenceLevel confidence level at which the test passesFAILEDconfidenceLevel confidence level at which the test failsRESULTname name of the result valuevalue value of the result value62

6.6. The XML SchemaRNGRNG_TEST_SUITE_RESULTname : xs:stringdate : xs:date 1..* warmup : xs:integer1..*SEEDseed : xs:integerdescription : xs:string1..*TESTname : xs:string1 1ANALYZEPARAMETERS0..*0..*RESULTSCHI_SQUAREname : xs:stringname : xs:string1..*KOLMOGOROV_SMIRNOVname : xs:string1..*PARAMETERname : xs:stringvalue : xs:string1..*result_statistic_tRESULTPASSEDFAILED1..* 1..*statistic_tPASSEDFAILEDprobability_t0..*RESULTname : xs:stringvalue : xs:string0..*FAILED0..* confidenceLevel : probability_t0..* PASSED0..*confidenceLevel : probability_tintegerFigure 6.5.: The XML-Schema63

A. Collection of Test ParametersThe following tables itemize tests and their parameters used in test suits or described in otherpublications.Test Numbers Iterations 1 Other Parametersχ 2 100000 10000 classes = 256χ 2 10000 10000 classes = 128Serial test 100000 1000 dimension = 2gridSize = 100Serial test 100000 1000 dimension = 3gridSize = 20Serial test 100000 1000 dimension = 4gridSize = 10Gap test 25000 1000 lowerGapLimit = 0upperGapLimit = 0.05maxGapCount = 30Gap test 25000 1000 lowerGapLimit = 0.45upperGapLimit = 0.55maxGapCount = 30Gap test 25000 1000 lowerGapLimit = 0.95upperGapLimit = 1maxGapCount = 30Maximum of t 2000 1000 t = 5bins = 5Maximum of t 2000 1000 t = 3bins = 3Collision test 16384 1000 dim = 2edge_length = 1024Collision test 16384 1000 dim = 4edge_length = 32Collision test 16384 1000 dim = 10edge_length = 4Run test 100000 1000 maxRunLength = 6Table A.1.: Test parameters used in [30]1 In the “Random Number Generator Test Suite”, the number of iterations is not a parameter of the test. Thetest must be wrapped with the iterate_test class. In this version this is not possible with the “Run test”because of lack of a wrapper class for the multiple_test base class.64

Test Numbers StepsRandom walk test n = 10 6 10 7 10 8 steps = 0¡¢¡¢¡1000n block test n = 10 4 steps = 10 6n block test n = 5000 steps = 10 8n block test n = 25000 steps = 10 7n block test n = 1500 steps = 10 9Table A.2.: Test parameters used in [29]Test Numbers Other Parametersχ 2 n = 1000000 classes = 100Serial test n = 500000 gridSize = 64dimension = 2Gap test n = 100000 lowerGapLimit = 0.5upperGapLimit = 0.6maxGapCount = 20Permutation test n = 200000 nrOfElements = 5Runs test n = 600000 maxRunLength = 7Coupon test n = 20000 different_coupons = 10maxSeq = 30Maximum of t n = 100000 t = 10bins = 10Poker test n = 100000 different_cards = 10Table A.3.: Test parameters used in the SPRNG test suite [21]65

B. ExamplesTo point out the ease of handling of the test suite and to show a number of possibilities, thereare some examples added to the source code. Most of the examples have a self-explanatoryname and contain a short description of the example inside the code.Here is the list of examples:bit_extract_example This is an example for the “Bit extract test“ in section 4.7. In the first part, thelower 10 bits are used to build a new random number, in the second part, bit number 20 of 10random numbers is used to build a new random number.bit_test_example This is an example for the “Bit test” in section 4.6. A mask of a length of 30 bitsis used to produce new random numbers.count_failings_example This is an example for the “Count failings test” in section 4.5. A test is run1000 times and it passes if it fails less than 100 times.doc_example This is the example from the documentation in section 4.1.foreign_rng_example This is an example of using a foreign random number generator as describedin section 6.5. The used random number generator is the original C version of the “MersenneTwister” which can be downloaded at http://www.math.keio.ac.jp/~nisimura/random/real1/mt19937-1.c. More detailed instructions are written in the source file.helsinki This is the same compilation as used in the “Comparative study of some pseudorandomnumber generators”, [30], excepting the runs test.iterating_example This is an example for the possibility of iterating tests as described in section 4.4.Each test is iterated 1000 times and analysed.iterator_seed_example This example shows how to seed a random number generator with iterators.Here, Boost’s “Mersenne Twister” is seeded with a vector filled by a linear congruentialgenerator.parallel_example This example shows how a simple parallel generator may be constructed and itsusage. A parallel generator of two different seeded “Lagged Fibonacci” generators is used.all_tests_example In this example all currently available tests are included.The parameters for the tests, excepting the “helsinki” example, are all examples. So, forreal tests they have to be changed or consciously accepted.66

C. Compiling the Test SuiteIf anyone does not want to run the Makefile or this file does not work, the RNGTS mayalso be compiled by hand. This is quite simple, one only has to consider three points:is the BOOST library installed?The BOOST library must be installed in order to compile the test suite. If it is not,the source can be found in [2]. If the library is installed once, one has to specify theinclude path. This is done with-I/path_to_boostis the “Runs test” used/included?If the “Runs test” is performed in the test suite or even if its header file is included,the BOOST Sandbox 1 has to be installed and specified in the include path. The “Runstest” uses the LAPACK and BLAS, so these libraries must also be available. 2 Becausethe included libraries are based on Fortran code, the g2c library has to be used. In theend we have to add the following arguments to the command line:-llapack -lblas -lg2c -I/path_to_boost-sandboxare any external random number generators used?Last but not least we can also use a external generator, If this is done, the generatormust be available as a pre-compiled object file, which has to be added to the argumentline likeext_gen.oIf no one of the three points above apply, the following command line may be used to compilethe test suit. The file containing the main routine is called RNG_test_suite_test.C.g++ -lm -I/path_to_boost -I. RNG_test_suite_test.C.C -o RNG_test_suite_test.C1 Also available at [2], via the “Sandbox CVS” link2 LAPACK and BLAS are installed on most systems. If not, they are available on the Internet at http://www.netlib.org/lapack/ and http://www.netlib.org/blas/67

Bibliography[1] Z. W. Birnbaum and F. H. Tingey. One-sided confidence contours for probability distributionfunctions. Annals of Mathematical Statistics, 22(4):592–596, 1951.[2] Booster. Boost libraries, 2002–2004. URL http://www.boost.org.[3] I. N. Bronstein, K. A. Semendjajew, G. Musiol, and H. Mühlig. Taschenbuch derMathematik. Harri Deutsch, Frankfurt am Main, 4 th edition, 1999. ISBN 3-8171-2004-1.[4] T. H. Chow. Tuning the collision test for stringency, 2000. URL http://citeseer.nj.nec.com/436535.html.[5] J. Clark. XSL Transformations (XSLT) Version 1.0, 1999. URL http://www.w3.org/TR/xslt.[6] A. Compagner. The Hierarchy of Correlations in Random Binary Sequences. Journalof Statistical Physics, 63(5/6):883–896, 1991.[7] R. R. Coveyou and R. D. Macpherson. Fourier Analysis of Uniform Random NumberGenerators. J. ACM, 14(1):100–119, 1967. ISSN 0004-5411. URL http://doi.acm.org/10.1145/321371.321379.[8] A. M. Ferrenberg, D. P. Landau, and Y. J. Wong. Monte Carlo Simulations: HiddenErrors from "Good" Random Number Generators. Physical Review Letters, 69(23):3382–3384, 1992.[9] M. Fischler. Distribution of minimum distance among n random points in d dimensions.Technical report, Fermilab (FNAL), 2001. URL http://www.slac.stanford.edu/spires/find/hep/www?r=fermilab-tm-2170. FERMILAB-TM-2170.[10] G. Gonnet. Repeating Time Test for U(0,1) Random Number Generators. Technicalreport, Informatik, ETH, Zurich, May 2003. URL http://www.inf.ethz.ch/personal/gonnet/RepetitionTest.html.[11] I. D. Hill and M. C. Pike. Algorithm 299: Chi-squared integral. Commun. ACM, 10(4):243–244, 1967. ISSN 0001-0782. URL http://doi.acm.org/10.1145/363242.363274.[12] I. D. Hill and M. C. Pike. Remark on Algorithm 299. ACM Trans. Math. Softw., 11(2):185, 1985. ISSN 0098-3500. URL http://doi.acm.org/10.1145/214392.214405.68

Bibliography[13] R. Häggkvist and P. H. Lundow. The Ising Partition Function for 2D Grids with PeriodicBoundary: Computation and Analysis. Journal of Statistical Physics, 108:429–457, 2002.[14] D. Ibbetson. Algorithm 209: Gauss. Commun. ACM, 6(10):616, 1963. ISSN 0001-0782. URL http://doi.acm.org/10.1145/367651.367664.[15] E. Ising. Beitrag zur Theorie des Ferromagnetismus. Zeitschrift für Physik, pages253–258, 1925.[16] D. E. Knuth. The Art of Computer Programming, Volume 2 (3rd Ed.): SeminumericalAlgorithms. Addison-Wesley Longman Publishing Co., Inc., 1997. ISBN 0-201-89684-2.[17] G. Marsaglia. A current view of random number generators. Computer Scienceand Statistics, 9(26):1–10, 1993. URL http://www.csis.hku.hk/~diehard/cdrom/linux.tar.gz:monkey.ps.[18] G. Marsaglia. The diehard test suite, 1995. URL http://stat.fsu.edu/~geo/diehard.html.[19] G. Marsaglia. The diehard test suite, 2003. URL http://www.csis.hku.hk/~diehard/.[20] G. Marsaglia and W. W. Tsang. Some difficult-to-pass tests of randomness. Journalof Statistical Software, 7(3):1–8, 2002. URL http://www.jstatsoft.org/v07/i03;http://www.jstatsoft.org/v07/i03/tuftests.c;http://www.jstatsoft.org/v07/i03/tuftests.pdf;http://www.jstatsoft.org/v07/i03/updates.[21] M. Mascagni. The scalable parallel random number generators library (sprng) for ascimonte carlo computations, 1999. URL http://sprng.cs.fsu.edu/.[22] M. Mascagni. A parallel version of the diehard test suite, 2003. URL http://www.cs.fsu.edu/~mascagni/research/.[23] U. Maurer. A Universal Statistical Test for Random Bit Generators. Journal of Cryptology,5(2):89–105, 1992.[24] O. E. Percus and P. A. Whitlock. Theory and application of Marsaglia’s monkey testfor pseudorandom number generators. ACM Trans. Model. Comput. Simul., 5(2):87–100, 1995. ISSN 1049-3301. URL http://doi.acm.org/10.1145/210330.210331.[25] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical Recipesin C: The Art of Scientific Computing. Cambridge University Press, 1992. ISBN0521437148. URL http://lib-www.lanl.gov/numerical/bookcpdf.html.69

Bibliography[26] A. Srinivasan, M. Mascagni, and D. Ceperley. Testing parallel random number generators.Parallel Comput., 29(1):69–94, 2003. ISSN 0167-8191.[27] I. Vattulainen. Framework for Testing Random Numbers in Parallel Calculations. PhysicalReview E, 59:7200, 1999.[28] I. Vattulainen, T. Ala-Nissila, and K. Kankaala. Physical Tests for Random Numbersin Simulations. Physical Review Letters, 73:2513–2516, 1994.[29] I. Vattulainen, T. Ala-Nissila, and K. Kankaala. Physical models as tests of randomness.Physical Review E, 52(3):3205–3214, 1995.[30] I. Vattulainen, K. Kankaala, J. Saarinen, and T. Ala-Nissila. A comparatitive studyof some pseudorandom number generators. Computer Physics Communications, 86:209–226, 1995.[31] D. Veillard. The XSLT C library for Gnome, 2003. URL http://xmlsoft.org/XSLT/xsltproc2.html.[32] E. Welzel. Rank of random matrices over gf[2], 1995. URL http://www.inf.ethz.ch/personal/emo/ps-files/SP-ExpRank.ps.[33] U. Wolff. Collective Monte Carlo Updating for Spin Systems. Physical Review Letters,62:361, 1989.70