TestU01: A C library for empirical testing of random number generators

ACM Transactions on Mathematical Software

Volumn 33, Issue 4, 2007, Pages

  L'ecuyer, Pierre ^a   Simard, Richard ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

Random number generators; Random number tests; Statistical software; Statistical test

Indexed keywords

C (PROGRAMMING LANGUAGE); COMPUTER SOFTWARE; FUNCTION EVALUATION; PROBLEM SOLVING; STATISTICAL TESTS;

RANDOM NUMBER GENERATORS; RANDOM NUMBER TESTS; STATISTICAL SOFTWARES;

RANDOM NUMBER GENERATION;

EID: 34548237327     PISSN: 00983500     EISSN: 15577295     Source Type: Journal    
DOI: 10.1145/1268776.1268777     Document Type: Article

References (135)

1. AGARWAL, R. C., ENENKEL, R. F., GUSTAVSON, F. G., KOTHARI, A., AND M. ZUBAIR. 2002. Fast pseudorandom-number generators with modulus 2* or 2* - 1 using fused multiply-add. IBM J. Res. and Dev. 46, 1, 97-116.
2. ANDERSON, T. W. AND DARLING, D. A. 1952. Asymptotic theory of certain goodness of fit criteria based on stochastic processes. Ann. Math. Stat. 23, 193-212.
3. BARKER, E. AND KELSEY, J. 2006. Recommendation for random number generation using deterministic random bit generators. SP-800-90, U.S. DoC/National Institute of Standards and Technology, http ://csrc.nist.gov/publications/nistpubs/.
4. BERLEKAMP, E. R. 1984. Algebraic Coding Theory. Aegean Park Press, Laguna Hills, CA.
5. BLCKEL, P. J. AND BREIMAN, L. 1983. Sums of functions of nearest neighbor distances, moment bounds, limit theorems and a goodness of fit test. Ann. Probab. 11, 1, 185-214.
6. BRENTJR.P. 2004. Note on Marsaglia's xorshift random number generators. J. Statis. Softw. 11, 5, 1-4.http ://www.jstatsoft.org/v11/ 105/brent.pdf.
7. BROWN, F. B. AND NAGAYA, Y. 2002. The MCNP5 random number generator. Tech. rep. LA-UR-02-3782, Los Alamos National Laboratory.
8. CARTER, G. D. 1989. Aspects of local linear complexity. Ph.D. thesis, University of London.
9. COUTURE, R. AND L'ECUYER, P. 1994. On the lattice structure of certain linear congruential sequences related to AWC/SWB generators. Mathem. Comput. 62, 206, 798-808.
10. COUTURE, R. AND L'ECUYER, P. 1997. Distribution properties of multiply-with-carry random number generators. Mathem. Comput. 66, 218, 591-607.
11. COVEYOU, R. R. 1969. Random number generation is too important to be left to chance. In Applied Probability and Monte Carlo Methods and Modern Aspects of Dynamics. Studies in Applied Mathematics 3. Society for Industrial and Applied Mathematics, Philadelphia, PA. 70-111.
12. DAEMEN, J. AND RIJMEN, V. 2002. The Design of Rijndael. Springer-Verlag, Berlin, Germany, http: //www.esat.kuleuvan.ac. be/~rijmen/rijndael/.
13. DARLING, D. A. 1960. On the theorems of Kolmogorov-Smirnov. Theo. Probab. Applic. V, 4, 356-360.
14. DENG, L.-Y. 2005. Efficient and portable multiple recursive generators of large order. ACM Trans. Model. Comput. Simul. 15, 1, 1-13.
15. DENG, L.-Y. AND LIN, D. K. J. 2000. Random number generation for the new century. Ame. Statis. 54, 2, 145-150.
16. DURBIN, J. 1973. Distribution theory for tests based on the sample distribution function. SIAM CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, PA.
17. EICHENAUER, J. AND LEHN, J. 1986. A nonlinear congruential pseudorandom number generator. Statistische Hefte 27, 315-326.
18. EICHENAUER-HERHMANN, J. 1992. Inversive congruential pseudorandom numbers: A tutorial. Inte. Statist. Rev. 60, 167-176.
19. ERDMANN, E. D. 1992. Empirical tests of binary keystreams. M.S. thesis, Department of Mathematics, Royal Holloway and Bedford New College, University of London.
20. FELLER, W. 1968. An Introduction to Probability Theory. Vol. 1, 3rd Ed. John Wiley, New York, NY.
21. FERRENBERG, A. M., LANDAU, D. P., AND WONG, Y. J. 1992. Monte Carlo simulations: Hidden errors from "good" random number generators. Phy. Rev. Lett. 69, 23, 3382-3384.
22. β An exhaustive analysis for β = 32 and a partial analysis for β = 48. Mathem. Comput. 54, 189 (Jan), 331-344.
23. FISHMAN, G. S. 1996. Monte Carlo: Concepts, Algorithms, and Applications. Springer Series in Operations Research. Springer-Verlag, Berlin, Germany.
24. 31 - 1. SIAM J. Sci. Statist. Comput. 7, 1, 24-45.
25. FÖLDES, A. 1979. The limit distribution of the length of the longest head run. Period Math. Hung. 10, 4, 301-310.
26. GALASSI, M., DAVIES, J., THEILER, J., GOUGH, B., JUNGMAN, G., BOOTH, M., AND ROSSI, F. 2004. GSL - GNU Scientific Library: Reference manual, http://www.gnu.org/software/gs1/.
27. GAMMEL, B. M. 2005. Matpack C++ Numerics and Graphics Library, Release 1.8.1. http://users.physik.tu-muenchen.de/gammel/matpack/index.html.
28. GOLDREIOH, O. 1999. Modern Cryptography, Probabilistic Proofs and Pseudo-Randomness. Springer-Verlag, Berlin, Germany.
29. GOOD, I. J. 1953. The serial test for sampling numbers and other tests for randomness. In Proceedings of the Cambridge Philosophical Society 49, 276-284.
30. GORDON, L., SCHILLING, M. F., AND WATERMAN, S. 1986. An extreme value theory for long head runs. Probab. Theo. Relat. Fields 72, 279-287.
31. GREENWOOD, R. E. 1955. Coupon collector's test for random digits. Math. Tables Aids Comput. 9, 1-5, 224, 229.
32. HELLEKALEK, P. 1995. Inversive pseudorandom number generators: Concepts, results, and links. In Proceedings of the 1995 Winter Simulation Conference, C. Alexopoulos, K. Kang, W. R. Lilegdon, and D. Goldsman, Eds. IEEE Press, 255-262.
33. HELLEKALEK, P. 1998. Don't trust parallel Monte Carlol In 12th Workshop on Parallel and Distributed Simulation. Banff, Canada. IEEE Computer Society, Los Alamitos, CA, 82-89.
34. HELLEKALEK, P. AND WEGENKITTL, S. 2003. Empirical evidence concerning AES. ACM Trans. Model. Comput. Simul. 13, 4, 322-333.
35. HOLIAN, B. L., PERCUS, O. E., WARNOCK, T. T., AND WHITLOCK, P. A. 1994. Pseudorandom number generator for massively parallel molecular-dynamics simulations. Phys. Rev. E 50, 2, 1607-1615.
36. IBM 1968. System/360 Scientific Subroutine Package. Version III, Programmer's Manual. IBM, White Plains, New York.
37. IMSL. 1997. IMSL STAT/LIBRARY. Visual Numerics Inc., Houston, TX. http://www.vni.com/books/dod/pdf/STATVol_2.pdf.
38. INTEL. 2003. Vector statistical library notes. Tech. rep. version 3, Intel Corporation, http://www.intal.com/software/products/mkl/docs/ vslnotes.pdf.
39. JAMES, F. 1994. RANLUX: A Fortran implementation of the high-quality pseudorandom number generator of Lüscher. Comput. Phys. Comm. 79, 111-114.
40. JENKINS, B. 1996. ISAAC, in fast software encryption. In Proceedings of the 3rd International Workshop. Cambridge, UK, D. Gollmann, Ed. Lecture Notes in Computer Science, vol. 1039. Springer-Verlag, 41-49. http://burtleburtle.net/bob/rand/isaacafa.html.
41. KENDALL, M. G. AND BABINGTON-SMITH, B. 1939. Second paper on random sampling numbers. J. Royal Statis. Soc. Suppl. 6, 51-61.
42. KIRKPATRLCK, S. AND STOLL, E. 1981. A very fast shift-register sequence random number generator. J. Comput. Physics 40, 517-526.
43. KIRSCHENHOPER, P., PRODINGER, H., AND SZPANKOWSKI, W. 1994. Digital search trees again revisited: The internal path length perspective. SIAM J. Comput. 23, 598-616.
44. KNUTH, D. E. 1981. The Art of Computer Programming. Vol. 2: Seminumerical Algorithms, 2nd Ed. Addison-Wesley, Reading, MA.
45. KNUTH, D. E. 1998. The Art of Computer Programming. Vol. 2: Seminumerical Algorithms, 3rd ed. Addison-Wesley, Reading, MA.
46. LAGARIAS, J. C. 1993. Pseudorandom numbers. Statis. Sci. 8, 1, 31-39.
47. L'ECUYER, P. 1988. Efficient and portable combined random number generators. Comm. ACM 31, 6, 742-749 and 774. (See also the correspondence in the same journal, 32, 8 (1989) 1019-1024.)
48. L'ECUYER, P. 1990. Random numbers for simulation. Comm,. ACM 33, 10, 85-97.
49. L'ECUYER, P. 1992. Testing random number generators. In Proceedings of the 1992 Winter Simulation Conference. IEEE Press, 305-313.
50. L'ECUYER, P. 1994. Uniform random number generation. Ann. Operat. Res. 53, 77-120.
51. L'ECUYER, P. 1996a. Combined multiple recursive random number generators. Operat. Res. 44, 5, 816-822.
52. L'ECUYER, P. 1996b. Maximally equidistributed combined Tausworthe generators. Mathem. Comput. 65, 213, 203-213.
53. L'ECUYER, P. 1997a. Bad lattice structures for vectors of non-successive values produced by some linear recurrences. INFORMS J. Comput. 9, 1, 57-60.
54. L'ECUYER, P. 1997b. Tests based on sum-functions of spacings for uniform random numbers. J. Stat. Comput. Simul. 59, 251-269.
55. L'ECUYER, P. 1999a. Good parameters and implementations for combined multiple recursive random number generators. Operat. Res. 47, 1, 159-164.
56. L'ECUYER, P. 1999b. Tables of maximally equidistributed combined LFSR generators. Mathem. Comput. 68, 225, 261-269.
57. L'ECUYER, P. 2001. Software for uniform random number generation: Distinguishing the good and the bad. In Proceedings of the Winter Simulation Conference. IEEE Press, 95-105.
58. L'ECUYER, P. 2004. Random number generation. In Handbook of Computational Statistics, J. E. Gentle, W. Haerdle, and Y. Mori, Eds. Springer-Verlag, Berlin, Germany. 35-70. Chapter II.2.
59. L'ECUYER, P. circa 2006. Uniform random number generation. In Stochastic Simulation, S. G. Henderson and B. L. Nelson, Eds. Handbooks of Operations Research and Management Science. Elsevier Science. To appear.
60. L'ECUYER, P. AND ANDRES, T. H. 1997. A random number generator based on the combination of four LCGs. Mathem. Comput. Simul. 44, 99-107.
61. L'ECUYER, P. BLOUIN, R. AND COUTURE, R. 1993. A search for good multiple recursive random number generators. ACM Tran. Model. Comput. Simul. 3, 2, 87-98.
62. L'ECUYER, P. AND BUIST, E. 2005. Simulation in Java with SSJ. In Proceedings of the Winter Simulation Conference. IEEE Press, 611-620.
63. L'ECUYER, P. CORDEAU, J.-F., AND SIMARD, R. 2000. Close-point spatial tests and their application to random number generators. Operat. Res. 48, 2, 308-317.
64. L'ECUYER, P. AND GRANGER- PICHÉ, J. 2003. Combined generators with components from different families. Mathem. Comput Simul. 62, 395-404.
65. L'ECUYER, P. AND HELLEKALEK, P. 1998. Random number generators: Selection criteria and testing. In Random, and Quasi-Random Point Sets, P. Hellekalek and G. Larcher, Eds. Lecture Notes in Statistics, vol. 138. Springer-Verlag, Berlin, Germany. 223-265.
66. r. ACM Trans. Math. Soft. 25, 3, 367-374.
67. L'ECUYER, P. AND SIMARD, R. 2001. On the performance of birthday spacings tests for certain families of random number generators. Mathem. Comput. Simul. 55, 1-3, 131-137.
68. L'ECUYER, P., SIMARD, R., CHEN, E. J., AND KELTON, W. D. 2002. An object-oriented random-number package with many long streams and substreams. Operat. Res. 50, 6, 1073-1075.
69. L'ECUYER, P., SIMARD, R., AND WEGENKITTL, S. 2002. Sparse serial tests of uniformity for random number generators. SIAM J. Scient. Comput. 24, 2, 652-668.
70. r. In Proceedings of the Winter Simulation Conference, J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick, Eds. IEEE Press, 683-689.
71. L'ECUYER, P. AND TOUZIN, R. 2004. On the Deng-Lin random number generators and related methods. Statis. Comput. 14, 5-9.
72. LEWIS, P. A. W., GOODMAN, A. S., AND MILLER, J. M. 1969. A pseudo-random number generator for the system/360. IBM Syst. J. 8, 136-143.
73. LIANG, Y. AND WHITLOCK, P. A. 2001. A new empirical test for parallel pseudo-random number generators. Mathem. Comput. Simul. 55, 1-3, 149-158.
74. LUSCHER, M. 1994. A portable high-quality random number generator for lattice field theory simulations. Comput. Physics Comm. 79, 100-110.
75. MAPLESOPT. 2006. Maple User Manual. Waterloo Maple Inc., Waterloo, Canada, http://www.maplesoft.com/products/maple/.
76. MARSAGLIA, G. 1972. The structure of linear congruential sequences. In Applications of Number Theory to Numerical Analysis, S. K. Zaremba, Ed. Academic Press, 249-285.
77. MARSAGLIA, G. 1985. A current view of random number generators. In Computer Science and Statistics, Sixteenth Symposium on the Interface. Elsevier Science Publishers, North-Holland, Amsterdam, The Netherlands. 3-10.
78. MARSAGLIA, G. 1996. DIEHARD: A battery of tests of randomness, http://stat.fsu.edu/"geo/diehard.html.
79. MARSAGLIA, G. 1997. A random number generator for C. Posted to the electronic billboard sci. math. num- analysis.
80. MARSAGLIA, G. 1999. Random numbers for C: The END? Posted to the electronic billboard sci. crypt.random-numbers.
81. MARSAGLIA, G. 2002. Good 64-bit RNG's. Posted to the electronic billboard sci.crypt.random-numbers.
82. MARSAGLIA, G. 2003. Xorshift RNGs. J. Statis. Soft. 8, 14, 1-6. http://www. J statsoft.org/v08/i14/xorshift.pdf.
83. MARSAGLIA, G., ANANTHANARAYANAN, K., AND PAUL, N. 1973. How to use the McGill random number package SUPER-DUPER. Tech. rep., School of Computer Science, McGill University, Montreal, Canada.
84. MARSAGLIA, G., NARASIMHAN, B., AND ZAMAN, A. 1990. A random number generator for PC's. Comput. Physics Comm. 60, 345-349.
85. MARSAGLIA, G. AND TSANG, W W. 2002. Some difficult-to-pass tests of randomness. J. Statist. Soft. 7, 3, 1-9. http://www.jstatsoft.org/v07/103/tuftests.pdf.
86. MARSAGLIA, G. AND TSAY, L.-H. 1985. Matrices and the structure of random number sequences. Lira. Algebra Applic. 67, 147-156.
87. MARSAGLIA, G. AND ZAMAN, A. 1991. A new class ofrandom number generators. Ann. Appl. Proba. 1, 462-480.
88. MARSAGLIA, G. AND ZAMAN, A. 1993a. The KISS generator. Tech. rep., Department of Statistics, University of Florida.
89. MARSAGLIA, G. AND ZAMAN, A. 1993b. Monkey tests for random number generators. Comput. Math. Applic. 26, 9, 1-10.
90. MASCAGNI, M. AND SRINIVASAN, A. 2000. Algorithm 806: SPRNG: A scalable library for pseudoran-dom number generation. ACM Trans. Mathem. Soft. 26, 436-461.
91. MASSEY, J. L. 1969. Shift-register synthesis and BCH decoding. IEEE Trans. Inf. Theor IT-15, 122-127.
92. MATHSOFT INC. 2000. S-PLUS 6.0 Guide to Statistics. Vol. 2. Data Analysis Division, Seattle, WA.
93. MATSUMOTO, M. AND KURITA, Y. 1992. Twisted GFSR generators. ACM Trans. Model. Comput. Simul. 2, 3, 179-194.
94. MATSUMOTO, M. AND KURITA, Y. 1994. Twisted GFSR generators II. ACM Trans. Model. Comput. Simul. 4, 3, 254-266.
95. MATSUMOTO, M. AND NISHIMURA, T. 1998. Mersenne twister: A 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8, 1, 3-30.
96. MAURER, J., ABRAHAMS, D., DAWES, B., AND RIVERA, R. 2004. Boost random number library, http: //www.boost.org/libs/random/index.html.
97. MAURER, U M. 1992. A universal statistical test for random bit generators. J. Cryptol. 5, 2, 89-105.
98. MOLER, C. 2004. Numerical Computing with MATLAB. SIAM, Philadelphia, PA.
99. NAG. 2002. The NAG C Library Manual, Mark 7. The Numerical Algorithms Group. http ://www.nag.co.uk/numeric/cl/manual/pdf/G05/g05cac.pdf and http ://www.nag.co.uk/numeric/f1/manual/pdf/G05/g05kaf.pdf.
100. NIEDERREITER, H. 1991. The linear complexity profile and the jump complexity of keystream sequences. In Advances in Cryptology: Proceedings of EUROCRYPT90. Springer-Verlag, Berlin, Germany, 174-188.
101. NIEDERREITER, H. 1992. Random number generation and quasi-monte carlo methods. SIAM CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 63. SIAM, Philadelphia, PA.
102. NIST. 2001. Advanced encryption standard (AES). FIPS-197, U.S. DoC/National Institute of Standards and Technology. http://csrc.nist.gov/ CryptoToolkit/tkencryption.html.
103. NIST. 2002. Secure hash standard (SHS). FIPS-186-2, with change notice added in february 2004, U.S. DoC/National Institute of Standards and Technology, http://csro.nist.gov/CryptoToolkit/tkhash.html.
104. PANNETON, F. AND L'ECUYER, P. 2005. On the xorshift random number generators. ACM Trans. Model. Comput. Simul. 15, 4, 346-361.
105. PANNETON, F., L'ECUYER, P., AND MATSUMOTO, M. 2006. Improved long-period generators based on linear recurrences modulo 2. ACM Trans. Math. Soft. 32, 1, 1-16.
106. PERCUS, O. E. AND WHITLOCK, P. A. 1995. Theory and application of Marsaglia's monkey test for pseudorandom number generators. ACM Trans. Model. Comput. Simul. 5, 2, 87-100.
107. PRESS, W. H. AND TEUKOLSKY, S. A. 1992. Numerical Recipes in C. Cambridge University Press, Cambridge, UK.
108. PROJECT, T. G. 2003. R: An Enuironment for Statistical Computing and Graphics. The Free Software Foundation. Version 1.6.2. http://www.gnu.org/directory/GNU/R.html.
109. READ, T. R. C. AND CRESSIE, N. A. C. 1988. Goodness-of-Fit Statistics for Discrete Multivariate Data. Springer Series in Statistics. Springer-Verlag, Berlin, Germany.
110. RIJMEN, V., BOSSELSBRS, A., AND BARRETO, P. 2000. Optimised ANSI C code for the Rijndael cipher (now AES). Public domain software.
111. RIPLEY, B.D. 1990. Thoughts on pseudorandom number generators. J. Comput. Appl. Mathem. 31, 153-163.
112. RIPLEY, B. D. AND VENABLES, W N. 1994. Modern Applied Statistics with S-Plus. Springer-Verlag, Berlin, Germany.
113. RUKHIN, A., SOTO, J., NECHVATAL, J., SMID, M., BARKER, E., LEIGH, S., LEVENSON, M., VANGEL, M., BANKS, D., HECKERT, A., DRAY, J., AND VO, S. 2001. A statistical test suite for random and pseudorandom number generators for cryptographic applications. NIST special publication 800-22, National Institute of Standards and Technology (NIST), Gaithersburg, MD. http://csrc.nist.gov/rng/.
114. RUKHIN, A. L. 2001. Testing randomness: A suite of statistical procedures. Theo. Probab. Applic. 45, 1, 111-132.
115. RYABKO, B. Y., MONAREV, V A., AND SHOKIN, Y I. 2005. A new type of attack on block ciphers. Probl. Informa. Transmis. 41, 4, 385-394.
116. RYABKO, B. Y., STOGNIENKO, V. S., AND SHOKIN, Y I. 2004. A new test for randomness and its application to some cryptographic problems. J. Statist. Plan. Infer. 123, 365-376.
117. SCIFACE SOFTWARE. 2004. MuPAD. SciFace Software GmbH & Co.KG, Paderborn, Germany, http: //www.mupad.de/home.html.
118. SINCLAIR, C. D. AND SPURR, B. D. 1988. Approximations to the distribution function of the Anderson-Darling test statistic. J. Amer. Statist. Ass. 83, 404, 1190-1191.
119. STEPHENS, M. A. 1970. Use of the Kolmogorov-Smirnov, Cramér-Von Mises and related statistics without extensive tables. J. Royal Statis. Soc., Series B 33, 1, 115-122.
120. STEPHENS, M. S. 1986a. Tests based on EDF statistics. In Goodness-of-Fit Techniques, R. B. D'Agostino and M. S. Stephens, Eds. Marcel Dekker, New York, NY.
121. STEPHENS, M. S. 1986b. Tests for the uniform distribution. In Goodness-of-Fit Techniques, R. B. D'Agostino and M. S. Stephens, Eds. Marcel Dekker, New York, NY, 331-366.
122. TAKASHIMA, K. 1996. Last visit time tests for pseudorandom numbers. J. Japan. Soc. Comp. Satist. 9, 1, 1-14.
123. TEZUKA, S. 1995. Uniform Random Numbers: Theory and Practice. Kluwer Academic Publishers, Norwell, MA.
124. TEZUKA, S., L'ECUYER, P., AND COUTURE, R. 1994. On the add-with-carry and subtract-with-borrow random number generators. ACM Trans. Model. Comput. Simula. 3, 4, 315-331.
125. TOOTILL, J. P. R., ROBINSON, W D., AND EAGLE, D. J. 1973. An asymptotically random Tausworthe sequence. J. ACM 20, 469-481.
126. UGRIN-SPARAC, G. 1991. Stochastic investigations of pseudo-random number generators. Comput. 46, 53-65.
127. VATTULAINEN, I., ALA-NISSILA, T., AND KANKAALA, K. 1995. Physical models as tests of randomness. Physic. Rev. E 52, 3, 3205-3213.
128. WANG, M. Z. 1997. Linear complexity profiles and jump complexity. Inform. Proces. Let. 61, 165-168.
129. WEGENKTTTL, S. 1998. Generalized φ-divergence and frequency analysis in Markov chains. Ph.D. thesis, University of Salzburg, http://random.mat.sbg.ac.at/team/.
130. WEGENKTTTL, S. 2001. Entropy estimators and serial tests for ergodic chains. IEEE Trans. Inform. Theo. 47, 6, 2480-2489.
131. WEGENKTTTL, S. 2002. A generalized φ-divergence for asymptotically multivariate normal models. J. Multivari. Anal. 83, 288-302.
132. WICHMANN, B. A. AND HILL, I. D. 1982. An efficient and portable pseudo-random number generator. App. Statis. 31, 188-190. See also corrections and remarks in the same journal by Wichmann and Hill, 33 (1984) 123; McLeod 34 (1985) 198-200; Zeisel 35 (1986) 89.
133. p - 1. ACM Trans. Mathem. Softw. 23, 2, 255-265.
134. ZIFF, R. M. 1998. Four-tap shift-register-sequence random-number generators. Comput. Physics 12, 4, 385-392.
135. ZIV, J. AND LEMPEL, A. 1978. Compression of individual sequences via variable-rate coding. IEEE Trans. on Inform. Theo. 24, 5, 530-536.