Parallel statistical computing for statistical inference

Journal of Statistical Theory and Practice

Volumn 6, Issue 3, 2012, Pages 536-565

  Guo, Guangbao ^a  

^a SHANDONG UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Nonparametric inference; Regression; Statistical computing; Stochastic processes

Indexed keywords

EID: 84865266921     PISSN: 15598608     EISSN: 15598616     Source Type: Journal    
DOI: 10.1080/15598608.2012.695705     Document Type: Review

References (97)

1. Adams, N. M., S. P. J. Kirby, P. Harris, and D. B. Clegg. 1996. A review of parallel processing for statistical computation. Statist. Computing, 6, 37-49. (Pubitemid 126716898)
2. Arnal, J., V.Migallon, J. Penades, and D. B. Szyld. 2008. Newton additive and multiplicative Schwarz iterative methods. IMA J. Numer. Anal., 28, 143-161.
3. Azzini, I., R. Girardi, and M. Ratto. 2007. Parallelization of Matlab codes under Windows platform for Bayesian estimation: a dynare application. Working Paper 1, Euro-area Economy Modelling Centre. Available at: http://eemc.jrc.ec.europa.eu/.
4. Björck, A. 1996. Numerical methods for least squares problems. Philadelphia, SIAM.
5. Beddo, V. 2002. Applications of parallel programming in statistics. PhD dissertation, University of California, Los Angeles.
6. Beliakov, G. 2011. Parallel calculation of the median and order statistics on GPUs with application to robust regression. arXiv:1104.2732v1.
7. Benzi, M., and T. Dayar. 1995. The arithmetic mean method for finding the stationary vector of Markov chains. Parallel Algorithms Appl., 6, 25-37.
8. Benzi, M., F. Sgallari, and G. Spaletta. 1995. A parallel block projection method of the Cimmino type for finite Markov chains. In Computations with Markov chains, ed. W. J. Stewart, 65-80. Dordrecht, Kluwer Academic.
9. Benzi, M., and M. Tuma. 2002. A parallel solver for large-scale Markov chains. Appl. Numer. Math., 41, 135-153. (Pubitemid 34154709)
10. Bouaricha, A., and R. B. Schnabel. 1993. Parallel tensor methods for nonlinear equations and nonlinear least squares. PPSC, 639-643.
11. Bouyouli, R., K. Jbilou, R. Sadaka, and H. Sadok. 2006. Convergence properties of some block Krylov subspace methods for multiple linear systems. J. Comput. Appl. Math., 196, 498-511. (Pubitemid 44177514)
12. Brockwell, A. 2006. Parallel Markov chain Monte Carlo simulation by prefetching. J. Comput. Graphical Stati., 15, 246-261. (Pubitemid 43514420)
13. Bru, R., F. Pedroche, and D. B. Szyld. 2005. Additive Schwarz iterations for Markov chains. SIAM J. Matrix Anal. Appl., 27, 445-458.
14. Buckner, J., J. Wilson, M. Seligman, B. Athey, S. Watson, and F. Meng. 2010. The gputools package enables GPU computing in R. Bioinformatics, 26, 134-135.
15. Burrage, K., P. M. Burrage, and T. Tian. 2004. Numerical methods for strong solutions of stochastic differential equations: An overview. Proc. R. Soc. Lond. A, 460, 373-402. (Pubitemid 38333299)
16. Bylina, J. 2004. A distributed approach to solve large Markov chains. Proceedings from EuroNGIWorkshop: New Trends in Modeling, Quantitative Methods and Measurements, 145-154.
17. Gliwice, Jacek Skalmierski Computer Studio. Chilson, J., R. Ng, A. Wagner, and R. Zamar. 2006. Parallel computation of high-dimensional robust correlation and covariance matrices. Algorithmica, 45, 403-431. (Pubitemid 43876797)
18. Coleman, T. F., and P. E. Plassmann. 1992. A parallel nonlinear least-squares solver: Theoretical analysis and numerical results. SIAM J. Sci. Stat. Comput., 13, 771-793.
19. Craiu, R. V., and X. L. Meng. 2005. Multi-process parallel antithetic coupling for forward and backward Markov chain Monte Carlo. Ann. Stat., 33, 661-697. (Pubitemid 41053719)
20. Craiu, R. V., J. S. Rosenthal, and C. Yang. 2009. Learn from thy neighbor: Parallel-chain and regional adaptive MCMC. J. Am. Stat. Associ., 104, 1454-1466.
21. Creel, M. 2005. User-friendly parallel computations with econometric examples. Comput. Econ., 26(2), 107-128. (Pubitemid 41631015)
22. Creel,M., andW. L. Goffe. 2008. Multi-core CPUs, clusters, and grid computing: A tutorial. Comput. Econom., 32(4), 353-382.
23. Dai, B., Y. Peng, and B. Gong. 2010. Parallel option pricing with BSDE method on GPU. Ninth International Conference on Grid and Cloud Computing, 191-195, IEEE.
24. Dennis, J. E., and T. Steihaug. 1998. A Ferris-Mangasarian technique applied to linear least squares problems. Tech. Rep. CRPC-TR 98740. Rice University.
25. Dhillon, I. S., and D. S. Modha. 2000. A parallel data-clustering algorithm for distributed memory multiprocessors. In Large-scale parallel data mining, Lecture notes in artificial intelligence, ed. M. J. Zaki and C. T. Ho, vol. 1759, 245-260. New York, Springer-Verlag.
26. Fischer, M., and P. Kemper. 2001. Distributed numerical Markov chain analysis. In Proc. 8th Euro PVM/MPI, vol. 2131 of LNCS, ed. Y. Cotronis and J. Dongarra, 272-279. (Pubitemid 33348555)
27. Santorini, Greece. Flegal, J. M., and G. L. Jones. 2010. Implementing Markov chain Monte Carlo: Estimating with confidence. In Handbook of Markov chain Monte Carlo, ed. S. Brooks, A. Gelman, G. Jones, and X. Meng., p. 175-197, Boca Raton, FL, Chapman and Hall/CRC Press.
28. Flynn, M. J. 1966. Very high-speed computing systems. Proc. IEEE, 54, 1901-1909.
29. Flynn, M. J. 1972. Some computer organizations and their effectiveness. IEEE Trans. Computers, C-21, 948-960.
30. Gelman, A., and D. B. Rubin. 1992. Inference from iterative simulation using multiple sequences (with discussion). Statis. Sci., 7, 457-511.
31. Gentle J. E., W. Härdle, and Y. Mori, eds. 2004. Handbook of computational statistics. New York, Springer.
32. Geyer, C. J. 1991. Markov chainMonte Carlo maximum likelihood. In Computing science and statistics: Proceedings of the 23rd Symposium on the Interface, ed. E. M. Keramidas, 156-163. Fairfax Station, VA, Interface Foundation.
33. Guo, G. 2008. Schwarz methods for quasi-likelihood in generalized linear models. Commun. Stat. Simul Comput., 37, 2027-2036.
34. Guo, G., and S. Lin. 2010. Schwarz method for penalized quasi-likelihood in generalized additive models. Commun. Statist. Theory Methods, 39, 1847-1854.
35. Guo, G., and W. Zhao. 2012. Schwarz methods for quasi stationary distributions of Markov chains. Calcolo, 49, 21-39.
36. Gursoy, A. 2003. Data decomposition for parallel k-means clustering. In PPAM, Lecture Notes in Computer Science, vol. 319, ed. R.Wyrzykowski, J. Dongarra,M. Paprzycki, and J.Wasniewski, 241-248. New York, Springer.
37. Havranek, T., and Z. Stratkos. 1989. On practical experience with parallel processing of linear models. Bull. Inter. Statis. Inst., 53, 105-117.
38. Hayfield, T., and J. Racine. 2008. Nonparametric econometrics: The np package. J. Stat. Software, 27, 1-32.
39. Kontoghiorghes, E. 1999. Special issue on parallel processing and statistics. Comput. Statist. Data Anal., 31, 373-516.
40. Keese, A. 2003. A review of recent developments in the numerical solution of stochastic PDEs (stochastic finite elements). Informatikbericht 2003-6. Braunschweig, Germany, Technische Universitat Braunschweig.
41. Keese, A., and H. G. Matthies. 2003. Hierarchical parallel solution of stochastic systems. In Computational fluid and solid mechanics, vol. 2, ed. K.-J. Bathe, 2023-2025. Amsterdam: Elsevier.
42. Keese, A., and H. G. Matthies. 2004. Parallel computation of stochastic groundwater flow. Proc. NIC Symposium. p. 399-408, Germany.
43. Kontoghiorghes, E. 2000. Parallel algorithms for linear models: Numerical methods and estimation problems, Advances in computational economics, vol. 15, Boston, MA, Kluwer Academic Publishers.
44. Kontoghiorghes, E. 2006. Handbook of parallel computing and statistics. Boca Raton, FL, CRC Press.
45. Kwiatkowska, M., D. Parker, Y. Zhang, and R. Mehmood. 2004. Dual-processor parallelization of symbolic probabilistic model checking. MASCOTS'04, 123-130. IEEE Computer Society, Volendam, The Netherlands. (Pubitemid 40455426)
46. Lee, A., C. Yau, M. Giles, A. Doucet, and C. Holmes. 2010. On the utility of graphics cards to perform massively parallel simulation with advanced Monte Carlo methods. J. Comp. Graph. Stat., 19(4), 769-789.
47. Liu, H., Y. Peng, D. Wei, and B. Dai. 2011. X10 implementation of parallel option pricing with BSDE method. ACM SIGPLAN X10 Workshop. In Proceedings of the ACM SIGPLAN X10'11 Workshop, California, USA, June 2011.
48. Lozano, E., and E. Acuña. 2003. Parallel computation of kernel density estimates classifiers and their ensembles. Proc. International Conference on Computer, Communication and Control Technologies, Orlando, FL.
49. Lukasik, S. 2007. Parallel computing of kernel density estimates with MPI. Lecture Notes Computer Sci., 4489, 726-734.
50. Lubinsky, B., and F. Nicolls. 2011. Fast implementation of the FRAME algorithm using a GPU Gibbs sampler. PRASA2011, Johannesburg, South Africa.
51. Marek, I., and D. B. Szyld. 2004. Algebraic Schwarz methods for the numerical solution of Markov chains. Linear Algebra Appl., 386, 67-81.
52. Mehmood, R., and J. Crowcroft. 2005. Parallel iterative solution method for large sparse linear equation systems. Tech. Rep. UCAM-CL-TR-650, Computer Laboratory, University of Cambridge, UK.
53. Murray, L. 2012. GPU acceleration of the particle filter: TheMetropolis resampler. arXiv:1202.6163.
54. Hasenbusch, M., and S. Schaefer. 2010. Speeding up parallel tempering simulations. Phys. Rev., E82, 046707.
55. Heeswijk, M., Y. Miche, E. Oja, and A. Lendasse. 2011. GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing, 74(16), 2430-2437.
56. Hegland, M., I. McIntosh, and B. A. Turlach. 1999. A parallel solver for generalized additive models. Comput. Stat. Data Anal., 31(4), 377-396.
57. Hussain, H. M., K. Benkrid, A. T. Erdogan, and H. Seker. 2011. Highly parameterized k-means clustering on FPGAs: Comparative results with GPPs and GPUs. Proc. ReConFig (2011), Cancun, Mexico.
58. Nagel, K., and M. Rickert. 2001. Parallel implementation of the TRANSIMS microsimulation. Parallel Comput., 27, 1611-1639.
59. Nakano, J. 2004. Parallel computing techniques. In Handbook of computational statistics, ed. J. E. Gentle, W. Hadle, Y. Mori, 237-266. Berlin, Germany, Springer.
60. Niemi, J., and M. Wheeler. 2011. Efficient Bayesian inference in stochastic chemical kinetic models using graphical processing units. arXiv:1101.4242v1.
61. Owens, J., M. Houston, D. Luebke, S. Green, J. Stone, and J. Phillips. 2008. GPU computing. Proc. IEEE 96(5), 879-899.
62. Pagan, A., and A. Ullah. 1999. Nonparametric econometrics. New York, Cambridge University Press.
63. Pan, J., and D. Manocha. 2011. Fast GPU-based locality sensitive hashing for k-nearest neighbor computation. GISACM (2011), 211-220.
64. Peng, Y., B. Gong, H. Liu, and Y. Zhang. 2010. Parallel computing for option pricing based on the backward stochastic diffierential equation. High performance computing and applications (Lecture Notes in Computer Science, vol. 5938), 325-330. Berlin, Springer.
65. Peng, Y., B. Gong, H. Liu, and B. Dai. 2011. Option pricing on the GPU with backward stochastic differential equation. PAAP, 19-23.Fourth International Symposium on Parallel Architectures, Algorithms and Programming.
66. Platen, E., and N. Bruti-Liberati. 2010. Numerical solution of SDEs with jumps in finance. Applications of mathematics. Berlin, Springer.
67. Preis, T. 2011. GPU-computing in econophysics and statistical physics. EPJ-Special Topics, 194, 87-119.
68. Racine, J. 1995. Parallel distributed kernel estimation. Comput. Stat. Data Anal., 40, 293-302.
69. Racine, J., J. Hart, and Q. Li. 2006. Testing the significance of categorical predictor variables in nonparametric regression models. Econometric Rev., 25, 523-544. (Pubitemid 44905378)
70. Renaut, R. A. 1998. A parallel multisplitting solution of the least squares problem. Numer. Linear Algebra Appl., 5, 11-31. (Pubitemid 128475830)
71. Rossini, A. J., L. Tierney, and N. Li. 2007. Simple parallel statistical computing in R. J. Comput. Graph. Stat., 16, 399-420.
72. Rue, H. 2001. Fast sampling of Gaussian Markov random fields. J. R. Stat. Soc. B, 63, 325-338. (Pubitemid 33267689)
73. Ruoming, J., and Agrawal, G. 2001. A Middleware for developing parallel data mining applications. Proc. First SIAM Conference on Data Mining. Chicago, IL, 2001.
74. Sarkar, A., N. Benabbou, and R. Ghanem. 2006. Domain decomposition of stochastic PDEs and its parallel. HPCS 2006, 14-17.
75. Schmidberger, M. 2009. Parallel computing for biological data. Dissertation, LMU Munchen, Fakultat fur Mathematik, Informatik und Statistik, Munich, Germany.
76. Silverman, B. W. 1985. Density estimation for statistics and data analysis. London, Chapman and Hall.
77. Steinsland, I. 2007. Parallel exact sampling and evaluation of Gaussian Markov random fields. Comput. Stat. Data Anal., 51, 2969-2981. (Pubitemid 46186539)
78. Stewart, W. J. 2007. Performance modeling and Markov chains. In Formal methods for performance evaluation, ed. M. Bernardo and J. Hillston, 1-33. SFM 2007, LNCS 4486. New York, Springer.
79. Strid, I. 2010. Efficient parallelisation of Metropolis-Hastings algorithms using a prefetching approach. Comput. Stat. Data Anal., 54, 2814-2835.
80. Subber, W. and A. Sarkar. 2010. Domain decomposition of stochastic PDEs: A novel preconditioner and its parallel performance. In High performance computing systems and applications, ed. D. Mewhort, N. Cann, G. Slater, and T. Naughton, 251-268. New York, Springer.
81. Suchard, M. A., Q. Wang, C. Chan, J. Frelinger, A. Cron, and M. West. 2010. Understanding GPU programming for statistical computation: Studies in massively parallel massive mixtures. J. Comput. Graph. Stat., 19(2), 419-438.
82. Suri, R., D. Deodhare, and P. Nagabhushan. 2002. Parallel Levenberg-Marquardt-based neural network training on linux clusters-A case study. ICVGIP, Ahmadabad, India.
83. Temple Lang, D. 1997. A multi-threaded extension to a high level interactive statistical computing environment. PhD dissertation, University of California, Berkeley.
84. Tibbits,M.M., M. Haran, and J. C. Liechty. 2010. Parallel multivariate slice sampling. Stat. Comput., 20, 1-16.
85. Trebst, S., M. Troyer, and U. Hansmann. 2006. Optimized parallel tempering simulations of proteins. J. Chem. Phys., 124,174903.
86. Tran, M. 2010. A parallel four step domain decomposition scheme for coupled forward backward stochastic differential equation. arXiv:1008.0353v1.
87. Weare, J. 2007. Efficient conditional path sampling of stochastic differential equations by parallel marginalization. Proc. Natl. Acad. Sci. USA, 104, 12657-12662. (Pubitemid 47255210)
88. Whiley, M. and S. P.Wilson. 2004. Parallel algorithms forMarkov chain Monte Carlo in latent spatial Gaussian models. Stat. Comput., 14, 171-179. (Pubitemid 39046708)
89. Wilkinson, D. 2006. Parallel Bayesian computation. In Handbook of parallel computing and statistics, ed. E. J. Kontoghiorghes, 477-508.Boca Raton, FL, Chapman and Hall.
90. Xiu, D., and G. E. Karniadakis. 2003. Modeling uncertainty in flow simulations via generalized polynomial chaos. J. Comput. Phys., 187, 137-167 (Pubitemid 36515575)
91. Xu, M., E. Wegman, and J. Miller. 1991. Parallelizing multiple linear regression for speed and redundancy: An empirical study. J. Stat Comput. Simulation, 39, 205-214.
92. Yan, J., M. K. Cowles, S. Wang, and M. P. Armstrong. 2007. Parallelizing MCMC for Bayesian spatiotemporal geostatistical models. Stat. Comput., 17, 323-335.
93. Yang, T. 1996. Execution time analysis for least squares problems on massively parallel distributed memory computers. Proc. International Conference on Computational Modeling and Computing (CMCP-96), Dubna, Russia.
94. Zareski, D., B. Wade, P. Hubbard, and P. Shirley. 1995. Efficient parallel global illumination using density-estimation. Proc. ACM Parallel Rendering Symposium, 47-54. Atlanta, GA, 1995.
95. Zhang, Y., D. Parker, and M. Kwiatkowska. 2005. A wavefront parallelisation of CTMC solution using MTBDDs. In DSN'05, 732-742. IEEE Computer Society Press, Yokohama, Japan.
96. Zhou, H., K. Lange, and M. A. Suchard. 2010. Graphics processing units and high-dimensional optimization. Stat. Sci., 25(3), 311-324.
97. Zhu, W., and Y. Li. 2010. GPU-accelerated differential evolutionary Markov chain Monte Carlo method for multi-objective optimization over continuous space. Proc. 7th IEEE ICAC-BADS, Washington, DC.