Scalable strategies for computing with massive data

Journal of Statistical Software

Volumn 55, Issue 14, 2013, Pages 1-19

  Kane, Michael J ^a   Emerson, John W ^a   Weston, Stephen ^b  

^a YALE UNIVERSITY   (United States)

^b Yale University   (United States)

Author keywords

Concurrent programming; Memory mapping; Parallel programming; Shared memory; Statistical computing

Indexed keywords

EID: 84888087862     PISSN: 15487660     EISSN: None     Source Type: Journal    
DOI: 10.18637/jss.v055.i14     Document Type: Article

References (41)

1. Adler D, Gläser C, Nenadic O, Oehlschlägel J, Zucchini W (2013). FF: Memory-Efficient Storage of Large Data on Disk and Fast Access Functions. R package version 2.2-12, URL http://CRAN.R-project.org/package=ff.
2. Baglama J, Reichel L (2005). "Augmented Implicitly Restarted Lanczos Bidiagonalization Methods." SIAM Journal of Scientific Computing, 27(1), 19-42.
3. Baglama J, Reichel L (2012). Irlba: Fast Partial SVD by Implicitly-Restarted Lanczos Bidiag-onalization. R package version 1.0.2, URL http://CRAN.R-project.org/package=irlba.
4. Bennet J, Lanning S (2007). "The Netflix Prize." In Proceedings of the KDD Cup and Workshop, San Jose, California. URL http://www.cs.uic.edu/~liub/KDD-cup-2007/NetflixPrize-description.pdf.
5. Boral H, Alexander W, Clay L, Copeland G, Danforth S, Franklin M, Hart B, Smith M, Valduriez P (1990). "Prototyping Bubba, A Highly Parallel Database System." IEEE Transactions on Knowledge and Data Engineering, 2(1), 4-24.
6. Chen Q, Therber A, Hsu M, Zeller H, Zhang B, Wu R (2009)."Efficiently Support MapReduce-like Computation Models Inside Parallel DBMS." In Proceedings of the 2009 International Database Engineering and Applications Symposium, pp. 43-53. ACM, New York.
7. Dawes B, et al. (2013). "The Boost C++ Libraries." URL http://www.boost.org/.
8. DeWitt DJ, Gerber RH, Graefe G, Heytens ML, Kumar KB, Muralikrishna M (1986). "GAMMA - A High Performance Dataflow Database Machine." In Proceedings of the Twelfth International Conference on Very Large Data Bases (VLDB'86), August 25-28, Kyoto, Japan, pp. 228-237. URL http://www.vldb.org/conf/1986/P228.PDF.
9. Efron B (2005). "Bayesians, Frequentists, and Scientists." Journal of the American Stistical Association, 100(469), 1-5.
10. Emerson JW, Kane MJ (2013a). Biganalytics: A Library of Utilities for big.matrix Objects of Package bigmemory. R package version 1.1.1, URL http://CRAN.R-project.org/package=biganalytics.
11. Emerson JW, Kane MJ (2013b). "The bigmemory Project Website." URL http://www.bigmemory.org/.
12. Emerson JW, Kane MJ, Eddelbuettel D, Allaire JJ, Francois R (2013). BH: The Boost C++ Libraries. R package version 1.51.0-3, URL http://CRAN.R-project.org/package=BH.
13. Gaujoux R (2013). NMF: Algorithms and Framework for Nonnegative Matrix Factorization (NMF). R package version 0.17, URL http://CRAN.R-project.org/package=NMF.
14. Gaujoux R, Seoighe C (2010). "A Flexible R Package for Nonnegative Matrix Factorization." BMC Bioinformatics, 11(1), 367.
15. Ihaka R, Temple Lang D (2008). "Back to the Future: Lisp as a Base for a Statistical Computing System." In P Brito (ed.), COMPSTAT 2008 - Proceedings in Computational Statistics, pp. 21-33. Physica-Verlag, Heidelberg.
16. Kane MJ (2013). synchronicity: Boost Mutex Functionality for R. R package version 1.1.2, URL http://CRAN.R-project.org/package=synchronicity.
17. Kane MJ, Emerson JW (2013a). The Preprocessed Airline On-Time Performance Data, URL http://euler.stat.yale.edu/~mjk56/Airline.tar.bz2.
18. Kane MJ, Emerson JW (2013b). Bigtabulate: table-, tapply-, and split-like Functionality for matrix and big.matrix Objects. R package version 1.1.2, URL http://CRAN.R-project.org/package=bigtabulate.
19. Kane MJ, Emerson JW, Haverty P (2013a). Bigmemory: Manage Massive Matrices with Shared Memory and Memory-Mapped Files. R package version 4.4.5, URL http://CRAN.R-project.org/package=bigmemory.
20. Kane MJ, Lewis BW, Emerson JW (2013b). Bigalgebra: BLAS and LAPACK Routines for Native R Matrices and big.matrix objects. R package version 0.8.2, URL https://R-Forge.R-project.org/R/?group_id=556.
21. Kath R (1993). Managing Memory-Mapped Files, URL http://msdn.microsoft.com/en-us/library/ms810613.aspx.
22. Lewis BW (2012). Doredis: Foreach Parallel Adapter for the rredis Package. R package version 1.0.5, URL https://github.com/bwlewis/doRedis.
23. Lumley T (2013). Biglm: Bounded Memory Linear and Generalized Linear Models. R package version 0.9-1, URL http://CRAN.R-project.org/package=biglm.
24. Miller AJ (1992). "Algorithm AS 274: Least Squares Routines to Supplement Those of Gentlemen." Journal of the Royal Statistical Society C, 41(2), 458-478.
25. R Core Team (2013a). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.
26. R Core Team (2013b). "R Installation and Administration." URL http://CRAN.R-project.org/doc/manuals/R-admin.html.
27. Revolution Analytics (2013a). Domc: Foreach Parallel Adaptor for the multicore Package. R package version 1.3.0, URL http://CRAN.R-project.org/package=doMC.
28. Revolution Analytics (2013b). Dosnow: Foreach Parallel Adaptor for the snow Package. R package version 1.0.7, URL http://CRAN.R-project.org/package=doSNOW.
29. Revolution Analytics, Weston S (2013a). Doparallel: Foreach Parallel Adaptor for the parallel Package. R package version 1.0.3, URL http://CRAN.R-project.org/package=doParallel.
30. Revolution Analytics, Weston S (2013b). Foreach: Foreach Looping Construct for R. R package version 1.4.1, URL http://CRAN.R-project.org/package=foreach.
31. RITA (2009). "The Airline On-Time Performance Data Set Website." Research and Innovation Technology Administration, Bureau of Transportation Statistics, URL http://stat-computing.org/dataexpo/2009/.
32. SAS Institute Inc (2011). The SAS System, Version 9.3. SAS Institute Inc., Cary, NC. URL http://www.sas.com/.
33. SQLite Development Team (2013). "SQLite Database Engine." URL http://www.sqlite.org/.
34. The Apache Software Foundation (2013). "Apache Hadoop." URL http://hadoop.apache.org/.
35. The MathWorks, Inc (2011). MATLAB - The Language of Technical Computing, Version R2011b. The MathWorks, Inc., Natick, Massachusetts. URL http://www.mathworks.com/products/matlab/.
36. Tierney L, Rossini AJ, Li N, Sevcikova H (2013). Snow: Simple Network of Workstations. R package version 0.3-13, URL http://CRAN.R-project.org/package=snow.
37. Urbanek S (2011). Multicore: Parallel Processing of R Code on Machines with Multiple Cores or CPUs. R package version 0.1-7, URL http://CRAN.R-project.org/package=multicore.
38. Weston S (2013). Dompi: Foreach Parallel Adaptor for the Rmpi Package. R package version 0.2, URL http://CRAN.R-project.org/package=doMPI.
39. Wickham H (2011). "The Split-Apply-Combine Strategy for Data Analysis." Journal of Statistical Software, 40(1), 1-29. URL http://www.jstatsoft.org/v40/i01/.
40. Yu H (2002). "Rmpi: Parallel Statistical Computing in R." R News, 2(2), 10-14. URL http://cran.r-project.org/doc/Rnews/Rnews_2002-2.pdf.
41. Yu H (2013). "The Rmpi R package."R package version 0.6-3, URL http://CRAN.R-project.org/package=Rmpi.