Batchjobs and batchexperiments: Abstraction mechanisms for using R in batch environments

Journal of Statistical Software

Volumn 64, Issue 11, 2015, Pages 1-25

  Bischl, Bernd ^a   Lang, Michel ^a   Mersmann, Olaf ^a   Rahnenführer, Jörg ^a   Weihs, Claus ^a  

^a TU DORTMUND UNIVERSITY   (Germany)

Author keywords

Batch computing; High performance computing; R; Reproducibility

Indexed keywords

EID: 84925345468     PISSN: 15487660     EISSN: None     Source Type: Journal    
DOI: 10.18637/jss.v064.i11     Document Type: Article

References (40)

1. Adaptive Computing Inc (2014). Terascale Open-Source Resource and QUEue Man- ager (TORQUE). URL http://www.adaptivecomputing.com/products/open-source/ torque/.
2. Amazon Inc (2015). Amazon Elastic Compute Cloud (EC2). Seattle. URL http://aws.amazon.com/en/ec2/.
3. Anderson E (1935). “The Irises of the Gasp_e Peninsula." Bulletin of the American Iris Society, 59, 2-5.
4. Apache Software Foundation (2014). Apache Hadoop. URL http://hadoop.apache.org/.
5. Bischl B, Lang M, Bengtsson H (2015a). BatchJobs: Batch Computing with R. R package version 1.6, URL http://CRAN.R-project.org/package=BatchJobs.
6. Bischl B, Lang M, Mersmann O (2015b). BatchExperiments: Statistical Experiments on Batch Computing Clusters. R package version 1.4.1, URL http://CRAN.R-project.org/package=BatchExperiments.
7. Bui P, Yu L, Thrasher A, Carmichael R, Lanc I, Donnelly P, Thain D (2011). “Scripting Distributed Scienti_c Workows Using Weaver." Concurrency and Computation: Practice and Experience, 24(15), 1685-1707.
8. Burger M, Juenemann K, Koenig T (2015). RUnit: R Unit Test Framework. R package version 0.4.28, URL http://CRAN.R-project.org/package=RUnit.
9. Cleveland W, Guha S, Hafen R, Li J, Rounds J, Xi B, Xia J (2011). “Divide and Recombine for the Analysis of Complex Big Data." Technical Report 2, Department of Statistics, Purdue University.
10. Dean J, Ghemawat S (2008). “MapReduce: Simpli_ed Data Processing on Large Clusters." Communications of the ACM, 51, 107-113.
11. Eddelbuettel D (2015). CRAN Task View: High-Performance and Parallel Com- puting with R. Version 2015-01-15, URL http://CRAN.R-project.org/view=HighPerformanceComputing.
12. Gentzsch W (2001). “Sun Grid Engine: Towards Creating a Compute Power Grid." In First IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 35-36.
13. Gromping U (2014). CRAN Task View: Design of Experiments (DoE) & Analysis of Experimental Data. Version 2014-12-07, URL http://CRAN.R-project.org/view=ExperimentalDesign.
14. Hill J, Hambley M, Forster T, Mewissen M, Sloan T, Scharinger F, Trew A, Ghazal P (2008). “SPRINT: A New Parallel Framework for R." BMC Bioinformatics, 9(558).
15. Hipp DR (2015). SQLite. URL http://www.sqlite.org/.
16. Homann TJ (2011). “Passing in Command Line Arguments and Parallel Cluster/Multicore Batching in R with batch." Journal of Statistical Software, Code Snippets, 39(1), 1-11. URL http://www.jstatsoft.org/v39/c01/.
17. Horner J (2011). brew: Templating Framework for Report Generation. R package version 1.0- 6, URL http://CRAN.R-project.org/package=brew.
18. Hothorn T, Leisch F (2011). “Case Studies in Reproducibility." Brie_ngs in Bioinformatics, 12(3), 288-300.
19. Kane M, Emerson JW, Weston S (2013). “Scalable Strategies for Computing with Massive Data." Journal of Statistical Software, 55(14), 1-19. URL http://www.jstatsoft.org/v55/i14/.
20. Knaus J (2013). snowfall: Easier Cluster Computing (Based on snow). R package version 1.84-6, URL http://CRAN.R-project.org/package=snowfall.
21. Leisch F, Eugster M, Hothorn T (2011). “Executable Papers for the R Community: The R2 Platform for Reproducible Research." Procedia Computer Science, 4, 618-626.
22. Liaw A, Wiener M (2002). “Classi_cation and Regression by randomForest." R News, 2(3), 18-22. URL http://CRAN.R-project.org/doc/Rnews/.
23. Mersmann O (2014). sendmailR: Send Email Using R. R package version 1.2-1, URL http://CRAN.R-project.org/package=sendmailR.
24. Mersmann O, Bischl B (2012). soobench: Single Objective Optimization Benchmark Functions. R package version 1.0-73, URL http://CRAN.R-project.org/package=soobench.
25. Platform Computing (2012). Load Sharing Facility (LSF). URL http://www.platform.com/.
26. R Core Team (2014). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.
27. Revolution Analytics (2014). foreach: Foreach Looping Construct for R. R package version 1.4.2, URL http://CRAN.R-project.org/package=foreach.
28. Revolution Analytics (2010). nws: R Functions for NetWorkSpaces and Sleigh. R package version 1.7.0.1, URL http://CRAN.R-project.org/package=nws.
29. Schmidberger M, Morgan M, Eddelbuettel D, Yu H, Tierney L, Mansmann U (2009). “State of the Art in Parallel Computing with R." Journal of Statistical Software, 31(1), 1-27. URL http://www.jstatsoft.org/v31/i01/.
30. Therneau TM, Atkinson EJ (1997). “An Introduction to Recursive Partitioning Using the rpart Routine." Technical Report 61, Section of Biostatistics, Mayo Clinic, Rochester. URL http://www.mayo.edu/hsr/techrpt/61.pdf.
31. Tierney L, Rossini AJ, Li N, Sevcikova H (2013). snow: Simple Network of Workstations. R package version 0.3-13, URL http://CRAN.R-project.org/package=snow.
32. Urbanek S (2014). multicore: Parallel Processing of R Code on Machines with Multiple Cores or CPUs. R package version 0.2, URL http://CRAN.R-project.org/package=multicore.
33. Vanschoren J, Blockeel H, Pfahringer B, Holmes G (2012). “Experiment Databases. A New Way to Share, Organize and Learn from Experiments." Machine Learning, 87(2), 127-158.
34. Wickham H (2011a). “testthat: Get Started with Testing." The R Journal, 3(1), 5-10.
35. Wickham H (2011b). “The Split-Apply-Combine Strategy for Data Analysis." Journal of Statistical Software, 40(1), 1-29. URL http://www.jstatsoft.org/v40/i01/.
36. Wickham H (2014). testthat: Testthat Code. Tools to Make Testing Fun. R package version 0.9.1, URL http://CRAN.R-project.org/package=testthat.
37. Wickham H, James DA, Falcon S (2014). RSQLite: SQLite Interface for R. R package version 1.0.0, URL http://CRAN.R-project.org/package=RSQLite.
38. Yoo A, Jette M, Grondona M (2003). “SLURM: Simple Linux Utility for Resource Management." In D Feitelson, L Rudolph, U Schwiegelshohn (eds.), Job Scheduling Strategies for Parallel Processing, volume 2862 of Lecture Notes in Computer Science, pp. 44-60. Springer-Verlag.
39. Yu H (2002). “Rmpi: Parallel Statistical Computing in R." R News, 2(2), 10-14.
40. Yu H (2014). Rmpi: Interface (Wrapper) to MPI (Message-Passing Interface). R package version 0.6-5, URL http://CRAN.R-project.org/package=Rmpi.