Developing systems for real-time streaming analysis

Journal of Computational and Graphical Statistics

Volumn 21, Issue 3, 2012, Pages 561-580

  Michalak, Sarah ^a   Dubois, Andrew ^a   Dubois, David ^a   Wiel, Scott Vander ^a   Hogden, John ^a  

^a LOS ALAMOS NATIONAL LABORATORY   (United States)

Author keywords

Massive datasets; Radio astronomy; Real time methods; Streaming data

Indexed keywords

EID: 84865387869     PISSN: 10618600     EISSN: None     Source Type: Journal    
DOI: 10.1080/10618600.2012.657144     Document Type: Article

References (49)

1. Asadi, N., Meng, T., and Wong, W. (2008), "Reconfigurable Computing for Learning Bayesian Networks," in Proceedings of the 16th International ACM/SIGDA Symposium on Field Programmable Gate Arrays, pp. 203-211.[563]
2. Babcock, B., Datar, M., and Motwani, R. (2003), "Load Shedding Techniques for Data Stream Systems," in Proceedings of the 2003 Workshop on Management and Processing of Data Streams.[577]
3. Babcock, B., Datar, M., Motwani, R., and O'Callaghan, L. (2003), "Maintaining Variance and K-Medians Over Data Stream Windows," in Proceedings of the 22nd ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 234-243.[563]
4. Bigelow, D., Brandt, S., Bent, J., and Chen, H. (2010), "Mahanaxar: Quality of Service Guarantees in High- Bandwidth, Real-Time Streaming Data Storage," in Proceedings of the 26th IEEE Symposium on Massive Storage Systems and Technologies (MSST2010), pp. 123-133.[572]
5. Bower, G. (2007), "Mining for the Ephemeral," Science, 318, 759.[561]
6. Brockwell, A. (2006), "Parallel Markov Chain Monte Carlo Simulation by Pre-Fetching," Journal of Computational and Graphical Statistics, 15, 246-261.[563]
7. Buckner, J., Wilson, J., Seligman, M., Athey, B., Watson, S., and Meng, F. (2010), "The gputools Package Enables GPU Computing in R," Bioinformatics, 26, 134.[563]
8. Carr, S., and Kennedy, K. (1994), "Improving the Ratio of Memory Operations to Floating-Point Operations in Loops," ACM Transactions on Programming Languages and Systems (TOPLAS), 16, 1768-1810.[573]
9. Caudle, K., and Wegman, E. (2009), "Nonparametric Density Estimation of Streaming Data Using Orthogonal Series," Computational Statistics and Data Analysis, 53, 3980-3986.[563, 565]
10. Chambers, J. M., James, D., Lambert, D., and Wiel, S. (2006), "Monitoring Networked Applications With Incremental Quantile Estimation," Statistical Science, 21, 463-475.[563]
11. Chambers, J. M. (1998), Programming With Data: A Guide to the S Language, New York: Springer-Verlag.[565]
12. Chen, F., Lambert, D., and Pinheiro, J. (2000), "Incremental Quantile Estimation for Massive Tracking," in Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 516-522.[563]
13. Chen, Y., Dong, G., Han, J., Wah, B., and Wang, J. (2002) "Multi-Dimensional Regression Analysis of Time-Series Data Streams," in Proceedings of the 28th International Conference on Very Large Data Bases.[563]
14. Chi, Y., Wang, H., and Yu, P. (2005), "Loadstar: Load Shedding in Data Stream Mining," in Proceedings of the 31st International Conference on Very Large Data Bases, pp. 1302-1305.[577]
15. Cleveland, W., and Sun, D. (2000), "Internet Traffic Data," Journal of the American Statistical Association, 95, 979-985.[565]
16. Cormode, G., and Muthukrishnan, S. (2005), "What's New: Finding Significant Differences in Network Data Streams," IEEE/ACM Transactions on Networking (TON), 13, 1219-1232.[563]
17. Cortes, C., Fisher, K., Pregibon, D., and Rogers, A. (2000), "Hancock:ALanguage for Extracting Signatures From Data Streams," in Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 9-17.[563]
18. Ding, C., and Kennedy, K. (1999), "Improving Cache Performance in Dynamic Applications Through Data and Computation Reorganization at Run Time," ACM SIGPLAN Notices, 34, 229-241.[573]
19. DuBois, A., Connor, C., Michalak, S., Taylor, G., and DuBois, D. (2009), "Application of the IBM Cell Processor to Real-Time Cross-Correlation of a Large Antenna Array Radio Telescope," Technical Report no. LA-UR- 09-03483, Los Alamos National Laboratory.[563]
20. DuBois, D., DuBois, A., Boorman, T., Connor, C., and Poole, S. (2010), "Sparse Matrix-Vector Multiplication on a Reconfigurable Supercomputer With Application," ACM Transactions on Reconfigurable Technology and Systems (TRETS), 3(1), Article 2.[573]
21. Fahmy, S., Cheung, P., and Luk, W. (2005), "Hardware Acceleration of Hidden Markov Model Decoding for Person Detection," in Proceedings of the Design, Automation and Test in Europe Conference and Exhibition (DATE'05), pp. 8-13. [563]
22. Gropp, W., Lusk, E., and Skjellum, A. (1999), Using MPI: Portable Parallel Programming With the Message Passing Interface, Cambridge, MA: MIT Press.[573]
23. Huber, P. (1964), "Robust Estimation of a Location Parameter," The Annals of Mathematical Statistics, 35, 73-101.[566, 569]
24. Hulten, G., and Domingos, P. (2003), "VFML -A Toolkit for Mining High-Speed Time-Changing Data Streams," Available at http://www.cs.washington.edu/dm/vfml/ [563]
25. IBM (2008), "Cell Broadband Engine Programming Handbook Including the PowerXCell 8i," Available at https://www-01.ibm.com/chips/techlib/techlib.nsf/techdocs/7A77CCDF14FE70D5852575CA0074E8ED [562]
26. Kemball, A., Cordes, J., Lazio, J., Backer, D., Bower, G., Bhatnagar, S., and Plante, R. (2009), "Technology Development for Computational Radio Astronomy: 2010-2020," Technical Report, Astro2010: The Astronomy and Astrophysics Decadal Survey, Technology Development Papers, no. 46. Available at http://www8.nationalacademies.org/astro2010/DetailFileDisplay.aspx?id=548 [563]
27. Lee, A., Yau, C., Giles, M., Doucet, A., and Holmes, C. (2009), "On the Utility of Graphics Cards to Perform Massively Parallel Simulation of Advanced Monte Carlo Methods," Available at http://arxiv.org/abs/0905.2441 [563]
28. Lee, V., Kim, C., Chhugani, J., Deisher, M., Kim, D., Nguyen, A., Satish, N., Smelyanskiy, M., Chennupaty, S., Hammarlund, P., Singhal, R., and Dubey, P. (2010), "Debunking the 100X GPU vs. CPU Myth: An Evaluation of Throughput Computing on CPU and GPU," in Proceedings of the 37th Annual International Symposium on Computer Architecture, pp. 451-460. [574]
29. Lorimer, D. (2001), "SIGPROC - Pulsar Signal Processing Programs," Technical Report, Arecibo Technical Memo 2001-01; Arecibo: NAIC. Available at http://www.jb.man.ac.uk/drl/sigproc [569]
30. Meuer, H. (2008), "31st TOP500 List Topped by First-Ever Petaflop/s Supercomputer," Scientific Computing [online]. Available at http://www.scientificcomputing.com/31st-TOP500-List-Topped-by-First-ever-Petaflops-Supercomputer.aspx[562]
31. Microsoft (2010), "Microsoft Developers Network Library: MMX, SSE, and SSE2 Intrinsics," Available at http://msdn.microsoft.com/en-us/library/y0dh78ez(v=VS.71).aspx [573]
32. Pharr, M., and Fernando, R. (2005), GPU Gems 2: Programming Techniques for High-Performance Graphics and General-Purpose Computation, Taunton, MA: Addison-Wesley Professional.[574]
33. Prost, J., Treumann, R., Hedges, R., Jia, B., and Koniges, A. (2001), "MPI-IO/GPFS, an Optimized Implementation of MPI-IO on Top of GPFS," in Proceedings of the 2001 ACM/IEEE Conference on Supercomputing.[572]
34. R Development Core Team (2008), R: A Language and Environment for Statistical Computing, Vienna: R Foundation for Statistical Computing.[565]
35. Romein, J., Broekema, P., van Meijeren, E., van der Schaaf, K., and Zwart, W. (2006), "Astronomical Real-Time Streaming Signal Processing on a Blue Gene/L Supercomputer," in Proceedings of the Eighteenth Annual ACM Symposium on Parallelism in Algorithms and Architectures, pp. 59-66.[563]
36. Siemion, A., Von Korff, J., McMahon, P., Korpela, E., Werthimer, D., Anderson, D., Bower, G., Cobb, J., Foster, G., Lebofsky, M., Van Leeuwen, J., and Wagner, M. (2010), "New SETI Sky Surveys for Radio Pulses," Acta Astronautica, 67, 1342-1349.[564]
37. Silberstein, M., Schuster, A., Geiger, D., Patney, A., and Owens, J. (2008), "Efficient Computation of Sum- Products on GPUs Through Software-Managed Cache," in Proceedings of the 22nd Annual International Conference on Supercomputing, pp. 309-318.[563]
38. Stamatakis, A., and Ott, M. (2008), "Exploiting Fine-Grained Parallelism in the Phylogenetic Likelihood Function With MPI, Pthreads, and OpenMP: A Performance Study," Pattern Recognition in Bioinformatics, 424-435.[563, 573]
39. Stankovic, J., and Rajkumar, R. (2004), "Real-Time Operating Systems," Real-Time Systems, 28, 237-253.[571]
40. Suchard, M., and Rambaut, A. (2009), "Many-Core Algorithms for Statistical Phylogenetics," Bioinformatics, 25, 1370.[563]
41. Suchard, M., Wang, Q., Chan, C., Frelinger, J., Cron, A., andWest, M. (2010), "Understanding GPU Programming for Statistical Computation: Studies in Massively Parallel Massive Mixtures," Journal of Computational and Graphical Statistics, 19, 419-438.[563]
42. Tatbul, N., Çetintemel, U., Zdonik, S., Cherniack, M., and Stonebraker, M. (2003), "Load Shedding on Data Streams," in Proceedings of the ACM Workshop on Management and Processing of Data Streams (MPDS 03).[577]
43. Tibbits, M., Haran, M., and Liechty, J. (2011), "Parallel Multivariate Slice Sampling," Statistics and Computing, 21, 415-430.[563]
44. Tukey, J.W. (1962), "The Future of Data Analysis," The Annals of Mathematical Statistics, 33, 1-67.[566]
45. VanNieuwpoort, R., and Romein, J. (2009), "UsingMany-Core Hardware to Correlate RadioAstronomy Signals," in Proceedings of the 23rd International Conference on Supercomputing, pp. 440-449.[563]
46. Vuduc, R., Chandramowlishwaran, A., Choi, J., Guney, M., and Shringarpure, A. (2010), "On the Limits of GPU Acceleration," Available at http://www.usenix.org/event/hotpar10/tech/full_papers/main.pdf [574]
47. Welch, J., Backer, D., Blitz, L., Bock, D., Bower, G., Cheng, C., Croft, S., Dexter, M., Engargiola, G., Fields, E., et al. (2009), "The Allen Telescope Array: The First Widefield, Panchromatic, Snapshot Radio Camera for Radio Astronomy and SETI," Proceedings of the IEEE, 97(8), 1438-1447.[562]
48. Zhou, H., Lange, K., and Suchard, M. (2010), "Graphics Processing Units and High-Dimensional Optimization," Available at http://arxiv.org/pdf/1003.3272.pdf[563]
49. Zhu, Y., and Shasha, D. (2002), "StatStream: Statistical Monitoring of Thousands of Data Streams in Real Time," in Proceedings of the 28th International Conference on Very Large Data Bases, p. 369.[563]