RIoTBench: An IoT benchmark for distributed stream processing systems

Concurrency and Computation: Practice and Experience

Volumn 29, Issue 21, 2017, Pages

  Shukla, Anshu ^a   Chaturvedi, Shilpa ^a   Simmhan, Yogesh ^a  

^a INDIAN INSTITUTE OF SCIENCE   (India)

Author keywords

benchmark; big data applications; dataflows; distributed stream processing; Internet of Things; performance evaluation

Indexed keywords

BENCHMARKING; DATA STREAMS; DECISION MAKING; DISTRIBUTED PARAMETER CONTROL SYSTEMS; ENVIRONMENTAL TECHNOLOGY; PREDICTIVE ANALYTICS; REAL TIME SYSTEMS; WINDOWS OPERATING SYSTEM;

BIG DATA APPLICATIONS; DATA FLOW; DISTRIBUTED STREAM PROCESSING; EMERGING TECHNOLOGIES; INTERNET OF THING (IOT); PERFORMANCE EVALUATION; REAL-TIME DATA PROCESSING; SENSORS AND ACTUATORS;

INTERNET OF THINGS;

EID: 85030641647     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.4257     Document Type: Article

References (58)

1. Zaslavsky A. Internet of Things and ubiquitous sensing. IEEE Computing Now. September, 2013.
2. Chandrasekaran S, Cooper O, Deshpande A, et al. Telegraphcq: continuous dataflow processing for an uncertain world. In: Conference on Innovative Data Systems Research (CIDR), Asilomar, CA, USA; 2003:668-668.
3. Cerwall P. Ericsson mobility technical report, Ericsson, Stockholm, Sweden; 2016. https://www.ericsson.com/assets/local/mobility-report/documents/2016/ericsson-mobility-report-november-2016.pdf. Accessed June 23, 2017.
4. Canvas D. Sense your city: data art challenge. 2015. https://www.postscapes.com/visualizing-urban-data-sense-your-city-data-art-challenge/. Accessed June 6, 2017.
5. Simmhan Y, Aman S, Kumbhare A, et al. Cloud-based software platform for data-driven smart grid management. IEEE/AIP Comput Sci Eng. 2013;July/August 1-11;15(4):38-47.
6. Wolf G. The data-driven life. 2010. http://www.nytimes.com/2010/05/02/magazine/02self-measurement-t.html. Accessed June 23, 2017.
7. Perera C, Zaslavsky A, Christen P, Georgakopoulos D. Context aware computing for the Internet of Things: a survey. Commun Surv Tut. 2014;16(1):414-454.
8. Stonebraker M, Cetintemel U, Zdonik S. The 8 requirements of real-time stream processing. ACM SIGMOD Record. 2005;34(4):42-47.
9. Toshniwal A, Taneja S, Shukla A, et al. Storm@ twitter. In: ACM International Conference on Management of Data (SIGMOD), Snowbird, UT, USA; 2014:147-156.
10. Zaharia M, Das T, Li H, Shenker S, Stoica I. Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters. In: USENIX Hot Cloud, Boston, MA, USA; 2012:1-6.
11. Carbone P, Ewen S, Haridi S, et al. Apache flink: stream and batch processing in a single engine. IEEE Data Eng Bull. 2015;38(4):28-38.
12. Amazon. AWS for IoT. 2017. https://aws.amazon.com/iot/how-it-works/. Accessed June 23, 2017.
13. Microsoft. Azure IoT. 2017. https://www.microsoft.com/en-in/server-cloud/internet-of-things/overview.aspx. Accessed June 23, 2017.
14. Çetintemel U, Abadi D, Ahmad Y, et al. Data Stream Management: Processing High-Speed Data Streams. Berlin, Heidelberg: Springer Berlin Heidelberg; 2016:337-359.
15. Arasu A, Cherniack M, Galvez E, et al. Linear road: a stream data management benchmark. In: International Conference on Very Large Databases (VLDB), Toronto, Canada; 2004:480-491.
16. Simmhan Y, Hedge M, Kumar MM, Amrutur B, Sundaresan R. IISc Smart Campus Project. 2017. https://smartx.cds.iisc.ac.in. Accessed June 23, 2017
17. Shukla A, Simmhan Y. Benchmarking distributed stream processing platforms for IoT applications. In: Technology Conference on Performance Evaluation & Benchmarking (TPCTC), New Delhi, India; 2016:90-106.
18. Carney D, Çetintemel U, Cherniack M, et al. Monitoring streams: a new class of data management applications. In: International Conference on Very Large Databases (VLDB), Hong Kong, China; 2002:215-226.
19. Chen J, DeWitt DJ, Tian F, Wang Y. Niagaracq: a scalable continuous query system for internet databases. ACM SIGMOD Record. 2000;29(2):379-390. https://doi.org/10.1145/335191.335432.
20. Babu S, Widom J. Continuous queries over data streams. ACM SIGMOD Record. September 2001;30(3):109-120. https://doi.org/10.1145/603867.603884.
21. Balazinska M, Balakrishnan H, Madden SR, Stonebraker M. Fault-tolerance in the borealis distributed stream processing system. ACM Trans on Database Syst (TODS). 2008;33(1):3:1-3:44. https://doi.org/10.1145/1331904.1331907.
22. Biem A, Bouillet E, Feng H, et al. IBM infosphere streams for scalable, real-time, intelligent transportation services. In: ACM International Conference on Management of Data (SIGMOD), Indianapolis, IN, USA; 2010:1093-1104.
23. Suhothayan S, Gajasinghe K, Loku Narangoda I, et al. Siddhi: a second look at complex event processing architectures. In: ACM Workshop on Gateway Computing Environments, Seattle, WA, USA; 2011:43-50.
24. Cugola G, Margara A. Processing flows of information: from data stream to complex event processing. ACM Comput Surv (CSUR). 2012;44(3):15:1-15:62. https://doi.org/10.1145/2187671.2187677.
25. Neumeyer L, Robbins B, Nair A, Kesari A. S4: distributed stream computing platform. In: IEEE International Conference on Data Mining Workshops (ICDMW), Sydney, Australia; 2010:170-177.
26. Pallickara S, Ekanayake J, Fox G. Granules: a lightweight, streaming runtime for cloud computing with support, for map-reduce. In: IEEE International Conference on Cluster Computing and Workshops (CLUSTERW). IEEE, New Orleans, LA, USA; 2009:1-10.
27. Apache trident. 2015. https://storm.apache.org/releases/1.0.1/Trident-tutorial.html. Accessed June 23, 2017.
28. Gulisano V, Jimenez-Peris R, Patino-Martinez M, Soriente C, Valduriez P. Streamcloud: an elastic and scalable data streaming system. IEEE Trans Parallel Distrib Syst. 2012;23(12):2351-2365.
29. Kumbhare AG, Simmhan Y, Frincu M, Prasanna VK. Reactive resource provisioning heuristics for dynamic dataflows on cloud infrastructure. IEEE Trans on Cloud Comput. 2015;3(2):105-118.
30. Nabi Z, Bouillet E, Bainbridge A, Thomas C. Of streams and storms. Technical Report, IBM, Dublin, Ireland; 2014.
31. Agrawal D, Butt A, Doshi K, et al. Sparkbench—a spark performance testing suite. In: Technology Conference on Performance Evaluation and Benchmarking (TPCTC). Springer, New Delhi, India; 2015:26-44.
32. Li C, Berry R. Cepben: a benchmark for complex event processing systems. In: Technology Conference on Performance Evaluation and Benchmarking (TPCTC) Springer, Trento, Italy; 2013:125-142.
33. Lu R, Wu G, Xie B, Hu J. Stream bench: towards benchmarking modern distributed stream computing frameworks. In: IEEE/ACM International Conference On Utility And Cloud Computing (UCC), London, UK; 2014:69-78.
34. Joshi A, Nambiar R, Brey M. Benchmarking Internet of Things solutions. In: Workshop on Big Data Benchmarks, Springer, Potsdam, Germany; 2014:29-36.
35. Huang S, Huang J, Dai J, Xie T, Huang B. The hibench benchmark suite: characterization of the mapreduce-based data analysis. In: IEEE International Conference on Data Engineering Workshops (ICDEW), Long Beach, CA, USA; 2010:41-51.
36. Gao W, Zhu Y, Jia Z, et al. Bigdatabench: a big data benchmark suite from internet services. IEEE International Symposium on High Performance Computer Architecture (HPCA), Orlando, FL; 2014:488-499.
37. Ghazal A, Rabl T, Hu M, et al. Bigbench: towards an industry standard benchmark for big data analytics. In: ACM International Conference on Management of Data (SIGMOD); New York, NY, USA; 2013:1197-1208.
38. Nambiar RO, Poess M. The making of TPC-DS. In: International Conference on Very Large Databases (VLDB), Seoul, Korea; 2006:1049-1058.
39. Gu L, Zhou M, Zhang Z, et al. Chronos: an elastic parallel framework for stream benchmark generation and simulation. In: IEEE International Conference on Data Engineering (ICDE), Seoul, South Korea; 2015:101-112.
40. Arlitt M, Marwah M, Bellala G. Iotabench: an Internet of Things analytics benchmark. In: ACM/SPEC International Conference on Performance Engineering (ICPE). New York, NY, USA: ACM; 2015: 133-144. https://doi.org/10.1145/2668930.2688055.
41. Ali MI, Gao F, Mileo A. Citybench: a configurable benchmark to evaluate RSP engines using smart city datasets. In: International Semantic Web Conference (ISWC), Bethlehem, PA, USA; 2015:374-389.
42. Wallis M, Wilson B. Iot-connect: an industry-standard benchmarks for embedded systems. Technical Report, EEMBC, El Dorado Hills, CA, USA; 2017.
43. Varshney P, Simmhan Y. Demystifying fog computing: characterizing architectures, applications and abstractions. In: IEEE International Conference on Fog and Edge Computing, Madrid, Spain; 2017:1-10.
44. Apache spark programming guide. http://spark.apache.org/docs/latest/programming-guide.html. Accessed June 23, 2017.
45. Jennings C, Shelby Z, Arkko J, Keranen A, Bormann C. Media types for sensor measurement lists (senml). Technical Report draft-ietf-core-senml-04, Internet Engineering Task Force (IETF), Fremont, CA, USA; 2016.
46. Jazayeri MA, Liang SH, Huang CY. Implementation and evaluation of four interoperable open standards for the internet of things. Sensors. 2015;15(9):24343-24373.
47. Liang S, Huang ACY, Khalafbeigi T, et al. Ogc sensorthings api part 1: sensing. Technical Report, Open Geospatial Consortium, Wayland, MA, USA; 2016.
48. Zhou Q, Simmhan Y, Prasanna V. Knowledge-infused and consistent complex event processing over real-time and persistent streams. Future Gener Comp Syst(FGCS). 2016. https://doi.org/10.1016/j.future.2016.10.030.
49. Bloom BH. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM. 1970;13(7):422-426.
50. Durand M, Flajolet P. Loglog counting of large cardinalities. In: European Symposium on Algorithms, Springer, Budapest, Hungary; 2003:605-617.
51. Kalman RE. A new approach to linear filtering and prediction problems. ASME J Basic Eng. 1960;82(Series D):35-45.
52. Alon N, Matias Y, Szegedy M. The space complexity of approximating the frequency moments. In: Acm Symposium on Theory of Computing. ACM, Philadelphia, PA, USA; 1996:20-29.
53. Quinlan J. Induction of decision trees. Machine Learning. 1986;1(1):81-106.
54. Xchart java library. April 2015. http://knowm.org/open-source/xchart/. Accessed: June 23, 2017;
55. Banos O, Garcia R, Holgado-Terriza JA, et al. mhealthdroid: a novel framework for agile development of mobile health applications. In: International Workshop on Ambient Assisted Living, Springer, Belfast, UK; 2014:91-98. https://archive.ics.uci.edu/ml/datasets/MHEALTH+Dataset.
56. The Research Perspective Ltd. Cer smart metering project. Technical Report, Dublin, Ireland, Commission for Energy Regulation. 2012. https://www.ucd.ie/issda/data/commissionforenergyregulationcer. Accessed June 23, 2017.
57. Donovan B, Work DB. Using coarse GPS data to quantify city-scale transportation system resilience to extreme events. In: Transportation Research Board, Washington, DC, USA; 2014, Number 15-5465.
58. 535 B. Uber foil dataset. 2016. https://github.com/fivethirtyeight/uber-tlc-foil-response. Accessed June 23, 2017.