Quantity is nothing without quality: Automated QA/QC for streaming environmental sensor data

BioScience

Volumn 63, Issue 7, 2013, Pages 574-585

  Campbell, John L ^a   Rustad, Lindsey E ^a   Porter, John H ^b   Taylor, Jeffrey R ^c   Dereszynski, Ethan W ^d   Shanley, James B ^e   Gries, Corinna ^f   Henshaw, Donald L ^g   Martin, Mary E ^h   Sheldon, Wade M ⁱ   Boose, Emery R ^j  

^a USDA FOREST SERVICE   (United States)

^b UNIVERSITY OF VIRGINIA   (United States)

^c NATIONAL ECOLOGICAL OBSERVATORY NETWORK   (United States)

^d Oregon State University   (United States)

^e U S GEOLOGICAL SURVEY   (United States)

^f UNIVERSITY OF WISCONSIN MADISON   (United States)

^g PACIFIC NORTHWEST RESEARCH STATION   (United States)

^h UNIVERSITY OF NEW HAMPSHIRE   (United States)

ⁱ UNIVERSITY OF GEORGIA   (United States)

^j HARVARD UNIVERSITY   (United States)

more..

Author keywords

computers in biology; environmental science; informatics; instrumentation

Indexed keywords

EID: 84881496397     PISSN: 00063568     EISSN: None     Source Type: Journal    
DOI: 10.1525/bio.2013.63.7.10     Document Type: Article

References (46)

1. Altintas I, Barney O, Jaeger-Frank E. 2006. Provenance collection support in the Kepler scientific workflow system. Pages 118-132 in Moreau L, Foster I, eds. Proceedings of the International Provenance and Annotation Workshop. Lecture Notes in Computer Science, vol. 4145. Springer. (Pubitemid 44577314)
2. Barseghian D, et al. 2010. Workflows and extensions to the Kepler scientific workflow system to support environmental sensor data access and analysis. Ecological Informatics 5: 42-50.
3. Belhajjame K, Wolstencroft K, Corcho O, Oinn T, Tanoh F, William A, Goble C. 2008. Metadata management in the Taverna workflow system. Pages 651-656 in Priol T, Lefevre L, Buyya R, eds. CCGRID 2008: Eighth IEEE International Symposium on Cluster Computing and the Grid. Institute of Electrical and Electronics Engineers.
4. Benson BJ, Bond BJ, Hamilton MP, Monson RK, Han R. 2010. Perspectives on next-generation technology for environmental sensor networks. Frontiers in Ecology and the Environment 8: 193-200.
5. Collins SL, et al. 2006. New opportunities in ecological sensing using wireless sensor networks. Frontiers in Ecology and the Environment 4: 402-407. [COSEPUP] Committee on Science, Engineering, and Public Policy. 2009. (Pubitemid 44520736)
6. Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age. National Academies Press.
7. Daly C, Redmond K, Gibson W, Doggett M, Smith J, Taylor G, Pasteris P, Johnson G. 2005. Opportunities for improvements in the quality control of climate observations. Paper presented at the 15th American Meteorological Society Conference on Applied Climatology; 20-23 June 2005, Savannah, Georgia. (7 May 2013; https://ams.confex.com/ams/15AppClimate/techprogram/paper-94199. htm)
8. Dereszynski EW, Dietterich TG. 2011. Spatiotemporal models for dataanomaly detection in dynamic environmental monitoring campaigns. ACM Transactions on Sensor Networks 8 (art. 3). doi:10.1145/1993042.1993045
9. Diamond HJ, et al. 2013. U.S. Climate Reference Network after one decade of operations: Status and assessment. Bulletin of the American Meteorological Society. (19 April 2013; http://journals.ametsoc.org/doi/abs/10.1175/BAMS-D-12- 00170) doi:10.1175/BAMS-D-12-00170
10. Durre I, Menne MJ, Vose RS. 2008. Strategies for evaluating quality assurance procedures. Journal of Applied Meteorology and Climatology 47: 1785-1791.
11. Durre I, Menne MJ, Gleason BE, Houston TG, Vose RS. 2010. Comprehensive automated quality assurance of daily surface observations. Journal of Applied Meteorology and Climatology 49: 1615-1633.
12. Fiebrich CA, Crawford KC. 2001. The impact of unique meteorological phenomena detected by the Oklahoma Mesonet and ARS Micronet on automated quality control. Bulletin of the American Meteorological Society 82: 2173-2187. (Pubitemid 33617578)
13. Fiebrich CA, Grimsley DL, McPherson RA, Kesler KA, Essenberg GR. 2006. The value of routine site visits in managing and maintaining quality data from the Oklahoma Mesonet. Journal of Atmospheric and Oceanic Technology 23: 406-416.
14. Ganesan D, Cerpa A, Ye W, Yu Y, Zhao J, Estrin D. 2004. Networking issues in wireless sensor networks. Journal of Parallel and Distributed Computing 64: 799-814. (Pubitemid 40000771)
15. Glasgow HB, Burkholder JM, Reed RE, Lewitus AJ, Kleinman JE. 2004. Real-time remote monitoring of water quality: A review of current applications, and advancements in sensor, telemetry, and computing technologies. Journal of Experimental Marine Biology and Ecology 300: 409-448. (Pubitemid 38587566)
16. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH. 2009. The WEKA data mining software: An update. SIGKDD Explorations 11: 10-18.
17. Hamilton MP, Graham EA, Rundel PW, Allen MF, Kaiser W, Hansen MH, Estrin DL. 2007. New approaches in embedded networked sensing for terrestrial ecological observatories. Environmental Engineering Science 24: 192-204. (Pubitemid 46147644)
18. Harmel RD, King KW. 2005. Uncertainty in measured sediment and nutrient flux in runoff from small agricultural watersheds. Transactions of the ASAE 48: 1713-1721. (Pubitemid 41783619)
19. Hart JK, Martinez K. 2006. Environmental sensor networks: A revolution in the Earth system science? Earth-Science Reviews 78: 177-191. (Pubitemid 44356314)
20. Hefeeda M, Bagheri M. 2009. Forest fire modeling and early detection using wireless sensor networks. Ad Hoc and Sensor Wireless Networks 7: 169-224.
21. Hill DJ, Minsker BS. 2006. Automated fault detection for in-situ environmental sensors. In Gourbesville P, Cunge J, Guinot V, Liong SY, eds. Hydroinformatics 2006: Proceedings of the Seventh International Conference on Hydroinformatics. Research Publications. (7 May 2013; http://emsa.ncsa.illinois. edu/Documents/Conference/Hill-Minsker-HIC2006.pdf)
22. Honkavaara E, et al. 2009. Digital airborne photogrammetry: A new tool for quantitative remote sensing? A state-of-the-art review on radiometric aspects of digital photogrammetric images. Remote Sensing 1: 577-605.
23. Horsburgh JS, Jones AS, Stevens DK, Tarboton DG, Mesner NO. 2010. A sensor network for high frequency estimation of water quality constituent fluxes using surrogates. Environmental Modelling and Software 25: 1031-1044.
24. Hubbard KG, Guttman NB, You J, Chen Z. 2007. An improved QC process for temperature in the daily cooperative weather observations. Journal of Atmospheric and Oceanic Technology 24: 206-213. (Pubitemid 46380920)
25. Hubbard KG, You J. 2005. Sensitivity analysis of quality assurance using the spatial regression approach-A case study of the maximum/minimum air temperature. Journal of Atmospheric and Oceanic Technology 22: 1520-1530. (Pubitemid 41632298)
26. Kotamäki N, Thessler S, Koskiaho J, Hannukkala AO, Huitu H, Huttula T, Havento J, Järvenpää M. 2009. Wireless in-situ sensor network for agriculture and water monitoring on a river basin scale in southern Finland: Evaluation from a data user's perspective. Sensors 9: 2862-2883.
27. Lehrter JC, Cebrian J. 2010. Uncertainty propagation in an ecosystem nutrient budget. Ecological Applications 20: 508-524.
28. Lerner B, Boose E, Osterweil L, Ellison A, Clarke L. 2011. Provenance and quality control in sensor networks. Pages 98-103 in Jones MB, Gries C, eds. Proceedings of the Environmental Information Management Conference. University of California, Santa Barbara.
29. Liu Y, Minsker B, Hill D. 2007. Cyberinfrastructure technologies to support QA/QC and event-driven analysis of distributed sensing data. Paper presented at the International Workshop on Advances in Hydroinformatics; 4-7 June 2006, Niagara Falls, Canada.
30. Lynch C. 2008. Big data: How do your data grow? Nature 455: 28-29.
31. Moatar F, Miquel J, Poirel A. 2001. A quality-control method for physical and chemical monitoring data: Application to dissolved oxygen levels in the River Loire (France). Journal of Hydrology 252: 25-36. (Pubitemid 32812212)
32. Normander B, Haigh T, Christiansen JS, Jensen TS. 2008. Development and implementation of a near-real-time web reporting system on groundlevel ozone in Europe. Integrated Environmental Assessment and Management 4: 505-512.
33. Olden JD, Lawler JJ, Poff NL. 2008. Machine learning methods without tears: A primer for ecologists. The Quarterly Review of Biology 83: 171-193.
34. Peppler RA, et al. 2008. An overview of ARM Program Climate Research Facility data quality assurance. Open Atmospheric Science Journal 2: 192-216.
35. Porter J[H], et al. 2005. Wireless sensor networks for ecology. BioScience 55: 561-572.
36. Porter JH, Nagy E, Kratz TK, Hanson P[C], Collins SL, Arzberger P. 2009. New eyes on the world: Advanced sensors for ecology. BioScience 59: 385-397.
37. Porter JH, Hanson PC, Lin C-C. 2012. Staying afloat in the sensor data deluge. Trends in Ecology and Evolution 27: 121-129.
38. Richardson AD, Hollinger DY. 2007. A method to estimate the additional uncertainty in gap-filled NEE resulting from long gaps in the CO2 flux record. Agricultural and Forest Meteorology 147: 199-208. (Pubitemid 47615195)
39. Richardson AD, Braswell BH, Hollinger DY, Jenkins JP, Ollinger SV. 2009. Near-surface remote sensing of spatial and temporal variation in canopy phenology. Ecological Applications 19: 1417-1428.
40. Schimel D. 2011. The era of continental-scale ecology. Frontiers in Ecology and the Environment 9: 311.
41. Shafer MA, Fiebrich CA, Arndt DS, Fredrickson SE, Hughes TW. 2000. Quality assurance procedures in the Oklahoma Mesonetwork. Journal of Atmospheric and Oceanic Technology 17: 474-494. (Pubitemid 30341946)
42. Shuai Y, Schaaf CB, Strahler AH, Liu J, Jiao Z. 2008. Quality assessment of BRDF/albedo retrievals in MODIS operational system. Geophysical Research Letters 35 (art. L05407). doi:10.1029/2007GL032568 (Pubitemid 351676009)
43. Solomatine DP, Ostfeld A. 2008. Data-driven modelling: Some past experiences and new approaches. Journal of Hydroinformatics 10: 3-22.
44. Suri A, Iyengar SS, Cho E. 2006. Ecoinformatics using wireless sensor networks: An overview. Ecological Informatics 1: 287-293. (Pubitemid 44602586)
45. Szewczyk R, Osterweil E, Polastre J, Hamilton M, Mainwaring A, Estrin D. 2004. Habitat monitoring with sensor networks. Communications of the ACM 47: 34-40.
46. Young PC. 2002. Advances in real time flood forecasting. Philosophical Transactions of the Royal Society A 360: 1433-1450.